WO2014059810A1 - 人机交互方法及相关设备、系统 - Google Patents

人机交互方法及相关设备、系统 Download PDF

Info

Publication number
WO2014059810A1
WO2014059810A1 PCT/CN2013/080324 CN2013080324W WO2014059810A1 WO 2014059810 A1 WO2014059810 A1 WO 2014059810A1 CN 2013080324 W CN2013080324 W CN 2013080324W WO 2014059810 A1 WO2014059810 A1 WO 2014059810A1
Authority
WO
WIPO (PCT)
Prior art keywords
auxiliary light
light source
terminal device
camera
camera module
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Ceased
Application number
PCT/CN2013/080324
Other languages
English (en)
French (fr)
Inventor
方璡
杜健
陈妍
唐沐
金劲松
程俊
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Tencent Technology Shenzhen Co Ltd
Original Assignee
Tencent Technology Shenzhen Co Ltd
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Tencent Technology Shenzhen Co Ltd filed Critical Tencent Technology Shenzhen Co Ltd
Publication of WO2014059810A1 publication Critical patent/WO2014059810A1/zh
Priority to US14/677,883 priority Critical patent/US20150212727A1/en
Anticipated expiration legal-status Critical
Ceased legal-status Critical Current

Links

Classifications

    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
    • G06F3/01—Input arrangements or combined input and output arrangements for interaction between user and computer
    • G06F3/048—Interaction techniques based on graphical user interfaces [GUI]
    • G06F3/0487—Interaction techniques based on graphical user interfaces [GUI] using specific features provided by the input device, e.g. functions controlled by the rotation of a mouse with dual sensing arrangements, or of the nature of the input device, e.g. tap gestures based on pressure sensed by a digitiser
    • G06F3/0488—Interaction techniques based on graphical user interfaces [GUI] using specific features provided by the input device, e.g. functions controlled by the rotation of a mouse with dual sensing arrangements, or of the nature of the input device, e.g. tap gestures based on pressure sensed by a digitiser using a touch-screen or digitiser, e.g. input of commands through traced gestures
    • G06F3/04883—Interaction techniques based on graphical user interfaces [GUI] using specific features provided by the input device, e.g. functions controlled by the rotation of a mouse with dual sensing arrangements, or of the nature of the input device, e.g. tap gestures based on pressure sensed by a digitiser using a touch-screen or digitiser, e.g. input of commands through traced gestures for inputting data by handwriting, e.g. gesture or text
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
    • G06F3/01—Input arrangements or combined input and output arrangements for interaction between user and computer
    • G06F3/017—Gesture based interaction, e.g. based on a set of recognized hand gestures
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
    • G06F3/01—Input arrangements or combined input and output arrangements for interaction between user and computer
    • G06F3/03—Arrangements for converting the position or the displacement of a member into a coded form
    • G06F3/041—Digitisers, e.g. for touch screens or touch pads, characterised by the transducing means
    • G06F3/042—Digitisers, e.g. for touch screens or touch pads, characterised by the transducing means by opto-electronic means
    • G06F3/0425—Digitisers, e.g. for touch screens or touch pads, characterised by the transducing means by opto-electronic means using a single imaging device like a video camera for tracking the absolute position of a single or a plurality of objects with respect to an imaged reference surface, e.g. video camera imaging a display or a projection screen, a table or a wall surface, on which a computer generated image is displayed or projected
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F2203/00—Indexing scheme relating to G06F3/00 - G06F3/048
    • G06F2203/048—Indexing scheme relating to G06F3/048
    • G06F2203/04808—Several contacts: gestures triggering a specific function, e.g. scrolling, zooming, right-click, when the user establishes several contacts with the surface simultaneously; e.g. using several fingers or a combination of fingers and pen

Definitions

  • the invention relates to the technical field of human-computer interaction, in particular to a human-computer interaction method and related equipment and system. Background technique
  • Human-Computer Interaction Techniques usually refer to technologies that enable human-to-terminal device conversations in an efficient manner through input and output devices of terminal devices (such as computers, smartphones, etc.).
  • the terminal device provides a large amount of relevant information and prompting instructions to the person through the output or display device, and the person inputs the relevant operation instruction to the terminal device through the input device, and can control the terminal device to perform the corresponding operation.
  • Human-computer interaction technology is one of the important contents in the design of computer user interface. It is closely related to the fields of cognition, ergonomics and psychology.
  • gesture input is more and more popular because of its intuitive operation and high user experience.
  • gesture input is generally achieved by capturing and understanding gestures directly from a normal camera.
  • Practice has found that the normal camera captures and understands the poor anti-interference ability of the gesture, resulting in low control accuracy.
  • the ordinary camera directly captures and understands the poor anti-interference ability of the gesture, and the manipulation accuracy is low.
  • a human-computer interaction method and related device and system are provided, which can improve the anti-interference ability of a gesture input, thereby improving the accuracy of the manipulation.
  • the human-computer interaction method includes:
  • the terminal device captures an auxiliary light source formed after the gesture touches the auxiliary light curtain by using the camera module; the terminal device determines a position of the auxiliary light source in the image captured by the camera module and/ Or motion trajectory; and
  • the terminal device executes a corresponding operation instruction according to the position and/or motion trajectory.
  • a terminal device comprising:
  • a camera module that captures an auxiliary light source formed by the gesture after touching the auxiliary light curtain
  • Determining a module determining a position and/or a motion trajectory of the auxiliary light source in an image captured by the camera module
  • the execution module executes a corresponding operation instruction according to the position and/or the motion trajectory.
  • a human-computer interaction system includes an auxiliary light curtain, a camera, and a terminal device, the camera is built in the terminal device, or is wired Or connecting to the terminal device in a wireless manner, the shooting area of the camera covers a working coverage area of the auxiliary light curtain, wherein:
  • the auxiliary light curtain is adapted to form an auxiliary light source
  • the camera captures an auxiliary light source formed by the gesture after touching the auxiliary light curtain;
  • the terminal device includes:
  • a determining module configured to determine a position and/or a motion track of the auxiliary light source in the image captured by the camera
  • the terminal device can use the camera module to capture the auxiliary light source formed after the gesture touches the auxiliary light curtain, and determine the position and/or motion track of the auxiliary light source in the image captured by the camera module.
  • the code corresponding to the location and/or the motion track can be queried and the operation instruction corresponding to the code can be executed.
  • the invention is based on the auxiliary light source as the basis for human-computer interaction, which not only has very good anti-interference ability and higher control accuracy, but also has good commercial use value.
  • FIG. 1 is a flow chart of a human-computer interaction method according to an embodiment of the present invention
  • FIG. 2 is a flow chart of another human-computer interaction method according to another embodiment of the present invention
  • FIG. 4 is a schematic diagram of image processing of a camera according to an embodiment of the present invention
  • FIG. 5 is a schematic diagram of square image of a camera image according to an embodiment of the present invention
  • FIG. 6 is another embodiment of the present invention.
  • FIG. 7 is a schematic diagram of a motion trajectory in a square region of a camera image according to an embodiment of the present invention.
  • FIG. 8 is a structural diagram of a terminal device according to an embodiment of the present invention.
  • FIG. 9 is a structural diagram of a human-machine interaction system in accordance with one embodiment of the present invention. detailed description
  • the embodiment of the invention provides a human-computer interaction method and related device and system.
  • the terminal device uses the camera module to capture an auxiliary light source formed by the gesture touching the auxiliary light curtain, and determines the auxiliary light source in the camera module.
  • a position and/or a motion trajectory in the image is captured, and the terminal device then executes a corresponding operational command based on the position and/or motion trajectory.
  • the human-computer interaction method of the embodiment of the invention can improve the anti-interference ability of the gesture input, thereby improving the accuracy of the manipulation. The following is a detailed description.
  • FIG. 1 is a flow chart of a human-computer interaction method according to an embodiment of the present invention. As shown in FIG. 1, the human-computer interaction method in this embodiment starts from step 101.
  • step 101 the terminal device uses the camera module to capture the auxiliary light source formed by the gesture touching the auxiliary light curtain.
  • the terminal device implementing the human-computer interaction method may be a computer with a computing capability and having computing power, a smart phone, a television, various home smart devices, a commercial smart device, and office intelligence.
  • the device, the mobile Internet device (MID), and the like are not specifically limited in the embodiment of the present invention.
  • the camera module may be built in a terminal device, including but not limited to: a terminal device such as a notebook computer, a tablet computer, a smart phone, or a personal digital assistant (PDA), for example,
  • PDA personal digital assistant
  • the camera built into the terminal device with a camera, a smart phone, a tablet, a PDA, etc. can also be externally connected to the terminal device.
  • the camera module can be connected to the terminal device through a Universal Serial Bus (USB).
  • USB Universal Serial Bus
  • the camera module can be connected to the terminal device via a Wide Area Network (WAN), or the camera module can be connected to the terminal device via Bluetooth, Wi-Fi, infrared or other wireless means.
  • the camera may be built in the human-computer interaction terminal, externally connected to the human-machine interaction terminal, or a combination of two modes, between the camera and the human-computer interaction terminal.
  • the connection method can be: wired connection, wireless connection or a combination of two connection methods.
  • the terminal device may use the camera module to capture an image of the auxiliary light source formed after the gesture touches the auxiliary light curtain, and process the image to obtain an auxiliary formed only after the gesture touches the auxiliary light curtain. An image of the light source, thereby implementing the above step 101.
  • the camera module may be an infrared light camera.
  • the auxiliary light curtain may be an infrared light-assisted light curtain, and the auxiliary light source formed after the gesture touches the auxiliary light curtain is highlighted. Auxiliary light source.
  • the camera module may be a visible light camera.
  • the auxiliary light curtain may be a visible light auxiliary light curtain.
  • the auxiliary light source formed after the gesture touches the auxiliary light curtain is a dark auxiliary. light source.
  • the terminal device determines a position and/or a motion trajectory of the auxiliary light source in the image captured by the camera module.
  • the terminal device may determine that the auxiliary light source falls into a checkered area in the captured image of the camera module, and a square area in the captured image of the incoming camera module as a position of the auxiliary light source in the image captured by the camera module; and if the gesture is touched by a sliding method
  • the light curtain forms an auxiliary light source
  • the terminal device can determine the number and direction of the checkered areas in the image captured by the camera module through which the auxiliary light source passes, and the number of checkered areas in the image taken by the camera module through which the auxiliary light source passes The direction acts as an auxiliary source of light in the image captured by the camera module.
  • the image captured by the camera module may be uniformly divided into a plurality of checkered areas by the terminal device at a certain angle (such as the upper left corner).
  • step 103 the terminal device queries the code corresponding to the location and/or the motion track.
  • control software of the terminal device may query the location where the auxiliary light source falls from the stored mapping relationship between the stored checkered area and the code according to the auxiliary light source falling into the checkered area in the image captured by the camera module.
  • control software of the terminal device may be based on the number and direction of the checkered areas in the image captured by the camera module through which the auxiliary light source passes, from the number, direction, and encoding of the stored checkered areas.
  • mapping relationship the code corresponding to the number and direction of the checkered areas in the image captured by the camera module through which the auxiliary light source passes is queried.
  • step 104 the terminal device acquires an operation instruction corresponding to the code from the mapping relationship between the stored code and the operation instruction according to the queried code, and executes an operation instruction corresponding to the code.
  • the operation instruction may be a computer operation instruction (such as opening, closing, zooming in, zooming out, etc.) or a TV remote control instruction (such as power on, power off, volume up, volume down, Change the next channel, change to a channel, mute and other remote control operation commands).
  • a computer operation instruction such as opening, closing, zooming in, zooming out, etc.
  • a TV remote control instruction such as power on, power off, volume up, volume down, Change the next channel, change to a channel, mute and other remote control operation commands.
  • the auxiliary light curtain is overlapped or parallel to the display screen.
  • the auxiliary light curtain is an infrared light assisting light curtain and superimposed with a visible light curtain to indicate the position of the auxiliary light curtain.
  • the terminal device may use a camera module to capture an auxiliary light source formed after the gesture touches the auxiliary light curtain, and determine a position and/or a motion track of the auxiliary light source in the image captured by the camera module. Further, the code corresponding to the location and/or the motion track may be queried, and the code corresponding to the code is obtained from the mapping relationship between the stored code and the operation instruction according to the code. The instruction is operated and the operation instruction corresponding to the code is executed. Therefore, the human-computer interaction method described in FIG. 1 is based on the auxiliary light source as the basis for human-computer interaction, which not only has very good anti-interference ability and higher control accuracy, but also has good commercial use value.
  • FIG. 2 is a flow chart of another human-computer interaction method according to another embodiment of the present invention.
  • the human-computer interaction method described in Fig. 2 it is assumed that the gesture touches the auxiliary light curtain in a click manner to form an auxiliary light source.
  • the human-computer interaction method may include the following steps.
  • step 201 the terminal device uses the camera module to capture the auxiliary light source formed by clicking the auxiliary light curtain.
  • auxiliary light curtain can use a laser plus a word grating as the light source.
  • the light source can diffuse a single laser beam onto the screen by a grating effect of a word grating to realize the auxiliary light curtain.
  • the laser in FIG. 3 may be an infrared laser
  • the camera module may be an infrared camera module, and when the laser in FIG.
  • the auxiliary light curtain is an infrared laser
  • the auxiliary light curtain is an infrared
  • the light-assisted light curtain at this time, the auxiliary light source formed by the infrared light camera module and the auxiliary light source formed by the auxiliary light curtain is a high-light auxiliary light source.
  • the laser in FIG. 3 may also be a visible light laser
  • the camera module is correspondingly a visible light camera module
  • the auxiliary light curtain is a corresponding visible light assist.
  • Light curtain at this time, the auxiliary light source formed by the visible light camera module after clicking the auxiliary light curtain is a dark dark auxiliary light source.
  • the handset can also be used to illuminate the screen to implement an auxiliary light curtain, which is both singular and efficient, and low in cost.
  • the laser of Figure 3 is an infrared laser and the camera module is an infrared camera module.
  • the auxiliary light curtain is an infrared light assisted light curtain.
  • the auxiliary light source formed by the camera module captured by the auxiliary light curtain is a highlighted auxiliary light source.
  • the specific implementation of the foregoing step 201 may include:
  • the terminal device uses the camera module to capture an image of the auxiliary light source that is formed by the gesture-clicking auxiliary light curtain, and processes the image to obtain only the gesture to click on the auxiliary light curtain.
  • An image of a bright auxiliary light source is an image of a bright auxiliary light source.
  • FIG. 4 For a specific implementation process of processing the camera image to obtain an image that only forms a highlight auxiliary light source after the gesture is clicked on the auxiliary light curtain, reference may be made to FIG. 4.
  • A denotes an image of a secondary auxiliary light source (indicated by a circle) that is captured by a camera module under normal conditions, and a camera module is captured under low exposure conditions.
  • An image of a high-assist auxiliary light source (indicated by a circle) is formed after the gesture is clicked on the auxiliary light curtain.
  • the image captured by the camera module under low exposure conditions carries a hand-shaped, other illumination in addition to the highlighted auxiliary light source (indicated by a circle) formed by the gesture click-assisted light curtain.
  • C indicates an image in which B is removed from the background impurity
  • D indicates that after the background impurity is completely processed, only the image in which the gesture is clicked on the auxiliary light curtain to form a highlighted auxiliary light source (indicated by a circle) is displayed.
  • the manner and process of performing background impurity treatment on the image of the background impurity carried are all attempts known to those skilled in the art, and are not described in detail in this embodiment.
  • the terminal device determines the position of the auxiliary light source in the image captured by the camera module.
  • the terminal device may determine a checkered area in the captured image of the camera module into which the auxiliary light source falls. And the position of the checkered area in the image taken by the camera module in which the auxiliary light source falls in is used as the position of the auxiliary light source in the image captured by the camera module.
  • the terminal device may be connected to the camera module through a camera interface, and the terminal device may use an angle (such as the upper left corner) of the image captured by the camera module as an origin,
  • the image captured by the camera module is evenly divided into a plurality of checkered areas.
  • the terminal device can drop the auxiliary light source into the 16th image captured by the camera module. The position of the checkered area as an auxiliary light source (for the circle representation) in the image taken by the camera module.
  • step 203 the terminal device queries the code corresponding to the location.
  • control software of the terminal device may query the auxiliary light source from the mapping relationship between the square area of the coded library and the code according to the checker area in which the auxiliary light source falls into the image captured by the camera module.
  • the mapping relationship between the square area of the coded library and the code is as shown in the table.
  • Table 1 indicates that the terminal device uses the upper left corner of the image captured by the camera module as the origin, and uniformly divides the image captured by the camera module into nine checkered regions. Those skilled in the art should understand that Table 1 is only an embodiment, and the user can also divide the images captured by the camera module into more square regions according to his own preferences, and customize more codes, thereby It can enrich the operation of the terminal device.
  • the code corresponding to the checkered area in which the auxiliary light source falls into the image captured by the camera module in the map area of the coded library and the coded relationship is I.
  • step 204 the terminal device acquires an operation instruction corresponding to the code from the mapping relationship between the code of the coded instruction map store and the operation instruction according to the queried code, and executes the operation instruction corresponding to the code.
  • mapping relationship between the coded area of the coded library and the code shown in Table 1 is combined, and the mapping relationship between the code stored in the coded instruction map library and the operation instruction is as shown in Table 2.
  • Table 3 shows: the code of the coded instruction map library shown in Table 2 is displayed on the nine checkered areas formed by the camera module in Table 1 being uniformly divided. The mapping relationship of the operation instructions.
  • the terminal device can utilize the camera module to capture gesture clicks Auxiliary light source formed after the auxiliary light curtain, and determining the position of the auxiliary light source in the image captured by the camera module, and then querying the code corresponding to the position, and obtaining the code from the mapping relationship between the stored code and the operation instruction according to the code Corresponding operation instructions, and executing the operation instructions corresponding to the code. Therefore, the human-computer interaction method described in FIG. 2 is based on the auxiliary light source as the basis for human-computer interaction, which not only has very good anti-interference ability and higher control accuracy, but also has high commercial use value.
  • the human-computer interaction method according to another embodiment of the present invention has been described in detail above. According to another embodiment of the present invention, another human-computer interaction method is also provided.
  • FIG. 6 is a flowchart of a human-computer interaction method according to another embodiment of the present invention.
  • the human-computer interaction method described in Fig. 6 an example in which the auxiliary light source is touched by the gesture to slide the auxiliary light curtain is described.
  • the human-computer interaction method includes at least the following steps.
  • step 601 the terminal device captures the auxiliary light source formed by the gesture sliding auxiliary light curtain by using the camera module.
  • auxiliary light curtain In one embodiment of the present invention, the specific implementation of the auxiliary light curtain has been described in detail in the previous embodiments, and this embodiment is not described.
  • the terminal device may use the camera module to capture an image of the auxiliary light source that is formed by the gesture sliding auxiliary light curtain, and process the image to obtain only the gesture to display the auxiliary light curtain. An image of the highlighted auxiliary light source.
  • the terminal device determines a motion trajectory of the auxiliary light source in the image captured by the camera module.
  • the terminal device can continuously identify the image sequence of the highlighted auxiliary light source formed only after the gesture sliding auxiliary light curtain is displayed by the control software, so that the auxiliary light source can be determined in the image captured by the camera module. Movement track.
  • the terminal device can determine the number and direction of the checkered area in the image captured by the camera module through which the auxiliary light source passes. And in the image taken by the camera module through which the auxiliary light source passes The number and direction of the checkered areas act as an auxiliary source of motion in the image captured by the camera module.
  • the terminal device queries the code corresponding to the motion track.
  • the mapping relationship between the number, direction, and coding of the square regions in the captured image of the camera module through which the auxiliary light source passes as shown in Table 4 may be pre-stored in the encoding library of the terminal device. Referring to Table 4 attached below, it is assumed that the image captured by the camera module is evenly divided into a plurality of checkered areas as shown in FIG. 7, and when the auxiliary light source passes down three square areas, the motion track corresponds to Table 4. The code a shown. When the auxiliary light source passes through the three squares to the right, the motion trajectory corresponds to the code b shown in Table 4. When the auxiliary light source obliquely passes up through three squares, the motion track corresponds to the code Co shown in Table 4.
  • Table 4 shows the number of squares in the captured image of the camera module, the direction and the mapping relationship between the three images.
  • the device can use the control software to query the auxiliary light source to pass downwards from the mapping relationship between the number, direction and coding of the square area in the image captured by the camera module of the code library stored in the auxiliary light source shown in Table 4. The corresponding code a when the square area is used.
  • step 604 the terminal device acquires an operation instruction corresponding to the code from the mapping relationship between the code of the coded instruction map store and the operation instruction according to the queried code, and executes the operation instruction corresponding to the code.
  • the coded instruction map is stored in Table 5. The mapping relationship between the code and the operation instruction.
  • auxiliary light source passes down the 3 squares, scroll down the content b. Enlarge the screen.
  • auxiliary light source passes diagonally up through 3 squares, zoom in on the screen c.
  • the control software of the terminal device may further obtain the operation instruction from the table 5 as "scrolling down the content". At this time, the terminal device knows that the operation instruction is executed, and the content is scrolled down.
  • the terminal device may use the camera module to capture the auxiliary light source formed after the gesture sliding auxiliary light curtain, and determine the motion track of the auxiliary light source in the image captured by the camera module, and then Querying the code corresponding to the motion track, and acquiring an operation instruction corresponding to the code from the stored mapping relationship between the code and the operation instruction according to the code, and executing an operation instruction corresponding to the code. Therefore, the human-computer interaction method described in FIG. 6 is based on the auxiliary light source as the human-computer interaction, which not only has better anti-interference ability and higher control accuracy, but also has high commercial use value.
  • a terminal device is also provided.
  • FIG. 8 is a structural diagram of a terminal device according to another embodiment of the present invention.
  • the terminal device may be a computer, a smart phone, a television, and various home smart devices, a commercial smart device, an office smart device, an MID, etc., which are loaded with control software, and are not specifically limited in the embodiment of the present invention.
  • the terminal device includes: a camera module 801, a determining module 802, and an executing module 803.
  • the camera module 801 captures an auxiliary light source formed by the gesture touching the auxiliary light curtain.
  • the determining module 802 determines a position and/or a motion trajectory of the auxiliary light source in the image captured by the camera module 801.
  • the execution module 803 executes a corresponding operation instruction according to the position and/or the motion track.
  • the camera module 801 captures an image of an auxiliary light source formed after the gesture touches the auxiliary light curtain, and processes the image to obtain an auxiliary light source formed only after the gesture touches the auxiliary light curtain. image.
  • the determining module 802 determines that the auxiliary light source falls into a checkered area in the image captured by the camera module 801; and/or determines that the camera module 801 passes by the auxiliary light source The number and direction of the checkered areas in the image.
  • the camera module The image taken by 801 (for example, the origin in the upper left corner) is evenly divided into a plurality of checkered areas.
  • the execution module 803 includes: a query sub-module 8031 and an acquisition sub-module 8032.
  • the query sub-module 8031 queries the code corresponding to the location and/or the motion track.
  • the obtaining sub-module 8032 obtains an operation instruction corresponding to the code from the mapping relationship between the stored code and the operation instruction according to the code, and executes an operation instruction corresponding to the code.
  • the query sub-module 8031 queries the auxiliary light source from the stored mapping relationship between the stored checkered area and the code according to the auxiliary light source falling into the square area in the captured image of the camera module 801.
  • the camera module 801 captures a code corresponding to a checkered area in the image.
  • the query sub-module 8031 may further query the side from the mapping relationship between the number of stored checkered areas, the direction, and the code according to the number and direction of the checkered areas in the image captured by the camera module 801. The code corresponding to the number and direction of the grid area.
  • the operation instruction may be any one of a computer operation instruction or a television remote control instruction, which is not limited in this embodiment.
  • the human-computer interaction method according to FIG. 1 may be performed by FIG. 8.
  • Step 101 can be executed by the camera module 801 shown in FIG.
  • Step 102 shown in Figure 1 can be performed by the validation module 802 shown in Figure 8.
  • the step 103 shown in Fig. 1 can be performed by the query sub-module 8031 of the execution module 803 shown in Fig. 8.
  • the step 104 shown in Fig. 1 can be performed by the acquisition submodule 8032 of the execution module 803 shown in Fig. 8.
  • each unit in the terminal device shown in FIG. 8 may be separately or completely combined into one or several additional units, or one of the units may be further split. This is constructed for a plurality of units that are functionally smaller, which can achieve the same operation without affecting the implementation of the technical effects of the embodiments of the present invention.
  • the above units are divided based on logical functions. In practical applications, the functions of one unit can also be implemented by multiple units, or the functions of multiple units can be implemented by one unit. In other embodiments of the invention, the terminal device may also include other modules. However, in practical applications, these functions can also be implemented by other units and can be implemented by multiple units.
  • processing elements and storage media including, for example, a central processing unit (CPU), a random access memory (RAM), a read only memory (ROM), and the like
  • a computer program (including program code) capable of executing the human-computer interaction method as shown in FIG. 1 is executed on a general-purpose computing device of a computer to construct a terminal device as shown in FIG. 8, and to implement an embodiment according to the present invention Human-computer interaction method.
  • the computer program can be recorded, for example, on a computer readable recording medium, and loaded in the above computing device through a computer readable recording medium and operated therein.
  • the terminal device shown in FIG. 8 can capture the auxiliary light source formed by the camera module after touching the auxiliary light curtain, and determine the position and/or motion track of the auxiliary light source in the image captured by the camera module, and then the position and/or motion can be queried.
  • the code corresponding to the track, and the operation instruction corresponding to the code is executed. Therefore, the terminal device and the person shown in FIG. 8 can use the auxiliary light source as the basis for human-computer interaction, which not only has very good anti-interference ability and higher control accuracy, but also has good commercial use value. .
  • the terminal device has been described in detail above. According to another embodiment of the present invention, a human-computer interaction system is also provided.
  • FIG. 9 is a structural diagram of a human-machine interaction system according to an embodiment of the present invention.
  • the human-machine interaction system may include an auxiliary light curtain 901, a camera 902, and a terminal device 903.
  • the camera 902 may be built in the terminal device 903 or connected to the terminal device 903 by wire or wirelessly.
  • the shooting area of the camera 902 covers the working coverage area of the auxiliary light curtain 901.
  • the camera 902 is connected to the terminal device 903 by wire as an example.
  • the auxiliary light curtain 901 is adapted to form an auxiliary light source.
  • the camera 902 captures an auxiliary light source formed by the gesture touching the auxiliary light curtain 901.
  • the terminal device 903 includes: a determining module 9031 and an executing module 9032.
  • the determining module 9031 determines a position and/or a motion track of the auxiliary light source in the image captured by the camera 902;
  • the execution module 9032 executes a corresponding operation instruction according to the position and/or the motion trajectory.
  • the camera 902 specifically captures an image of the auxiliary light source formed by the gesture touch with the auxiliary light curtain, and processes the image to obtain an auxiliary light source formed only after the gesture touches the auxiliary light curtain. Image.
  • the determining module 9031 of the terminal device 903 is specifically The auxiliary light source falls into a checkered area in the image captured by the camera 902; and/or determines the number and direction of the checkered areas in the captured image of the camera 902 through which the auxiliary light source passes.
  • the captured image of the camera 902 is evenly divided into a plurality of checkered areas.
  • the execution module 9032 of the terminal device 903 includes: a query submodule 90321 and an acquisition submodule 90322.
  • the query sub-module 90321 queries the code corresponding to the location and/or the motion track.
  • the obtaining sub-module 90322 obtains an operation instruction corresponding to the code from the mapping relationship between the stored code and the operation instruction according to the code, and executes an operation instruction corresponding to the code.
  • the query sub-module 90321 queries the auxiliary light source from the stored mapping relationship between the stored checkered area and the code according to the square area in the image captured by the camera 902.
  • the query sub-module 90321 may also query the square of the captured image from the number of stored checkered areas, the direction, and the mapping relationship between the three types of checkered regions according to the number and direction of the checkered areas in the image captured by the camera 902.
  • the code corresponding to the number and direction of the grid area.
  • the operation instruction may be a computer operation instruction or a television remote control instruction, which is not limited in this embodiment.
  • the camera 902 may be an infrared camera, and accordingly the auxiliary light curtain 901 may be an infrared light-assisted light curtain.
  • the camera 902 can also be a visible light camera, and accordingly the auxiliary light curtain 901 can be a visible light assisted light curtain.
  • the human-computer interaction method according to FIG. 1 may be a human-computer interaction method executed by each unit in the human-computer interaction system shown in FIG.
  • the step 101 shown in Fig. 1 can be performed by the camera 902 shown in Fig. 9.
  • the step 102 shown in Fig. 1 can be performed by the confirmation module 9031 shown in Fig. 9.
  • Step 103 shown in Figure 1 can be performed by query sub-module 90321 of execution module 9032 shown in Figure 9.
  • the step 104 shown in FIG. 1 can be performed by the acquisition sub-module 90322 of the execution module 803 shown in FIG.
  • each unit in the human-computer interaction system shown in FIG. 9 may be separately or completely combined into one or several other units, or one of the units may be further Splitting into a plurality of functionally smaller units is constructed, which can achieve the same operation without affecting the realization of the technical effects of the embodiments of the present invention.
  • the above units are divided based on logical functions. In practical applications, the functions of one unit can also be implemented by multiple units, or the functions of multiple units. Can be implemented by one unit. In other embodiments of the invention, the human-computer interaction system may also include other modules. However, in practical applications, these functions can also be implemented by other units, and can be implemented by multiple units.
  • a computer program capable of executing the human-computer interaction method as shown in FIG. 1 can be executed by running on a general-purpose computing device such as a computer including a processing element and a storage medium.
  • a human-computer interaction system as shown in FIG. 9 is constructed, and a human-computer interaction method according to an embodiment of the present invention is implemented.
  • the computer program can be recorded, for example, on a computer readable recording medium, and loaded in and executed in the above computing device by a computer readable recording medium.
  • the storage medium may include: a flash disk, a read only memory (Read-Only Memory,
  • ROM Read Only Memory
  • RAM Random Access Memory
  • CD Compact Disc
  • the terminal device can use the camera to capture the auxiliary light source formed by the gesture after touching the auxiliary light curtain, and determine the position and/or motion trajectory of the auxiliary light source in the image captured by the camera, and then can be queried.
  • the code corresponding to the position and/or the motion track, and the operation instruction corresponding to the code is obtained from the stored mapping relationship between the code and the operation instruction, and the operation instruction corresponding to the code is executed. Therefore, the human-computer interaction system shown in Figure 9 is the basis for human-computer interaction through the auxiliary light source, which not only has better anti-interference ability and higher control accuracy, but also has high commercial use value.
  • the auxiliary light curtain When the auxiliary light curtain is arranged on the desktop, the auxiliary light curtain needs to be arranged parallel to the desktop and at a certain distance, otherwise a light mark will be formed to affect the recognition.
  • the auxiliary light curtain can be arranged on the wall surface, the table top, or arranged on the air façade, so that the user can touch the auxiliary light curtain in the air to realize the human-machine interaction operation.
  • the auxiliary light curtain can adopt a visible light and infrared light double light curtain manner, so that when the gesture touches the auxiliary light curtain, the finger is illuminated by visible light, the human eye obtains feedback, and the camera captures the infrared light assisted light curtain and the finger. Spot (ie auxiliary light source).

Landscapes

  • Engineering & Computer Science (AREA)
  • General Engineering & Computer Science (AREA)
  • Theoretical Computer Science (AREA)
  • Human Computer Interaction (AREA)
  • Physics & Mathematics (AREA)
  • General Physics & Mathematics (AREA)
  • Multimedia (AREA)
  • User Interface Of Digital Computer (AREA)

Abstract

本发明涉及人机交互技术领域，公开了一种人机交互方法及相关设备、系统。其中，该方法包括：终端设备利用摄像头模块捕获手势触及辅助光幕后形成的辅助光源；所述终端设备确定所述辅助光源在所述摄像头模块拍摄图像中的位置和/或运动轨迹；所述终端设备根据所述位置和/或运动轨迹执行相对应的操作指令。实施本发明可以提高手势输入的抗干扰能力，从而提升操控准确率。

Description

人机交互方法及相关 i殳备、 系统 本专利申请要求 2012 年 10 月 15 日提交的中国专利申请号为 201210388925.9, 发明名称为 "一种人机交互方法及相关设备、 系统" 的优先 权， 该申请的全文以引用的方式并入本申请中。 技术领域
本发明涉及人机交互技术领域， 特别涉及一种人机交互方法及相关设备、 系统。 背景技术
人机交互技术 ( Human- Computer Interaction Techniques )通常是指通过终 端设备（如计算机、 智能手机等）的输入、 输出设备， 以有效的方式实现人与 终端设备对话的技术。其包括终端设备通过输出或显示设备给人提供大量有关 信息及提示请示等，人通过输入设备给终端设备输入有关操作指令， 可以控制 终端设备执行相应的操作。人机交互技术是计算机用户界面设计中的重要内容 之一， 它与认知学、 人机工程学、 心理学等学科领域有密切的联系。
人机交互技术已经由最初的键盘输入、 鼠标输入逐渐演进为触摸屏输入、 手势输入。 其中， 手势输入由于具有操控直观、 用户体验高等优点， 正越来越 受到人们的青睐。 而在实际应用中， 手势输入一般都是通过普通摄像头直接捕 捉和理解手势来实现的。 实践发现， 采用普通摄像头直接捕捉和理解手势的抗 干扰能力差， 从而导致操控准确率低。 发明内容
在现有技术中， 采用普通摄像头直接捕捉和理解手势的抗干扰能力差， 而 且会导致操控准确率较低。
有鉴于此， 根据本发明的一个方面， 提供一种人机交互方法及相关设备、 系统， 能够提高手势输入的抗干扰能力， 从而提升操控的准确率。
所述人机交互方法， 包括：
终端设备利用摄像头模块捕获手势触及辅助光幕后形成的辅助光源； 所述终端设备确定所述辅助光源在所述摄像头模块拍摄图像中的位置和 / 或运动轨迹； 及
所述终端设备根据所述位置和 /或运动轨迹执行相对应的操作指令。
相应地， 根据本发明的另一方面， 还提供的一种终端设备， 该终端设备包 括：
摄像头模块， 捕获手势触及辅助光幕后形成的辅助光源；
确定模块， 确定所述辅助光源在所述摄像头模块拍摄图像中的位置和 /或 运动轨迹；
执行模块， 根据所述位置和 /或运动轨迹执行相对应的操作指令。
相应地， 根据本发明的另一方面， 还提供一种人机交互系统， 所述人机交 互系统包括辅助光幕、摄像头以及终端设备， 所述摄像头内置在所述终端设备 中， 或以有线或无线方式与所述终端设备连接， 所述摄像头的拍摄区域覆盖所 述辅助光幕的工作覆盖区域， 其中：
所述辅助光幕， 供手势触及以形成辅助光源；
所述摄像头， 捕获手势触及所述辅助光幕后形成的辅助光源；
所述终端设备包括：
确定模块， 用于确定所述辅助光源在所述摄像头拍摄图像中的位置和 /或 运动轨迹； 及
执行模块， 用于根据所述位置和 /或运动轨迹执行相对应的操作指令。 由上面的技术方案可知，在本发明的上述方面中， 终端设备可以利用摄像 头模块来捕获手势触及辅助光幕后形成的辅助光源，并且确定辅助光源在摄像 头模块拍摄图像中的位置和 /或运动轨迹， 进而可以查询该位置和 /或运动轨迹 对应的编码并执行该编码对应的操作指令。可见， 本发明是通过辅助光源来作 为人机交互的依据的，这不仅具有非常良好的抗干扰能力以及更高的操控准确 率， 还具有很好的商业使用价值。 附图说明
为了更清楚地说明本发明实施例或现有技术中的技术方案，下面将对实施 例或现有技术描述中所需要使用的附图作筒单地介绍，显而易见地， 下面描述 中的附图仅仅是本发明的一些实施例，对于本领域普通技术人员来讲，在不付 出创造性劳动的前提下， 还可以根据这些附图获得其他的附图。 图 1是根据本发明的一个实施例的一种人机交互方法的流程图； 图 2是根据本发明的另一个实施例的另一种人机交互方法的流程图； 图 3是根据本发明的一个实施例的一种辅助光幕的实现示意图；
图 4是根据本发明的一个实施例的一种摄像头图像处理的示意图； 图 5是根据本发明的一个实施例的一种摄像头图像方格化的示意图； 图 6是根据本发明的另一个实施例的另一种人机交互方法的流程图； 图 7是根据本发明的一个实施例的一种摄像头图像方格区域中的运动轨 迹的示意图；
图 8是根据本发明的一个实施例的一种终端设备的结构图；
图 9是根据本发明的一个实施例的一种人机交互系统的结构图。 具体实施方式
下面将结合本发明实施例中的附图详细描述对本发明的各个实施例， 显 然， 所描述的实施例仅仅是本发明一部分实施例， 而不是全部的实施例。 基于 本发明中的实施例，本领域普通技术人员在没有作出创造性劳动前提下所获得 的所有其他实施例， 都属于本发明保护的范围。
本发明实施例提供了一种人机交互方法及相关设备、 系统， 在该人机交互 方法中， 终端设备利用摄像头模块捕获手势触及辅助光幕后形成的辅助光源 , 并确定该辅助光源在摄像头模块拍摄图像中的位置和 /或运动轨迹， 进而终端 设备根据该位置和 /或运动轨迹执行相对应的操作指令。 本发明实施例的人机 交互方法能够提高手势输入的抗干扰能力，从而提升操控的准确率。 以下分别 进行详细说明。
请参阅图 1 , 图 1是根据本发明的一个实施例的一种人机交互方法的流程 图。 如图 1所示， 本实施例中的人机交互方法从步骤 101开始。
在步骤 101、 终端设备利用摄像头模块捕获手势触及辅助光幕后形成的辅 助光源。
实现本发明的一个实施例中，实施人机交互方法的终端设备可以是装载有 控制软件的， 并且具有计算能力的电脑、 智能手机、 电视机以及各种家庭智能 设备、商用智能设备、办公智能设备、移动互联网设备（ Mobile Internet Devices, MID )等， 本发明实施例不作具体限定。 本实施例中， 所述摄像头模块可以内置在终端设备中，所述终端设备包括 但不局限于：笔记本电脑、平板电脑、智能手机、个人数字助理（Personal Digital Assistant, PDA )等终端设备， 例如内置于带摄像头的电脑、 智能手机、 平板 电脑、 PDA等终端设备的摄像头， 也可以外接于终端设备部署， 例如， 摄像 头模块可以通过通用串行总线（ Universal Serial BUS , USB )与终端设备连接， 或者摄像头模块也可以通过远程网 （ Wide Area Network, WAN )与终端设备 连接， 或者摄像头模块也可以通过蓝牙、 Wi-Fi、 红外线等无线方式与终端设 备连接。 在本发明的一个实施例中， 所述摄像头可内置于所述人机交互终端、 外接于所述人机交互终端或两种方式的组合，所述摄像头与所述人机交互终端 之间的连接方式可为： 有线连接、 无线连接或两种连接方式的组合。
在本发明的一个实施例中，终端设备可以利用摄像头模块来捕获携带有手 势触及辅助光幕后形成的辅助光源的图像， 并对该图像进行处理， 以获得仅显 示手势触及辅助光幕后形成的辅助光源的图像， 从而实现上述步骤 101。
其中，对利用摄像头模块对图像进行处理以获得仅显示手势触及辅助光幕 后形成的辅助光源的图像的具体实现过程，本发明实施例后续将举例进行详细 说明， 此处先不作细述。
在本发明的一个实施例中，所述摄像头模块可以是红外光摄像头，相应地， 所述辅助光幕可以为红外光辅助光幕，此时手势触及辅助光幕后形成的辅助光 源为高亮的辅助光源。
在本发明的另一个实施例中， 所述摄像头模块可以是可见光摄像头，相应 地， 所述辅助光幕可以为可见光辅助光幕， 此时手势触及辅助光幕后形成的辅 助光源为深暗的辅助光源。
其中， 关于辅助光幕的具体实现方式， 本发明实施例后续将举例进行详细 说明， 此处先不作细述。
在步骤 102、 终端设备确定辅助光源在摄像头模块拍摄图像中的位置和 / 或运动轨迹。
在本发明的一个实施例中，如果手势是以点击的方式来触及辅助光幕以形 成辅助光源，那么终端设备可以确定辅助光源落入摄像头模块拍摄图像中的方 格区域，并以辅助光源落入的摄像头模块拍摄图像中的方格区域作为所述辅助 光源在摄像头模块拍摄图像中的位置；而如果手势是以滑动的方式来触及辅助 光幕以形成辅助光源，那么终端设备可以确定辅助光源所经过的摄像头模块所 拍摄图像中的方格区域数量和方向，并以辅助光源所经过的摄像头模块所拍摄 图像中的方格区域数量和方向作为辅助光源在摄像头模块拍摄图像中的运动 轨迹。
本实施例中， 所述摄像头模块拍摄图像可为以某一个角（如左上角）为原 点被终端设备均匀地划分成多个方格区域。
在步骤 103、 终端设备查询上述位置和 /或运动轨迹对应的编码。
在本发明的一个实施例中，终端设备的控制软件可以根据辅助光源落入摄 像头模块所拍摄图像中的方格区域，从存储的方格区域与编码的映射关系中查 询辅助光源落入的所述摄像头模块拍摄图像中的方格区域对应的编码。
在本发明的另一个实施方式中，终端设备的控制软件可以根据辅助光源所 经过的摄像头模块所拍摄图像中的方格区域数量和方向，从存储的方格区域数 量、方向以及编码三者的映射关系中查询辅助光源所经过的摄像头模块所拍摄 图像中的方格区域数量和方向对应的编码。
其中， 关于方格区域与编码的映射关系， 以及方格区域数量、 方向以及编 码三者的映射关系，本发明实施例后续将举例进行详细说明，此处先不作细述。
在步骤 104、 终端设备根据查询到的编码， 从存储的编码与操作指令的映 射关系中获取该编码对应的操作指令， 并执行编码对应的操作指令。
其中， 关于编码与操作指令的映射关系， 本发明实施例后续将举例进行详 细说明， 此处先不作细述。
在本发明的一个实施例中， 所述操作指令可以是电脑操作指令（如打开、 关闭、 放大、 缩小等鼠标操作指令）或电视机遥控指令（如开机、 关机、 放大 音量、 减小音量、 换下一频道、 换上一频道、 静音等遥控器操作指令） 。
在本发明的一个实施例中， 所述辅助光幕与显示屏幕重叠或平行。所述辅 助光幕与显示屏幕平行时，所述辅助光幕为红外光辅助光幕并叠加一个可见光 光幕， 用以指示所述辅助光幕的位置。
在图 1所描述的人机交互方法中，所述终端设备可以利用摄像头模块来捕 获手势触及辅助光幕后形成的辅助光源，并且确定辅助光源在摄像头模块拍摄 图像中的位置和 /或运动轨迹， 进而可以查询该位置和 /或运动轨迹所对应的编 码，以及根据该编码从存储的编码与操作指令的映射关系中获取该编码对应的 操作指令并执行该编码对应的操作指令。 因此， 图 1所述的人机交互方法是通 过辅助光源来作为人机交互的依据的，这不仅具有非常良好的抗干扰能力以及 更高的操控准确率， 还具有很好的商业使用价值。
以上对根据本发明的一个实施例的人机交互方法进行了详细说明。 根据本发明的另一实施例， 还提供了一种人机交互方法。
请参阅图 2, 图 2是根据本发明的另一实施例的另一种人机交互方法的流 程图。在图 2所述的人机交互方法中，假设手势是以点击的方式来触及辅助光 幕以形成辅助光源的。 如图 2所示， 该人机交互方法可以包括以下步骤。
在步骤 201、 终端设备利用摄像头模块捕获手势点击辅助光幕后形成的辅 助光源。
在本发明的一个实施例中，辅助光幕的具体实现方式可以参照图 3 ,其中， 该辅助光幕可以采用激光器加一字光栅来作为光源。该光源通过一字光栅的光 栅效应可以将单束激光扩射到屏幕上， 从而实现所述辅助光幕。 进一步地， 为 了获得更可靠稳定的效果， 图 3中的激光器可以是红外激光器， 而摄像头模块 可以是红外光摄像头模块， 而当图 3中的激光器是红外激光器时， 则辅助光幕 即是红外光辅助光幕，此时红外光摄像头模块捕获到的手势点击辅助光幕后形 成的辅助光源即是高亮的辅助光源。在另一个实施例中， 图 3中的激光器也可 以是可见光激光器， 而摄像头模块也相应的是可见光摄像头模块， 而当图 3 中的激光器是可见光激光器时， 则辅助光幕为相应的可见光辅助光幕， 此时， 该可见光摄像头模块捕获到的手势点击辅助光幕后形成的辅助光源即是深暗 的辅助光源。
在本发明的一个实施例中， 手机也可以用来照亮屏幕以实现辅助光幕， 这 种方式既筒单又有效， 并且成本也低。
在本发明的一个实施例中，假设图 3中的激光器是一个红外激光器，摄像 头模块是一个红外光摄像头模块， 相应地， 该辅助光幕是红外光辅助光幕。 此 时，所述摄像头模块捕获到的手势点击辅助光幕后形成的辅助光源就是高亮的 辅助光源。 那么， 相应地， 上述步骤 201的具体实现可以包括：
终端设备利用摄像头模块来捕获携带有手势点击辅助光幕后形成高亮的 辅助光源的图像， 并对该图像进行处理， 以获得仅显示手势点击辅助光幕后形 成高亮的辅助光源的图像。
其中，对摄像头图像进行处理以获得仅显示手势点击辅助光幕后形成高亮 的辅助光源的图像的具体实现过程可以参考图 4。 在图 4中， A表示摄像头模 块在正常情况下捕获到的携带有手势点击辅助光幕后形成高亮的辅助光源（用 圆圈表示） 的图像， 而 B表示摄像头模块在低曝光条件下捕获到的携带有手 势点击辅助光幕后形成高亮的辅助光源 （用圆圈表示） 的图像。 从 B 中可以 看出，摄像头模块在低曝光条件下捕获到的图像除了携带有手势点击辅助光幕 后形成的高亮的辅助光源（用圆圈表示）之外， 还会携带有手型、 其他照明灯 光等背景杂质，这些背景杂质的存在会降低操控准确率。 C表示是对 B进行去 背景杂质处理的图像， D表示背景杂质彻底处理完毕后，仅显示手势点击辅助 光幕后形成高亮的辅助光源（用圆圈表示）的图像。 其中， 对携带的背景杂质 的图像进行去背景杂质处理的方式、过程均是本领域普通技术人员所公知的尝 试， 本实施例不作详细介绍。
在步骤 202、 终端设备确定辅助光源在摄像头模块所拍摄图像中的位置。 在本发明的一个实施例中，由于手势是以点击的方式来触及辅助光幕而形 成辅助光源的， 因此， 所述终端设备可以确定辅助光源落入的摄像头模块拍摄 图像中的方格区域，并以辅助光源落入的摄像头模块拍摄图像中的方格区域作 为辅助光源在摄像头模块所拍摄图像中的位置。
在本发明的一个实施例中，如图 5所示， 终端设备可以通过摄像头接口与 摄像头模块连接，终端设备可以以所述摄像头模块所拍摄图像的某一个角（如 左上角）为原点， 将该摄像头模块所拍摄图像均匀地划分成多个方格区域。 如 图 5所示， 假设辅助光源（用于圆圈表示）落入摄像头模块所拍摄图像中的第 16 个方格区域， 那么终端设备可以将辅助光源落入摄像头模块所拍摄图像中 的第 16个方格区域作为辅助光源 （用于圆圈表示）在摄像头模块所拍摄图像 中的位置。
在步骤 203、 终端设备查询上述位置对应的编码。
在本发明的一个实施例中，终端设备的控制软件可以根据辅助光源落入所 述摄像头模块所拍摄图像中的方格区域，从编码库存储的方格区域与编码的映 射关系中查询辅助光源落入的摄像头模块所拍摄图像中的方格区域对应的编 码。 在本发明的一个实施例中，编码库存储的方格区域与编码的映射关系如表
1所示。
表 1 编码库存储的方格区域与编码的映射关系
Figure imgf000010_0001
在本发明的一个实施例中，表 1表示终端设备以摄像头模块所拍摄图像的 左上角为原点，将摄像头模块所拍摄图像均匀地划分成 9个方格区域。本领域 技术人员应当理解，表 1仅仅是一个实施例而已， 用户也可以根据自己的偏好 将摄像头模块所拍摄图像均勾地划分成更多的方格区域， 并自定义更多的编 码， 从而可以丰富对终端设备的操作。
举例来说，假设辅助光源落入摄像头模块所拍摄图像中的方格区域的方格 区域参数为 "左边界 =0, 右边界 =图像宽 /3 , 上边界 =0, 下边界 =图像高 /3" , 那么，终端设备的控制软件可以根据辅助光源落入摄像头模块拍摄图像中的方 格区域（用 "左边界 =0, 右边界 =图像宽 /3 , 上边界 =0, 下边界 =图像高 /3" 表 示）， 可从表 1所示的编码库存储的方格区域与编码的映射关系中查询到辅助 光源落入所述摄像头模块拍摄图像中的方格区域对应的编码为 A。
在本发明的一个实施例中，假设辅助光源落入摄像头模块所拍摄图像中的 方格区域的方格区域参数为 "左边界=图像宽 *2/3 , 右边界 =图像宽， 上边界 = 图像高 *2/3 , 下边界 =图像高" ， 那么， 终端设备的控制软件可以根据辅助光 源落入摄像头模块所拍摄图像中的方格区域（用 "左边界=图像宽 *2/3 , 右边 界=图像宽， 上边界 =图像高 *2/3 , 下边界 =图像高" 表示） ， 可从表 1所示的 编码库存储的方格区域与编码的映射关系中查询到辅助光源落入摄像头模块 所拍摄图像中的方格区域对应的编码为 I。
在步骤 204、 终端设备根据查询到的编码， 从编码指令映射库存储的编码 与操作指令的映射关系中获取该编码对应的操作指令，并执行编码对应的操作 指令。
在本发明的一个实施例中，结合表 1所示的编码库存储的方格区域与编码 的映射关系， 假设编码指令映射库所存储的编码与操作指令的映射关系如表 2 所示。
表 2编码指令映射库存储的编码与操作指令的映射关系
Figure imgf000011_0001
在本发明的一个实施例中，表 3表示的是： 在表 1中的摄像头模块所拍摄 图像被均匀划分形成的 9个方格区域上显示表 2所示的编码指令映射库存储的 编码与操作指令的映射关系。
表 3 编码： A 编码： B 编码： C 操作指令： 放大音量 操作指令： 保留 操作指令： 换下一个频道 编码： D 编码： E 编码： F 操作指令： 减小音量 操作指令： 保留 操作指令： 保留 编码： G 编码： H 编码： I 操作指令： 静音 操作指令： 保留 操作指令： 换下一个频道 在图 2所描述的人机交互方法的一个实施例中，终端设备可以利用摄像头 模块来捕获手势点击辅助光幕后形成的辅助光源，并且确定辅助光源在摄像头 模块所拍摄图像中的位置， 进而可以查询该位置对应的编码， 以及根据该编码 从所存储的编码与操作指令的映射关系中获取该编码对应的操作指令，并执行 该编码对应的操作指令。 因此， 图 2所述的人机交互方法是通过辅助光源来作 为人机交互的依据的，这不仅具有非常良好的抗干扰能力以及更高的操控准确 率， 还具有较高的商业使用价值。
以上对根据本发明的另一个实施例的人机交互方法进行了详细说明。 根据本发明的另一个实施例， 还提供另一种人机交互方法。
请参阅图 6, 图 6是根据本发明的另一实施例的人机交互方法的流程图。 在图 6所述的人机交互方法中，以手势以滑动的方式来触及辅助光幕而形成辅 助光源为例加以说明。 如图 6所示， 该人机交互方法至少包括以下步骤。
在步骤 601、 终端设备利用摄像头模块捕获手势滑动辅助光幕后形成的辅 助光源。
在本发明的一个实施例中，辅助光幕的具体实现方式已经在前面实施例中 进行了详细介绍， 本实施例不作赞述。
在本发明的一个实施例中，终端设备可以利用摄像头模块来捕获携带有手 势滑动辅助光幕后形成高亮的辅助光源的图像， 并对该图像进行处理， 以获得 仅显示手势点击辅助光幕后形成的高亮的辅助光源的图像。
在步骤 602、 终端设备确定辅助光源在摄像头模块所拍摄图像中的运动轨 迹。
在本发明的一个实施例中，终端设备可以通过控制软件对仅显示手势滑动 辅助光幕后形成的高亮的辅助光源的图像序列进行连续识别，从而可以确定辅 助光源在摄像头模块所拍摄图像中的运动轨迹。
在本发明的一个实施例中，由于手势是以滑动的方式来触及辅助光幕形成 辅助光源的， 因此， 终端设备可以确定辅助光源所经过的摄像头模块所拍摄图 像中的方格区域数量和方向，并以辅助光源所经过的摄像头模块所拍摄图像中 的方格区域数量和方向作为辅助光源在摄像头模块拍摄图像中的运动轨迹。 在步骤 603、 终端设备查询上述运动轨迹对应的编码。
在本发明的一个实施例中，终端设备的编码库中可以预先存储如表 4所示 的辅助光源所经过的摄像头模块拍摄图像中的方格区域数量、方向以及编码三 者的映射关系。 结合下附的表 4, 假设摄像头模块所拍摄图像被均匀划分成如 图 7所示的多个方格区域， 并且当辅助光源向下经过 3个方格区域时， 该运动 轨迹对应了表 4所示的编码 a。 当辅助光源向右经过 3个方格区域时， 该运动 轨迹对应了表 4所示的编码 b。 当辅助光源斜向上经过 3个方格区域时， 该运 动轨迹对应了表 4所示的编码 Co
表 4所示的辅助光源所经过的摄像头模块拍摄图像中的方格区域数量、方 向以及编码三者的映射关系
Figure imgf000013_0001
设备可以通过控制软件从编码库存储的如表 4 所示的辅助光源所经过的摄像 头模块拍摄图像中的方格区域数量、方向以及编码三者的映射关系中， 查询出 辅助光源向下经过 3个方格区域时对应的编码 a。
在步骤 604、 终端设备根据查询到的编码， 从编码指令映射库存储的编码 与操作指令的映射关系中获取该编码对应的操作指令，并执行编码对应的操作 指令。
在本发明的一个实施例中，结合表 4所示的辅助光源所经过的摄像头模块 拍摄图像中的方格区域数量、方向以及编码三者的映射关系， 编码指令映射库 存储了如表 5所示的编码与操作指令的映射关系。
表 5 编码指令映射库存储的编码与操作指令的映射关系
编码 ^"曰 说明
a 向下滚动内容 当辅助光源向下经过 3个方格区域时， 就向下滚动内容 b 放大画面 当辅助光源斜向上经过 3个方格区域时， 就放大画面 c 翻下一页 当辅助光源向右经过 3个方格区域时， 翻下一页 在本发明的一个实施例中，当终端设备的控制软件根据辅助光源在摄像头 模块所拍摄图像中的运动轨迹，从表 4中查询出辅助光源在摄像头模块所拍摄 图像中的运动轨迹对应编码 a时，终端设备的控制软件可以进一步从表 5中获 取操作指令为 "向下滚动内容" ， 此时， 所述终端设备可知执行该操作指令， 实现向下滚动内容。
在图 6所描述的人机交互方法的实施例中，终端设备可以利用摄像头模块 来捕获手势滑动辅助光幕后形成的辅助光源，并且确定辅助光源在摄像头模块 所拍摄图像中的运动轨迹， 进而可以查询该运动轨迹对应的编码， 以及根据该 编码从存储的编码与操作指令的映射关系中获取该编码对应的操作指令并执 行该编码对应的操作指令。 因此， 图 6所述的人机交互方法是通过辅助光源作 为人机交互的依据的， 这不仅具有较佳的抗干扰能力以及更高的操控准确率， 还具有较高的商业使用价值。
以上对根据本发明的一个实施例的人机交互方法进行了详细说明。 根据本发明的另一个实施例， 还提供了一种终端设备。
请参阅图 8, 图 8是根据本发明的另一个实施例的终端设备的结构图。 该 终端设备可以是装载有控制软件的， 并且具有计算能力的电脑、 智能手机、 电 视机以及各种家庭智能设备、 商用智能设备、 办公智能设备、 MID 等， 本发 明实施例不作具体限定。 如图 8所示， 该终端设备包括： 摄像头模块 801、 确 定模块 802及执行模块 803。
摄像头模块 801 , 捕获手势触及辅助光幕后形成的辅助光源。
确定模块 802, 确定上述辅助光源在摄像头模块 801所拍摄图像中的位置 和 /或运动轨迹。
执行模块 803, 根据该位置和 /或运动轨迹执行相对应的操作指令。
在本发明的一个实施例中，所述摄像头模块 801捕获携带有手势触及辅助 光幕后形成的辅助光源的图像， 以及对该图像进行处理， 以获得仅显示手势触 及辅助光幕后形成的辅助光源的图像。
在本发明的一个实施例中，所述确定模块 802确定辅助光源落入所述摄像 头模块 801 所拍摄图像中的方格区域； 和 /或， 确定辅助光源所经过的所述摄 像头模块 801 所拍摄图像中的方格区域数量和方向。 其中， 所述摄像头模块 801所拍摄图像（例如以左上角为原点）被均匀划分成多个方格区域。
如图 8所示，本发明的一个实施例的终端设备中，所述执行模块 803包括： 查询子模块 8031及获取子模块 8032。
所述查询子模块 8031 , 查询该位置和 /或运动轨迹对应的编码。
所述获取子模块 8032, 根据该编码从存储的编码与操作指令的映射关系 中获取该编码对应的操作指令， 并执行该编码对应的操作指令。
在本发明的一个实施例中， 所述查询子模块 8031根据辅助光源落入所述 摄像头模块 801拍摄图像中的方格区域，从存储的方格区域与编码的映射关系 中查询辅助光源落入该摄像头模块 801拍摄图像中的方格区域对应的编码。
所述查询子模块 8031还可根据辅助光源所经过该摄像头模块 801所拍摄 图像中的方格区域数量和方向，从存储的方格区域数量、方向以及编码三者的 映射关系中查询所述方格区域数量和方向对应的编码。
在本发明的一个实施例中，上述操作指令可以为电脑操作指令或电视机遥 控指令的任意一种， 本实施例不作限定。
根据本发明的一个实施例， 根据图 1 所示的人机交互方法可以是由图 8
101可以由图 8所示的摄像头模块 801来执行。 图 1所示的步骤 102可以由图 8所示的确认模块 802来执行。 图 1所示的步骤 103可以由图 8所示的执行模 块 803的查询子模块 8031来执行。 图 1所示的步骤 104可以由图 8所示的执 行模块 803的获取子模块 8032来执行。
根据本发明的另一个实施例，图 8所示的终端设备中的各个单元可以分别 或全部合并为一个或若干个另外的单元来构成， 或者其中的某个（些）单元还 可以再拆分为功能上更小的多个单元来构成， 这可以实现同样的操作， 而不影 响本发明的实施例的技术效果的实现。上述单元是基于逻辑功能划分的，在实 际应用中，一个单元的功能也可以由多个单元来实现，或者多个单元的功能由 一个单元实现。 在本发明的其它实施例中， 终端设备也可以包括其它模块。 但 在实际应用中， 这些功能也可以由其它单元协助实现， 并且可以由多个单元协 作实现。
根据本发明的另一个实施例， 可以通过在包括中央处理单元（CPU )、 随 机存取存储器（RAM ) 、 只读存储器（ROM )等处理元件和存储介质的例如 计算机的通用计算设备上运行能够执行如图 1 中所示的人机交互方法的计算 机程序（包括程序代码）， 来构造如图 8中所示的终端设备， 以及来实现根据 本发明的实施例的人机交互方法。所述计算机程序可以记载于例如计算机可读 记录介质上， 并通过计算机可读记录介质装载于上述计算设备中， 并在其中运 行。
图 8 所示的终端设备可以利用摄像头模块捕获手势触及辅助光幕后形成 的辅助光源， 并且确定辅助光源在摄像头模块拍摄图像中的位置和 /或运动轨 迹， 进而可以查询到该位置和 /或运动轨迹所对应的编码， 并执行该编码所对 应的操作指令。 因此， 图 8所示的终端设备与人之间可以通过辅助光源来作为 人机交互的依据， 这不仅具有非常良好的抗干扰能力以及更高的操控准确率， 还具有很好的商业使用价值。
以上对根据本发明的一个实施例的终端设备进行了详细说明。 根据本发明的另一个实施例， 还提供一种人机交互系统。
请参阅图 9, 图 9是根据本发明的一个实施例的人机交互系统的结构图。 如图 9所示， 该人机交互系统可以包括辅助光幕 901、 摄像头 902以及终端设 备 903。 其中， 所述摄像头 902可以内置在该终端设备 903中， 或通过有线或 者无线方式与所述终端设备 903连接，所述摄像头 902的拍摄区域覆盖所述辅 助光幕 901的工作覆盖区域。 其中， 在图 9所示的人机交互系统中， 以摄像头 902通过有线方式与所述终端设备 903连接为例加以说明。
所述辅助光幕 901 , 供手势触及以形成辅助光源。
所述摄像头 902, 捕获手势触及所述辅助光幕 901后形成的辅助光源。 所述终端设备 903包括： 确定模块 9031及执行模块 9032。
所述确定模块 9031 , 确定辅助光源在所述摄像头 902拍摄图像中的位置 和 /或运动轨迹；
所述执行模块 9032, 根据该位置和 /或运动轨迹执行相对应的操作指令。 在本发明的一个实施例中，所述摄像头 902具体可捕获携带有手势触及辅 助光幕后形成的辅助光源的图像， 以及对上述图像进行处理， 以获得仅显示手 势触及辅助光幕后形成的辅助光源的图像。
在本发明的一个实施例中， 所述终端设备 903的确定模块 9031具体可确 定辅助光源落入所述摄像头 902所拍摄图像中的方格区域； 和 /或， 确定辅助 光源所经过的摄像头 902拍摄图像中的方格区域数量和方向。 所述摄像头 902 拍摄图像被均匀划分成多个方格区域。
本实施例中， 终端设备 903的执行模块 9032包括： 查询子模块 90321及 获取子模块 90322。
所述查询子模块 90321 , 查询该位置和 /或运动轨迹对应的编码。
所述获取子模块 90322, 根据该编码从存储的编码与操作指令的映射关系 中获取该编码对应的操作指令， 并执行该编码对应的操作指令。
在本发明的一个实施例中，所述查询子模块 90321根据辅助光源落入所述 摄像头 902所拍摄图像中的方格区域，从存储的方格区域与编码的映射关系中 查询辅助光源落入该摄像头 902所拍摄图像中的方格区域对应的编码。
所述查询子模块 90321还可根据辅助光源所经过的摄像头 902拍摄图像中 的方格区域数量和方向，从存储的方格区域数量、方向以及编码三者的映射关 系中查询拍摄图像中的方格区域数量和方向对应的编码。
在本发明的一个实施例中，上述操作指令可以为电脑操作指令或电视机遥 控指令， 本实施例不作限定。
在本发明的一个实施例中， 所述摄像头 902可以为红外光摄像头，相应地 所述辅助光幕 901可以为红外光辅助光幕。所述摄像头 902还可以为可见光摄 像头， 相应地所述辅助光幕 901可以为可见光辅助光幕。
根据本发明的一个实施例， 根据图 1 所示的人机交互方法可以是由图 9 所示的人机交互系统中的各个单元来执行的人机交互方法。例如， 图 1所示的 步骤 101可以由图 9所示的摄像头 902来执行。图 1所示的步骤 102可以由图 9所示的确认模块 9031来执行。 图 1所示的步骤 103可以由图 9所示的执行 模块 9032的查询子模块 90321来执行。 图 1所示的步骤 104可以由图 9所示 的执行模块 803的获取子模块 90322来执行。
根据本发明的另一个实施例，图 9所示的人机交互系统中的各个单元可以 分别或全部合并为一个或若干个另外的单元来构成， 或者其中的某个（些）单 元还可以再拆分为功能上更小的多个单元来构成， 这可以实现同样的操作， 而 不影响本发明的实施例的技术效果的实现。 上述单元是基于逻辑功能划分的， 在实际应用中，一个单元的功能也可以由多个单元来实现， 或者多个单元的功 能由一个单元实现。在本发明的其它实施例中，人机交互系统也可以包括其它 模块。 但在实际应用中， 这些功能也可以由其它单元协助实现， 并且可以由多 个单元协作实现。
根据本发明的另一个实施例，可以通过在包括处理元件和存储介质的例如 计算机的通用计算设备上运行能够执行如图 1 中所示的人机交互方法的计算 机程序（包括程序代码）， 来构造如图 9中所示的人机交互系统， 以及来实现 根据本发明的实施例的人机交互方法。所述计算机程序可以记载于例如计算机 可读记录介质上， 并通过计算机可读记录介质装载于上述计算设备中， 并在其 中运行。
其中，所述存储介质可以包括：闪存盘、只读存储器（ Read-Only Memory ,
ROM ) 、 随机存取器（Random Access Memory, RAM ) 、 磁盘或光盘等。
在图 9所示的人机交互系统中，终端设备可以利用摄像头来捕获手势触及 辅助光幕后形成的辅助光源，并且确定辅助光源在摄像头所拍摄图像中的位置 和 /或运动轨迹， 进而可以查询该位置和 /或运动轨迹所对应的编码， 以及才艮据 该编码从所存储的编码与操作指令的映射关系中获取该编码对应的操作指令， 并执行该编码对应的操作指令。 因此， 图 9所示的人机交互系统是通过辅助光 源作为人机交互的依据， 这不仅具有较佳的抗干扰能力以及更高的操控准确 率， 还具有较高的商业使用价值。 当辅助光幕布置在桌面时， 辅助光幕需要与桌面平行并间隔一定的距离设置， 否则会形成光痕而影响识别。 当然， 所述辅助光幕可以布置于墙面、 桌面， 或 布置于空中立面上， 这样用户可以在空中触及辅助光幕，从而实现人机互动操 作。 此外， 所述辅助光幕可以采用可见光和红外光双光幕方式， 使得手势触及 辅助光幕时， 手指被可见光照亮， 人眼获得反馈， 同时摄像头则捕获红外光辅 助光幕与手指形成的光斑（即辅助光源） 。
了本发明的各个实施例，但是它们不用以限制本发明的范围， 本发明的范围由 后附的权利要求来定义。在不脱离本发明的精神和原则的情况下， 所做的任何 修改、 等同替换、 改进等， 均应包含在权利要求的保护范围之内。

Claims

权 利 要 求
1、 一种人机交互方法， 包括：
终端设备利用摄像头模块捕获手势触及辅助光幕后形成的辅助光源； 所述终端设备确定所述辅助光源在所述摄像头模块拍摄图像中的位置和 / 或运动轨迹； 及
所述终端设备根据所述位置和 /或运动轨迹执行相对应的操作指令。
2、 根据权利要求 1所述的人机交互方法， 所述终端设备利用摄像头模块 捕获手势触及辅助光幕后形成的辅助光源的步骤， 包括：
所述终端设备利用摄像头模块捕获携带有手势触及辅助光幕后形成的辅 助光源的图像， 并对所述图像进行处理， 以获得仅显示所述手势触及辅助光幕 后形成的辅助光源的图像。
3、 根据权利要求 1或 2所述的人机交互方法， 所述终端设备确定所述辅 助光源在所述摄像头模块拍摄图像中的位置和 /或运动轨迹的步骤， 包括： 所述终端设备确定所述辅助光源落入的所述摄像头模块拍摄图像中的方 格区域；
和 /或， 所述终端设备确定所述辅助光源所经过的所述摄像头模块拍摄图 像中的方格区域数量和方向；
其中， 所述摄像头模块拍摄图像被均匀划分成多个所述方格区域。
4、 根据权利要求 3所述的人机交互方法， 所述终端设备根据所述位置和 / 或运动轨迹执行相对应的操作指令的步骤， 包括：
所述终端设备查询所述位置和 /或运动轨迹对应的编码； 和
所述终端设备根据所述编码，从存储的编码与操作指令的映射关系中获取 所述编码对应的操作指令， 并执行所述编码对应的操作指令。
5、 根据权利要求 4所述的方法， 所述终端设备查询所述位置和 /或运动轨 迹对应的编码的步骤， 包括：
所述终端设备根据所述辅助光源落入的所述摄像头模块拍摄图像中的方 格区域，从存储的方格区域与编码的映射关系中查询所述辅助光源落入的所述 摄像头模块拍摄图像中的方格区域对应的编码；
和 /或， 所述终端设备根据所述辅助光源所经过的所述摄像头模块拍摄图 像中的方格区域数量和方向，从存储的方格区域数量、方向以及编码三者的映 射关系中查询所述辅助光源所经过的所述摄像头模块拍摄图像中的方格区域 数量和方向对应的编码。
6、 根据权利要求 1所述的方法， 所述辅助光幕与显示屏幕重叠或平行。
7、 根据权利要求 6所述的方法， 所述辅助光幕与显示屏幕平行， 所述辅 助光幕为红外光辅助光幕并叠加一个可见光光幕，用以指示所述辅助光幕的位 置。
8、 一种终端设备， 包括：
摄像头模块， 用于捕获手势触及辅助光幕后形成的辅助光源；
确定模块，用于确定所述辅助光源在所述摄像头模块拍摄图像中的位置和
/或运动轨迹 ^ 及
执行模块， 用于根据所述位置和 /或运动轨迹执行相对应的操作指令。
9、 根据权利要求 8所述的终端设备， 所述摄像头模块用于捕获携带有手 势触及辅助光幕后形成的辅助光源的图像， 以及对所述图像进行处理， 以获得 仅显示所述手势触及辅助光幕后形成的辅助光源的图像。
10、根据权利要求 8或 9所述的终端设备， 所述确定模块用于确定所述辅 助光源落入的所述摄像头模块拍摄图像中的方格区域； 和 /或， 用于确定所述 辅助光源所经过的所述摄像头模块拍摄图像中的方格区域数量和方向； 其中， 所述摄像头模块拍摄图像被均匀划分成多个所述方格区域。
11、 根据权利要求 10所述的终端设备， 所述执行模块包括：
查询子模块， 用于查询所述位置和 /或运动轨迹对应的编码； 和
获取子模块， 用于根据所述编码，从存储的编码与操作指令的映射关系中 获取所述编码对应的操作指令， 并执行所述编码对应的操作指令。
12、 根据权利要求 11所述的终端设备， 所述查询子模块用于根据所述辅 助光源落入的所述摄像头模块拍摄图像中的方格区域，从存储的方格区域与编 码的映射关系中查询所述辅助光源落入的所述摄像头模块拍摄图像中的方格 区域对应的编码。
13、 根据权利要求 12所述的终端设备， 所述查询子模块用于根据所述辅 助光源所经过的所述摄像头模块拍摄图像中的方格区域数量和方向，从存储的 方格区域数量、方向以及编码三者的映射关系中查询所述辅助光源所经过的所 述摄像头模块拍摄图像中的方格区域数量和方向对应的编码。
14、 一种人机交互系统， 包括辅助光幕、 摄像头以及终端设备， 所述摄像 头内置在所述终端设备中， 或以有线或无线方式与所述终端设备连接， 所述摄 像头的拍摄区域覆盖所述辅助光幕的工作覆盖区域， 其中：
所述辅助光幕， 用于供手势触及以形成辅助光源；
所述摄像头， 用于捕获手势触及所述辅助光幕后形成的辅助光源； 所述终端设备包括：
确定模块， 用于确定所述辅助光源在所述摄像头拍摄图像中的位置和 /或 运动轨迹； 及
执行模块， 用于根据所述位置和 /或运动轨迹执行相对应的操作指令。
15、 根据权利要求 14所述的人机交互系统， 所述摄像头用于捕获携带有 手势触及辅助光幕后形成的辅助光源的图像， 以及对所述图像进行处理， 以获 得仅显示所述手势触及辅助光幕后形成的辅助光源的图像。
16、根据权利要求 14或 15所述的人机交互系统， 所述确定模块用于确定 所述辅助光源落入的所述摄像头拍摄图像中的方格区域； 和 /或， 被配置为确 定所述辅助光源所经过的所述摄像头拍摄图像中的方格区域数量和方向； 其 中， 所述摄像头拍摄图像被均匀划分成多个所述方格区域。
17、 根据权利要求 16所述的人机交互系统， 所述执行模块包括： 查询子模块， 用于查询所述位置和 /或运动轨迹对应的编码； 和
获取子模块， 用于根据所述编码，从存储的编码与操作指令的映射关系中 获取所述编码对应的操作指令， 并执行所述编码对应的操作指令。
18、 根据权利要求 17所述的人机交互系统， 所述查询子模块用于根据所 述辅助光源落入的所述摄像头拍摄图像中的方格区域，从存储的方格区域与编 码的映射关系中查询所述辅助光源落入的所述摄像头拍摄图像中的方格区域 对应的编码。
19、 根据权利要求 18所述的人机交互系统， 所述查询子模块用于根据所 述辅助光源所经过的所述摄像头拍摄图像中的方格区域数量和方向，从存储的 方格区域数量、方向以及编码三者的映射关系中查询所述辅助光源所经过的所 述摄像头拍摄图像中的方格区域数量和方向对应的编码。
20、根据权利要求 14所述的人机交互系统， 所述摄像头为红外光摄像头， 并且所述辅助光幕为红外光辅助光幕； 或者， 所述摄像头为可见光摄像头， 并 且所述辅助光幕为可见光辅助光幕。
21、一种包括程序代码的计算机程序， 当所述计算机程序运行在计算机上 时， 所述程序代码执行根据权利要求 1所述的人机交互方法的各步骤。
PCT/CN2013/080324 2012-10-15 2013-07-29 人机交互方法及相关设备、系统 Ceased WO2014059810A1 (zh)

Priority Applications (1)

Application Number Priority Date Filing Date Title
US14/677,883 US20150212727A1 (en) 2012-10-15 2015-04-02 Human-computer interaction method, and related device and system

Applications Claiming Priority (2)

Application Number Priority Date Filing Date Title
CN201210388925.9A CN103729131A (zh) 2012-10-15 2012-10-15 一种人机交互方法及相关设备、系统
CN201210388925.9 2012-10-15

Related Child Applications (1)

Application Number Title Priority Date Filing Date
US14/677,883 Continuation US20150212727A1 (en) 2012-10-15 2015-04-02 Human-computer interaction method, and related device and system

Publications (1)

Publication Number Publication Date
WO2014059810A1 true WO2014059810A1 (zh) 2014-04-24

Family

ID=50453226

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/CN2013/080324 Ceased WO2014059810A1 (zh) 2012-10-15 2013-07-29 人机交互方法及相关设备、系统

Country Status (3)

Country Link
US (1) US20150212727A1 (zh)
CN (1) CN103729131A (zh)
WO (1) WO2014059810A1 (zh)

Families Citing this family (9)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN103777746B (zh) * 2012-10-23 2018-03-13 腾讯科技（深圳）有限公司 一种人机交互方法、终端及系统
GB2548577A (en) * 2016-03-21 2017-09-27 Promethean Ltd Interactive system
CN107967100A (zh) * 2017-12-06 2018-04-27 Tcl移动通信科技（宁波）有限公司 基于移动终端摄像头的操作控制处理方法、及存储介质
WO2019128628A1 (zh) * 2017-12-26 2019-07-04 Oppo广东移动通信有限公司 输出模组、输入输出模组及电子装置
US10813195B2 (en) 2019-02-19 2020-10-20 Signify Holding B.V. Intelligent lighting device and system
CN111629129B (zh) * 2020-03-11 2021-10-15 甘肃省科学院 一种多位并发跟踪平均光亮度的超大平面拍摄系统
CN115113722A (zh) * 2021-03-23 2022-09-27 华为技术有限公司 一种人机交互方法、装置及相关电子设备
CN114740980B (zh) * 2022-04-22 2024-06-21 北京交通大学 一种基于认知过程和状态的人机交互模式设计方法及系统
CN117301091A (zh) * 2023-11-08 2023-12-29 国能浙能宁东发电有限公司 一种轨道式巡检机器人控制方法

Citations (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
WO2009128064A2 (en) * 2008-04-14 2009-10-22 Pointgrab Ltd. Vision based pointing device emulation
CN102012740A (zh) * 2010-11-15 2011-04-13 中国科学院深圳先进技术研究院 人机交互方法及系统
CN102221888A (zh) * 2011-06-24 2011-10-19 北京数码视讯科技股份有限公司 基于遥控器的控制方法及系统
CN102495674A (zh) * 2011-12-05 2012-06-13 无锡海森诺科技有限公司 一种红外人机交互方法及装置

Family Cites Families (9)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US9760214B2 (en) * 2005-02-23 2017-09-12 Zienon, Llc Method and apparatus for data entry input
US8441467B2 (en) * 2006-08-03 2013-05-14 Perceptive Pixel Inc. Multi-touch sensing display through frustrated total internal reflection
US8581852B2 (en) * 2007-11-15 2013-11-12 Microsoft Corporation Fingertip detection for camera based multi-touch systems
CN101231450B (zh) * 2008-02-25 2010-12-22 陈伟山 多点及物体触摸屏装置及多点触摸的定位方法
CN101770314A (zh) * 2009-01-01 2010-07-07 张海云 红外一字激光多点触摸屏装置及触摸定位方法
CN102597935A (zh) * 2009-09-01 2012-07-18 智能技术无限责任公司 具有提高的信噪比(snr)的交互输入系统和图像捕获方法
WO2011085023A2 (en) * 2010-01-06 2011-07-14 Celluon, Inc. System and method for a virtual multi-touch mouse and stylus apparatus
CN102523395B (zh) * 2011-11-15 2014-04-16 中国科学院深圳先进技术研究院 具有多点触控功能的电视系统、触控定位识别方法及系统
CN102701033A (zh) * 2012-05-08 2012-10-03 华南理工大学 一种基于图像识别技术的电梯按键及方法

Patent Citations (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
WO2009128064A2 (en) * 2008-04-14 2009-10-22 Pointgrab Ltd. Vision based pointing device emulation
CN102012740A (zh) * 2010-11-15 2011-04-13 中国科学院深圳先进技术研究院 人机交互方法及系统
CN102221888A (zh) * 2011-06-24 2011-10-19 北京数码视讯科技股份有限公司 基于遥控器的控制方法及系统
CN102495674A (zh) * 2011-12-05 2012-06-13 无锡海森诺科技有限公司 一种红外人机交互方法及装置

Also Published As

Publication number Publication date
US20150212727A1 (en) 2015-07-30
CN103729131A (zh) 2014-04-16

Similar Documents

Publication Publication Date Title
WO2014059810A1 (zh) 人机交互方法及相关设备、系统
WO2014063498A1 (zh) 人机交互方法、终端及系统
US9298266B2 (en) Systems and methods for implementing three-dimensional (3D) gesture based graphical user interfaces (GUI) that incorporate gesture reactive interface objects
US9513791B2 (en) Electronic device system with process continuation mechanism and method of operation thereof
US9047006B2 (en) Electronic device system with information processing mechanism and method of operation thereof
CN106415472B (zh) 一种手势控制方法、装置、终端设备和存储介质
KR102462364B1 (ko) 스크롤바를 이용한 이미지 디스플레이 방법 및 이를 위한 장치
KR20150130431A (ko) 제스처들을 이용한 터치 입력들의 향상
CN101727261A (zh) 页面操作方法与电子装置
US20140009395A1 (en) Method and system for controlling eye tracking
CN102819398A (zh) 一种在触摸屏设备上滑动控制摄像机的方法
JP2013533541A (ja) 文字の選択
US9182908B2 (en) Method and electronic device for processing handwritten object
WO2018010316A1 (zh) 桌面页面管理的方法及装置
CN111427505A (zh) 页面操作方法、装置、终端及存储介质
CN104914982B (zh) 一种电子设备的控制方法和装置
US20130238976A1 (en) Information processing apparatus, information processing method, and computer program
EP2899623A2 (en) Information processing apparatus, information processing method, and program
CN106155452A (zh) 一种单手操作的实现方法及终端
CN110333780A (zh) 功能触发方法、装置、设备及存储介质
CN103543824B (zh) 手势输入系统及方法
CN113687722A (zh) 电子设备的页面控制方法、装置、设备和存储介质
CN107422975A (zh) 移动终端的控制方法和控制装置
KR101381878B1 (ko) 마우스를 이용하여 터치 입력을 구현하기 위한 방법, 장치, 및 컴퓨터 판독 가능한 기록 매체
EP3101522A1 (en) Information processing device, information processing method, and program

Legal Events

Date Code Title Description
121 Ep: the epo has been informed by wipo that ep was designated in this application

Ref document number: 13847912

Country of ref document: EP

Kind code of ref document: A1

NENP Non-entry into the national phase

Ref country code: DE

32PN Ep: public notification in the ep bulletin as address of the adressee cannot be established

Free format text: NOTING OF LOSS OF RIGHTS PURSUANT TO RULE 112(1) EPC (EPO FORM 1205A DATED 17.09.2015)

122 Ep: pct application non-entry in european phase

Ref document number: 13847912

Country of ref document: EP

Kind code of ref document: A1