JP2013069341A - Input method - Google Patents

Abstract

PROBLEM TO BE SOLVED: To provide an input device that is user-friendly.SOLUTION: The input device relating to this invention includes an input part for receiving a prescribed motion image signal, a motion detection part for performing motion detection form the motion image signal inputted to the input part, a video signal processing part for outputting a prescribed video signal when motion detected by the motion detection part is prescribed motion, and a control part. The control part controls the motion detection part so as not to perform motion detection when the video signal processing part outputs the prescribed video signal.

Description

  The present invention relates to an input device for inputting a user instruction.

2. Description of the Related Art PCs and televisions that accept user operations via a graphical user interface (hereinafter referred to as GUI) and simultaneously give feedback of operation results to users have become widespread.

For example, Patent Document 1 measures the three-dimensional coordinates of points indicating the characteristics of the movement of a person's limbs,
Using the feature points other than the hand among the extracted feature points, the three-dimensional position / posture of the person is estimated, and each coordinate axis is uniquely determined according to the user posture using the estimated three-dimensional position of the user as a reference point. By configuring a user coordinate system and converting feature points of the user's limbs to the user coordinate system, feature quantities that do not depend on the user's position / posture are extracted,
A gesture recognition device is disclosed that estimates a gesture performed by a person by matching the time series change with a gesture model.

JP 2000-149025 A

However, when the gesture recognition device described in Patent Document 1 is used for an input operation to a device such as a television, it is difficult to distinguish whether a moving image captured of a user represents an input to the device. . For example, when the user points to the right with his hand to indicate input to the device and then returns his hand to the original position, the movement of returning his hand to the original position is recognized as an input operation pointing to the left. Sometimes. In other words, Patent Document 1 does not consider a mechanism regarding which part of movement is accepted as an input to the apparatus when the user performs a gesture.

Furthermore, there is a demand for continuous input in the operation of devices and software. For example, when operating a television, continuously pressing a volume control button of a remote controller or continuously pressing a channel switching button. However, in Patent Document 1, it is not considered to smoothly perform an operation of continuously inputting by a gesture command.

Therefore, in consideration of such circumstances, the present invention, for example, presents an operation effective for the operation to the user and simultaneously provides real-time feedback, and further allows continuous input operation, which is convenient for the user. An object is to provide an apparatus.

An input device according to the present invention includes an input unit to which a predetermined motion image signal is input, a motion detection unit that performs motion detection from the motion image signal input to the input unit, and the motion detected by the motion detection unit. A video signal processing unit that outputs a predetermined video signal when the video signal is a predetermined motion, and a control unit, wherein the control unit outputs the predetermined video signal by the video signal processing unit The motion detection unit controls the motion detection unit so as not to perform motion detection.

For example, after the user shakes his hand, the user's hand returns to the position before the hand is shaken, but the animation continues and the user moves his hand when the item is being selected. By not recognizing, the movement of returning to the position before the user's hand waved is not accepted as the item selection operation.

According to the present invention, for example, it is possible to provide an input device that presents a user with an effective operation for operation and at the same time provides a real-time feedback and can perform a continuous input operation and is convenient for the user.

1 is an overview diagram showing an input device of Example 1. FIG. 1 is a block diagram illustrating a configuration of an input device according to a first embodiment. FIG. 3 is an overview diagram illustrating a user operation method of the input device according to the first embodiment. FIG. 6 is an overview diagram illustrating correspondence between GUI movement and user operation according to the first exemplary embodiment. 3 is a flowchart illustrating the operation of the input device according to the first embodiment. FIG. 3 is an overview diagram illustrating a configuration of a GUI according to the first embodiment. FIG. 10 is an overview diagram illustrating a correspondence between a GUI movement and a user operation according to the second embodiment. 6 is a flowchart illustrating the operation of the input device according to the second embodiment. FIG. 10 is an overview diagram illustrating a correspondence between a GUI movement and a user operation according to the third embodiment. 12 is a flowchart illustrating the operation of the input device according to the third embodiment. FIG. 10 is a block diagram illustrating a configuration of an input device according to a fourth embodiment. FIG. 10 is an overview diagram illustrating correspondence between GUI movement and pointer movement according to the fourth embodiment.

  Each embodiment to which the present invention is applied will be described below.

The input device 100 according to the present exemplary embodiment is a device that can detect an operation of a user's hand from a moving image obtained by capturing an image of the user and change the GUI display according to the operation.
1 shows a display unit 101, an imaging unit 102, a user 103, a menu item 104, and a selection item 1.
05 shows an overview of the operating environment when the user uses the input device 100.

The display unit 101 is a display device of the input device 100, and is configured by a display device such as a liquid crystal display or a plasma display, for example. The display unit 101 is composed of a display panel, a panel control circuit, and a panel control driver, and displays a video composed of data supplied from a video processing unit 202 described later on the display panel. The imaging unit 102 is a device for inputting a moving image to the input device 100, and is, for example, a camera. The user 103 uses the input device 10
A user who performs an operation on 0. The menu item 104 is G displayed on the display unit 101.
It is a UI and a menu composed of a plurality of items. A selection item 105 is an item selected from the menu items 104 by a user operation.

For example, as illustrated in FIG. 2, the input device 100 includes a display unit 101, an imaging unit 102, and motion detection.
A (motion detection) unit 200, a system control unit 201, and a video processing unit 202 are provided.

The motion detection (motion detection) unit 200 is a circuit that receives a moving image signal from the imaging unit 102, detects a human motion from the received moving image signal, and outputs the detected motion as a predetermined command.

The system control unit 201 is composed of, for example, a microprocessor and detects motion (motion detection).
The operation of the video display unit 202 is controlled according to the command received from the unit 200.

The video processing unit 202 is configured by a processing device such as an ASIC, FPGA, or MPU, for example. The video processing unit 202 converts the GUI video data according to the control of the system control unit 201.
The data is converted into a form that can be processed by the display unit 101 and output.

FIG. 3 shows a user's operation when the user operates the input device of the present invention. In the present invention, the menu item 104 is moved according to the direction and speed of the user's hand waving when the user shakes his / her hand, and the user moves the hand to the left / right to select any item. The item can be selected by moving to. A recognition point 301 indicates the position of the user's hand recognized by the input device. The hand shake 302 indicates that the menu item 10
4 shows a movement of waving a hand to move 4, and a user operation input to the input device 100. In addition, the hand return 303 is performed after the user shakes his / her hand like the hand shake 302.
The movement of the user's hand returning to the position before waving is shown.

FIG. 4 shows the state of the GUI displayed on the user's hand and the display unit 101 when the user operates the input device 100.

Next, the operation of the input device 100 configured as described above will be described with reference to the flowcharts of FIGS. 1, 2, 3, 4, and 5.

The input device 100 is a device that can detect a motion of a user's hand from a moving image signal obtained by capturing an image of the user and change the display of the GUI according to the motion.

First, the flow of processing until the input device 100 detects the movement of the user's hand and changes the display of the GUI according to the operation will be described with reference to FIG. The user enters the input terminal 100
On the other hand, it is assumed that the operation of the input terminal 100 is started by pressing a power button (not shown). In response to the start of the operation, the system control unit 201 instructs the video processing unit 202 to display a GUI. In response to the instruction, the video processing unit 202 outputs a video signal suitable for input to the display unit 101. As a result, the GUI is displayed on the display unit 101.

In response to the start of the operation, the system control unit 201 instructs the imaging device 102 to start capturing a moving image. In response to the instruction, the imaging device 102 starts capturing a moving image and outputs the captured moving image data to the motion detection (motion detection) unit 200. The motion detection (motion detection) unit 200 detects the position of the user's hand from the received moving image using a method such as feature point extraction, and outputs the hand position data to the system control unit 201. System control unit 20
1 calculates the direction and speed of the user's hand movement from the hand position data, and follows the predetermined direction, and if the hand movement exceeds the predetermined speed, it follows the direction and the speed. The video processing unit 202 is instructed to change the GUI display. In response to the instruction, the video processing unit 202 changes data constituting the GUI and outputs a video signal suitable for input to the display unit 101 based on the data. Thereby, the GUI displayed on the display unit 101 is updated.

Next, details of an operation when the input device 100 accepts an operation of moving the hand by the user as an input will be described with reference to the flowcharts of FIGS. 4 and 5. First, the user starts operating the input terminal 100 according to a predetermined procedure (step 500). An example of the state of the GUI displayed on the user's hand and the display unit 101 at this time is shown in a display 1 of FIG. In response to the start of the operation, the input terminal 100 starts monitoring the movement of the user's hand (step 501).

When the user's hand moves in a specific direction at a speed higher than a predetermined speed (step 501: Ye)
s) As shown in display 2 of FIG. 4, the input terminal 100 starts an animation of the menu item 104 in accordance with the direction and speed of movement of the user's hand. On the other hand, when the user's hand is not moving at a speed equal to or higher than the predetermined speed (step 501: No), the input terminal 100 continues to monitor the movement of the user's hand. Next, the input device 100 monitors the continuation state of the animation operation started in step 502 (step 503: Yes).

Here, the animation is configured to move the menu item 104 over a predetermined time, and the input terminal 100 does not monitor the movement of the user's hand when the animation is operating. As a result, even if the user performs the hand return 303 operation while the animation is operating as shown in the display 3 of FIG. 4, the operation is not reflected in the movement of the menu item 104 and It will be in the state where the operation according to was reflected.

When the operation of the animation is finished (step 503: No), the input device 100
Confirms whether the user has instructed to end the operation, and if the operation is continued (step 504).
: No), the movement of the user's hand is monitored again. The user's hand and the state of the GUI displayed on the display unit 101 at this time are as shown in display 4 of FIG. Since another item is selected according to the direction in which the user moves his / her hand, the user can start the operation on the menu item 104 again. In step 504, when the user gives an instruction to end the operation (step 504: Yes), the input device 100 ends the operation such as monitoring the movement of the user's hand (step 505).

As described above, the input device 100 recognizes the direction and speed in which the user moves the hand, animates the menu item 104 according to the direction and the speed, and moves the user's hand while the animation is moving. Do not recognize. As a result, the user can return the hand to the original position without recognizing the return movement of the hand after waving and indicating the direction. Operations such as selection can be performed continuously and smoothly.

Further, in this embodiment, when a predetermined video signal is output from the video signal processing unit, the motion detection (motion detection) unit is configured not to perform motion detection (motion detection). The control unit measures the time from when the predetermined video signal is output by the processing unit, and when this time does not exceed the predetermined value, the motion detection (motion detection) unit does not perform motion detection (motion detection). Motion detection
The (motion detection) unit may be controlled.

Note that the GUI displayed on the display unit 101 is, for example, as shown in FIG. 6, menu configuration A or menu configuration B that allows the user to operate by moving his / her hand up or down, or movement that the user shakes his / her hand left or right A configuration such as menu configuration C that can be operated with the

Next, Example 2 will be described. In this embodiment, in addition to the operation method described in the input device 100 according to the first embodiment, an operation method is described in which the menu item can be operated even when the user shakes his or her hand at a predetermined speed or less. . The configuration of the input device 100 and the configuration of the GUI displayed on the display unit 101 are the same as those in the first embodiment, and the method for recognizing the movement of the user's hand is different.

Hereinafter, the present embodiment will be described with reference to the drawings. Also, in the following, the same reference numerals are given to those equivalent to the previous embodiment, and the description thereof is omitted to avoid duplication.

FIG. 7 shows the user's hand and the state of the GUI displayed on the display unit 101 when the user operates the input device 100 in this embodiment. The user's hand movements 701 and 702 indicate the movement of the hand when the user slowly moves his / her hand at a speed equal to or lower than a predetermined speed in the operation of the input terminal 100 described later.

The operation of the input device 100 of this embodiment will be described with reference to the flowcharts of FIGS.

First, the user starts operating the input terminal 100 in a predetermined procedure (step 800).
An example of the state of the GUI displayed on the user's hand and the display unit 101 at this time is shown in display 1 of FIG. In response to the start of the operation, the input terminal 100 starts monitoring the movement of the user's hand (step 801). When the user's hand is moving in a specific direction (step 801: Yes)
The speed of the user's hand movement is calculated (step 802).

At this time, if the movement of the user's hand is not faster than a predetermined speed (step 802).
: No), the menu item 104 is moved according to the direction and magnitude of the movement of the user's hand following the direction in which the user moves the hand as shown in the display 2 and the display 3 in FIG. On the other hand, when the user's hand moves at a speed equal to or higher than a predetermined speed in step 802, the operation is the same as in the first embodiment.

As described above, the input device 100 recognizes the direction and speed of the user's hand movement, and when the user's hand movement is moving at a predetermined speed or less, the direction and magnitude of the user's hand movement. If the menu item 104 is moved according to the above and the user's hand moves at a predetermined speed or higher,
The animation of the menu item 104 is started according to the direction and speed of the hand movement, and the movement of the user's hand is not recognized while the animation is moving.

Thereby, the user can grasp that the user's own hand movement is accurately recognized by the input device 100 with a small hand movement or a slow hand movement. Since it is possible to grasp whether the selection item 105 of the menu item 104 can be changed by the movement of the hand at high speed, the operation method and the operation state of the input device 100 according to the first embodiment can be grasped more smoothly. Can be operated more intuitively.

Next, Example 3 will be described. In this embodiment, in addition to the operation method described in the input device 100 of the first embodiment, the animation of the menu item 104 is started by the movement of the user's hand, and the user continues to move the hand while the animation is moving. If you have
A description will be given of an operation method in which an operation for continuing the movement of the animation can be performed regardless of the direction in which the user shakes his / her hand. The configuration of the input device 100 and the G displayed on the display unit 101
The configuration of the UI is the same as that of the first embodiment, and the method for recognizing the movement of the user shaking hands is different.

Hereinafter, the present embodiment will be described with reference to the drawings. Also, in the following, the same reference numerals are given to those equivalent to the previous embodiment, and the description thereof is omitted to avoid duplication.

FIG. 9 shows the user's hand and the state of the GUI displayed on the display unit 101 when the user operates the input device 100 in this embodiment. The user's hand movement 900 shows the movement of the hand when the user continues to shake his / her hand after the animation of the menu item 104 is started by the movement of the user's hand in the display 2 of FIG. Yes.

The operation of the input device 100 of this embodiment will be described with reference to the flowcharts of FIGS.

First, the user starts operating the input terminal 100 in a predetermined procedure (step 1000).
. An example of a GUI state displayed on the user's hand and the display unit 101 at this time is shown in a display 1 of FIG. In response to the start of the operation, the input terminal 100 starts monitoring the movement of the user's hand (step 1001). When the user's hand moves in a specific direction at a speed equal to or higher than a predetermined speed (step 1001: Yes), the input terminal 100 follows the direction and speed in which the user's hand moves as shown in display 2 of FIG. The animation of the menu item 104 is started.

Further, the input terminal 100 continues to monitor the movement of the user's hand (step 1003), and when the user's hand continues to move at a speed equal to or higher than a predetermined speed (step 1003: Yes).
The animation is continued regardless of the direction of the user's hand movement. At this time, as shown in the display 3 of FIG. 9, the next item is continuously selected according to the movement of the first hand recognized when the selection item 105 of the menu item 104 starts the animation. To work. On the other hand, if the movement of the user's hand does not continue to move at a speed equal to or higher than the predetermined speed in step 1003, the operation is the same as in the first embodiment.

Thus, the input device 100 recognizes the direction and speed in which the user moves his / her hand, and if the user's hand moves at a predetermined speed or higher, the menu item depends on the direction or speed in which the user moves. When the animation of 104 is started and the user's hand continues to move at a predetermined speed or higher, the menu item 104 continues to animate in the same direction, and the selection item 105
Works to change one after another.

As a result, when selecting a series of menu items, the user must first shake the hand in the direction in which the menu item is moved, and then select items one after another by continuing to move the hand regardless of the direction in which the hand is shaken. Therefore, menu items can be selected more smoothly and continuously.

Next, Example 4 will be described. In the present embodiment, when the input device 100 according to the first embodiment can input via a so-called pointing device such as a mouse or a touch pad or a sensor device such as an acceleration sensor, an operation method equivalent to that of the first embodiment is realized. The input device to be described will be described.

Hereinafter, the present embodiment will be described with reference to the drawings. In addition, in the following, the same reference numerals are assigned to the equivalents of the previous embodiment, and the description is omitted to avoid duplication. The input device 200 of the fourth embodiment is a so-called pointing device such as a mouse or a touchpad As shown in FIG. 11, motion information (pointing information and information indicating direction) is input to the system control unit 202 by the input unit 1100 as shown in FIG. Yes.

FIG. 12 shows the input unit 11 when the user operates the input device 100 in this embodiment.
The state of the input result obtained from 00 and the state of the GUI displayed on the display unit 101 are shown. A cursor 1200 is a cursor indicating the position of a specific coordinate on the screen based on the input result. In the present embodiment, the user can perform the same operation as in the first embodiment by handling the movement of the cursor 1200 equivalent to the movement of the user's hand in the first embodiment. Also,
When the input unit 1100 is configured by a sensor device such as an acceleration sensor, the cursor 120
By displaying the output information of the sensor device equivalent to the information on the movement of the user's hand in the first embodiment without displaying 0 on the display unit 101, the user performs the same operation as in the first embodiment. Can do.

As a result, the input device 100 according to the present embodiment allows the user to perform the same operation as in the conventional pointing operation using the same method as in the first embodiment, even when the input device 100 does not include the input unit such as the imaging unit 102. An operation for smoothly selecting a menu item can be performed simply by inputting a large movement in a specific direction to the input unit 1100 without any operation.

100 ... input device,
101 ... display unit,
102 ... Imaging unit,
103 ... user,
104 ... menu item,
105 ... selection item,
200: Motion detection (motion detection) unit,
201 ... system control unit,
202... Video processing unit,
301 ... recognition point,
302 ... Shaking hands,
303 ・ ・ ・ Return of hand,
701 ... Hand movement,
702 ... hand movement,
900 ... hand movement,
1100 ... input section,
1200 ... cursor,

Claims (10)

An input unit to which a predetermined motion image signal is input;
A motion detection unit that performs motion detection from the motion image signal input to the input unit;
A video signal processor that outputs a predetermined video signal when the motion detected by the motion detector is a predetermined motion;
A control unit,
The control unit controls the motion detection unit so that the motion detection unit does not perform motion detection when the predetermined video signal is output by the video signal processing unit. apparatus. The input device according to claim 1,
The control unit controls the motion detection unit so that the motion detection unit does not perform motion detection for a predetermined period after the predetermined video signal is output by the video signal processing unit. Input device. The input device according to claim 1,
The motion detection unit detects the speed of motion of the motion image signal input to the input unit,
The input unit, wherein the control unit controls the video signal processing unit so as to output a video of motion according to the detected speed of motion. The input device according to claim 1,
The motion detection unit detects the speed of motion of the motion image signal input to the input unit,
The input device according to claim 1, wherein the control unit controls the video signal processing unit to continuously output video when the magnitude of the detected speed of motion is a predetermined value or more. An input unit to which a predetermined motion image signal is input;
A motion detection unit that performs motion detection from the motion image signal input to the input unit;
A video signal processor that outputs a predetermined video signal when the motion detected by the motion detector is a predetermined motion;
A display unit for displaying an output from the video signal processing unit;
A control unit,
The control unit controls the motion detection unit so that the motion detection unit does not perform motion detection when the predetermined video signal is output by the video signal processing unit. apparatus. The display device according to claim 5,
The control unit controls the motion detection unit so that the motion detection unit does not perform motion detection for a predetermined period after the predetermined video signal is output by the video signal processing unit. Display device. The display device according to claim 5,
The motion detection unit detects the speed of motion of the motion image signal input to the input unit,
The display device, wherein the control unit controls the video signal processing unit so as to output a video of a motion corresponding to the speed of the detected motion. The display device according to claim 5,
The motion detection unit detects the speed of motion of the motion image signal input to the input unit,
The display device, wherein the control unit controls the video signal processing unit to continuously output video when the magnitude of the speed of the detected motion is equal to or greater than a predetermined value. An input unit for inputting predetermined motion information;
A video signal processing unit that outputs a predetermined video signal when the motion input by the input unit is predetermined motion information;
A control unit,
The control unit controls the motion detection unit so that the input unit does not input motion information when the predetermined video signal is output by the video signal processing unit. Input device. The input device according to claim 9, wherein
The input device, wherein the predetermined motion information includes at least one of pointing information and information indicating a direction.

Images