JP2010237765A - Information processing apparatus, focus movement control method, and focus movement control program - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To provide an information processing apparatus, a focus movement control method and a focus movement control program for smoothly performing an operation on a display screen by the gesture of a user photographed by a camera. <P>SOLUTION: In an information processing apparatus, an image is obtained through a camera, and whether or not a first hand shape matched with one or more hand shapes stored in a storage means is included in the image is detected, and when it is detected that the first hand shape is included in the image, the movement of a focus to a plurality of selection target objects is controlled based on the movement of the first hand shape. Then, when the first hand shape is moved by a prescribed distance or more with the base point as a center in a horizontal axial or vertical axial direction in the image, the focus is moved to the moving direction. <P>COPYRIGHT: (C)2011,JPO&INPIT

Description

  The present invention generally relates to an information processing apparatus, a focus movement control method, and a focus movement control program. More specifically, the present invention relates to an information processing apparatus, a focus movement control method, and a focus movement control program that recognize a user's gesture and perform control based on the recognized gesture.

  Conventionally, a method of operating an information processing apparatus such as a television or a personal computer (PC) by a user's gesture has been proposed. According to such a method, the information processing apparatus can be remotely operated without using an input device such as a mouse, a keyboard, or a remote controller.

  For example, there is a technique for detecting a movement of a recognition target object in a recognition area set in a shooting area by a camera, determining a movement category of the recognition target object based on the detected information, and executing a command corresponding to the category. It has been proposed (see Patent Document 1).

  In addition, by moving the user's specific hand shape, the cursor is moved on the display screen in correspondence with the specific hand shape photographed using the camera, and the virtual button displayed on the display screen is moved. The technique to select is proposed (refer patent documents 2 and 3).

JP 2007-34525 A JP-A-8-44490 JP 2004-78977 A

  In the above-described technique, it is necessary to recognize the user's gesture photographed by the camera, but due to the poor recognition accuracy, blurring occurs in the pointing (cursor on the display screen) due to the gesture, and the blur is caused for the user. It can be annoying.

  The present invention has been made in view of the above circumstances, and an information processing apparatus, a focus movement control method, and a focus that can smoothly perform an operation on a display screen by a user's gesture photographed by a camera. An object is to provide a movement control program.

  In order to solve the above-described problem, according to one aspect of the present invention, an information processing apparatus having a display, a storage unit storing a camera connected to the information processing apparatus and one or more hand shapes. And a hand shape recognition unit that detects whether or not a first hand shape that matches one or more hand shapes stored in the storage means is included in an image photographed by the camera; The hand shape recognition unit detects that the first hand shape that matches one or more hand shapes stored in the storage means is included in the image taken by the camera. A focus movement control unit that controls the movement of the focus with respect to the plurality of selection target objects displayed on the display based on the movement of the first hand shape, and the focus movement control Is provided with an information processing apparatus that moves the focus in the moving direction when the first hand shape moves more than a predetermined distance around the base point in the horizontal or vertical axis direction in the image. .

  According to another aspect of the present invention, in an information processing apparatus having a display and a storage unit that stores one or more hand shapes, focus movement with respect to a plurality of selection target objects displayed on the display is performed. A first hand shape that matches one or more hand shapes stored in the storage unit, wherein an image is acquired via a camera connected to the information processing apparatus. Is included in the image taken by the camera, and the first hand shape that matches one or more hand shapes stored in the storage means is detected by the camera. When it is detected that the image is included in the captured image, the focus is moved with respect to the plurality of selection target objects based on the movement of the first hand shape. If the first hand shape in the horizontal axis or the vertical axis direction in the image has moved a predetermined distance or more around the base point, the focus movement control method for moving the focus relative to the direction that the movement is provided.

  Furthermore, according to another aspect of the present invention, an image is acquired via a camera connected to the information processing apparatus in an information processing apparatus having a display and a storage unit that stores one or more hand shapes. And a step of detecting whether or not a first hand shape that matches one or more hand shapes stored in the storage means is included in the image photographed by the camera; When it is detected that the first hand shape that matches one or more hand shapes stored in the storage means is included in the image taken by the camera, the first hand shape is detected. And a step of controlling the movement of the focus with respect to the plurality of selection target objects displayed on the display based on the movement of the hand shape, and the first hand shape in the horizontal axis or vertical axis direction in the image is Base point If you moved a predetermined distance or more centered, focus moving control program to move the focus to the direction that the movement is provided.

  According to the present invention, it is possible to provide an information processing apparatus, a focus movement control method, and a focus movement control program capable of smoothly performing an operation on a display screen by a user's gesture photographed by a camera.

1 is a perspective view showing an information processing apparatus according to an embodiment of the present invention. The block diagram which shows the system configuration | structure of the information processing apparatus of FIG. The block diagram which shows a part of system configuration | structure of the information processing apparatus of FIG. 1 in detail. The figure which shows the example of the hand shape used in this embodiment. 4 is a flowchart illustrating an example of the operation of the focus movement control unit in FIG. 3 in the information processing apparatus in FIG. 1. In the flowchart of FIG. 5, (a) illustrates a case where the first hand shape has not moved more than a predetermined distance with respect to the base point, and (b) illustrates a case where the first hand shape is predetermined with respect to the base point. The figure explaining the case where it has moved more than the distance. In the flowchart of FIG. 5, (a) is a diagram for explaining resetting of the base point in the first axial direction, and (b) is a diagram for explaining resetting of the base point in the second axial direction. The figure which shows an example of the transition of the screen accompanying a focus movement which concerns on this embodiment. The figure which shows an example of the user interface (UI) displayed on a screen when the hand shape image | photographed via the camera is detected. When a predetermined application is executed, a predetermined hand shape photographed via the camera 107 is detected by the hand shape recognition unit in FIG. 3, and the focus movement control unit in FIG. 3 first receives a signal indicating the detection. The figure which shows an example of UI when doing. The figure which shows a mode that the focus in a selection object (button) moves when a predetermined hand shape is moved. The figure which shows a mode that UI shown in FIG. 10 changes, when the detected hand shape changes from a 1st hand shape to a 2nd hand shape. When the predetermined hand shape is detected by the hand shape recognition unit in FIG. 3 and the focus movement control unit in FIG. 3 first receives a signal indicating the detection, the default UI displayed according to the application to be operated is displayed. It is a figure which shows an example, (a) is a figure which shows UI displayed by AV applications, such as DVD viewing, (b) is a figure which shows UI displayed by a television viewing application.

  Hereinafter, embodiments of the present invention will be described with reference to the drawings.

  First, with reference to FIG.1 and FIG.2, the structure of the information processing apparatus which concerns on one Embodiment of this invention is demonstrated. This information processing apparatus is realized as, for example, a notebook personal computer 100 that can be driven by a battery (hereinafter abbreviated as computer 100).

  FIG. 1 is a perspective view of the computer 100 with the display unit opened. The computer 100 includes a main unit 101 and a display unit 102.

  The display unit 102 incorporates a display device composed of an LCD (Liquid Crystal Display) 103. The display unit of the LCD 103 is disposed almost at the center of the display unit 102. A camera 107 is provided in the upper center portion of the display unit of the LCD 103.

  The display unit 102 is supported by the main unit 101, and is rotatably attached to the main unit 101 between an open position where the upper surface of the main unit 101 is exposed and a closed position covering the upper surface of the main unit 101. It has been.

  The main unit 101 has a thin box-shaped housing. On the upper surface of the main unit 101, a power button 104 for turning on / off the computer 100, a keyboard 105, a touch pad 106, and the like are arranged. ing.

  FIG. 2 is a block diagram illustrating a system configuration of the computer 100.

  As shown in FIG. 2, the computer 100 includes a CPU 201, a main memory 202, a north bridge 203, a graphics controller 204, an LCD 103, a VRAM 205, a south bridge 206, a USB controller 207, an IDE controller 208, a camera 107, a USB device 209, and a hard disk. Drive (HDD) 210, optical disk drive (ODD) 211, BIOS-ROM 212, power button 104, keyboard 105, touch pad 106, embedded controller / keyboard controller (EC / KBC) 213, power supply circuit 221, battery 222, and AC adapter 223 and the like.

  The CPU 201 is a processor that controls the overall operation of the computer 100. The CPU 201 executes the OS and various application programs loaded in the main memory 202. The OS and various application programs are stored in a magnetic disk storage medium (hard disk) or the like mounted on the HDD 210 and loaded from the storage medium into the main memory 202.

  The CPU 201 also executes a BIOS program 230 (hereinafter referred to as BIOS) stored in the BIOS-ROM 212. The BIOS-ROM 212 takes the form of a nonvolatile memory such as a flash EEPROM so that the program can be rewritten.

  The BIOS 230 is a program that controls various hardware components of the computer 100, and is read from the BIOS-ROM 212 when the computer 100 is activated.

  The north bridge 203 is a bridge device that connects the local bus of the CPU 201 and the south bridge 206. The north bridge 203 includes a memory controller that controls access to the main memory 202. The north bridge 203 has a function of communicating with the graphics controller 204 via an AGP (Accelerated Graphics Port) bus or the like.

  The graphics controller 204 is a controller that controls the LCD 103 used as a display monitor of the computer 100. The graphics controller 204 outputs a video signal corresponding to display data written in the VRAM 205 by the OS or application program to the LCD 103.

  The south bridge 206 controls each device on an LPC (Low Pin Count) bus and a PCI (Peripheral Component Interconnect) bus. The south bridge 206 includes a USB controller 207 for controlling the camera 107 and the USB device 209, and an IDE controller 208 for controlling the HDD 210 and the ODD 211.

  The camera 107 is configured by a CCD (Charge Coupled Device) image sensor or a CMOS (Complementary Metal Oxide Semiconductor) image sensor. The camera 107 captures an image (moving image) around the computer 100 and supplies the image to a hand shape recognition unit 301 described later. An image (moving image) captured by the camera 107 can be displayed on a display monitor of the computer 100. The frame rate of the image supplied by the camera 107 is, for example, 30 frames / second. In this embodiment, it is assumed that the camera 107 is a USB device compliant with the USB standard. The camera 107 may be an external type or a built-in type.

  The HDD 210 is a storage device having a hard disk controller and a magnetic disk storage medium. The magnetic disk storage medium stores various software including the OS and various data. The ODD 211 is a drive unit for driving a storage medium such as a DVD storing video content such as a DVD title or a CD storing music data.

  The EC / KBC 213 is a one-chip microcomputer in which an embedded controller (EC) for power management and a keyboard controller (KBC) for controlling the keyboard 105 and the touch pad 106 are integrated. The EC / KBC 213 is always turned on by the power from the power supply circuit 221 regardless of whether the computer 100 is turned on or off. The EC / KBC 213 turns on / off the computer 100 in cooperation with the power supply circuit 221 in response to the operation of the power button 104 by the user.

  The power supply circuit 221 uses the power from the battery 222 provided in the main unit 101 or the power from the external power source supplied via the AC adapter 223 under the control of the EC / KBC 213, Supply power to the device.

  FIG. 3 is a block diagram showing a part of the configuration of the computer 100 in more detail.

  As shown in FIG. 3, the image captured by the camera 107 is supplied to the hand shape recognition unit 301.

  The hand shape recognition unit 301 determines whether the image supplied from the camera 107 includes a hand shape that matches any one of hand shapes stored (registered) in advance in the hand shape database 302. . For example, the hand shape recognition unit 302 searches for one of hand shapes stored in advance in the hand shape database 302 from the image supplied from the camera 107.

  In the hand shape database 302, as shown in FIG. 4, a fist (five fingers are bent (FIG. 4A)) and a palm (five fingers are extended (FIG. 4B) )) And thumb-up signs (the state in which only the thumb is extended and the other four fingers are bent (FIG. 4C)) are stored in advance in association with corresponding identifiers.

  Here, the first hand shape (in this embodiment, the fist in FIG. 4A) is displayed on the display monitor and selects an icon (hereinafter referred to as a selection target object) that can be selected by the cross key. It is used for The second hand shape (in this embodiment, the thumb-up sign in FIG. 4C) is used to determine the selected selection target object as an execution target.

  The hand shape recognition unit 301 further calculates hand shape position information (coordinates) using a relative coordinate system two-dimensional coordinate system in which the base point position changes based on the passage of time or the movement of the hand shape. In the present embodiment, the hand shape recognition unit 301 uses the center of a rectangular area circumscribing the hand shape as position information (X, Y) of the hand shape in the relative coordinate system coordinate system. The position information of the hand shape is not limited to the center of the rectangular area circumscribing the hand shape, and may be, for example, the coordinates of the center of gravity of the hand shape.

  The focus movement control unit 303 receives the hand shape identifier detected by the hand shape recognition unit 301 and the position information of the hand shape from the hand shape recognition unit 301. Further, the focus movement control unit 303 selects a selection target object or determines the selected selection target object as an execution target according to the movement of the hand shape photographed by the camera. The operation of the focus movement control unit 303 will be described in detail below with reference to FIGS.

  In the present embodiment, the hand shape recognition unit 301 is realized by a dedicated hardware engine (LSI; not shown), and the focus movement control unit 303 is realized by software executed by the CPU 201 (FIG. 2). However, the hand shape recognition unit 301 and the focus movement control unit 303 may be realized by hardware or software. The software and hand shape database 302 are stored in, for example, the HDD 210 (FIG. 2).

  Next, the operation of the focus movement control unit 303 will be described with reference to FIGS. FIG. 5 is a flowchart illustrating an example of the operation of the focus movement control unit 303 in the computer 100. Here, the position of the hand shape is specified by the coordinates of the relative coordinate system two-dimensional coordinate system. That is, the hand shape moves in a two-dimensional space consisting of a horizontal direction (X-axis direction; first axial direction) and a vertical direction (Y-axis direction; second axial direction). Unless otherwise stated, attention is focused only on the movement in the lateral direction (X-axis direction).

  First, in step S <b> 501, the focus movement control unit 303 waits for the hand shape recognition unit 301 to detect the first hand shape (the fist of FIG. 4A) via the camera 107.

  When the fist is detected by the hand shape recognition unit 301 and the focus movement control unit 303 receives a signal indicating that the fist is detected (including an identifier corresponding to the fist and position information of the fist) (YES in step S501). The focus movement control unit 303 determines whether or not the hand shape has moved by a predetermined distance Lx (for example, 0.05 m) or more with respect to the base point of the two-dimensional coordinate system (step S502). In the following, it is assumed that the fist is continuously detected by the hand shape recognition unit 301 unless otherwise specified.

  If the fist has not moved more than the predetermined distance Lx with respect to the base point (NO in step S502), the operation returns to the process in step S502 again. On the other hand, if the fist has moved more than the predetermined distance Lx with respect to the base point (YES in step S502), the focus movement control unit 303 resets the base point (step S503). FIG. 6A shows the case where the fist has not moved more than the predetermined distance Lx with respect to the base point, and FIG. 6B shows the case where the fist has moved more than the predetermined distance Lx with respect to the base point. Yes.

  Here, the resetting of the base point in step S503 will be described with reference to FIG.

  FIG. 7A schematically shows a state of resetting the base point on the X axis when the fist moves more than a predetermined distance Lx in the X axis (first axis) direction. In this case, the focus movement control unit 303 sets, as a new base point in the X-axis direction, a second predetermined predetermined direction from the point immediately before the fist moves over the predetermined distance Lx in the X-axis to the direction in which the fist moves in the X-axis. Reset to a point that is a distance away. The second predetermined distance is preferably set to 1 × Lx or more and 2 × Lx or less, and is set to 1.8 × Lx in the present embodiment.

  On the other hand, FIG. 7B schematically shows a state of resetting the base point on the Y axis (second axis) when the fist moves more than a predetermined distance Lx in the X axis direction. In this case, the focus movement control unit 303 determines that the fist has moved more than the predetermined distance Lx in the X-axis direction from the point immediately before the fist moves more than the predetermined distance Lx in the X-axis as the new base point in the Y-axis direction. Reset to the point at which was detected.

  Returning to FIG. 5, next, the focus movement control unit 303 moves the base point as time elapses (step S504). The following two methods can be considered for the movement of the base point.

In the first method, when the gripping fist has not moved more than the predetermined distance Lx from the base point after a predetermined time (for example, 1 second) has elapsed from step S503, the focus movement control unit 303 determines from the elapsed time from step S503. Also, the point at which the fist is present before the predetermined time has elapsed is set as a new base point. That is, when the time elapsed from step S503 is T seconds and the predetermined time is T ₀ seconds (T ≧ T ₀ ), the focus movement control unit 303 is in the state where (T−T ₀ ) seconds have elapsed since step S503. The point that had a fist in the base is used as a new base point.

  In the second method, after step S503, the focus movement control unit 303 moves the X axis (first axis) in a direction in which the base point approaches the hand fist at a constant speed (eg, 0.002 m / sec) as time passes. ) Move the base point along. Therefore, the value of the base point Y-axis (second axis) does not change before and after step S504. When the value of the base point on the X-axis matches the value of the X-axis of the fist, the focus movement control unit 303 stops the movement of the base point.

  Next, in step S505, the focus movement control unit 303 waits for the hand shape recognition unit 301 to detect the second hand shape (the thumb-up sign in FIG. 4C) via the camera 107.

  When the thumb-up sign is detected by the hand shape recognition unit 301 and the focus movement control unit 303 receives a signal (including an identifier corresponding to the thumb-up sign and position information of the thumb-up sign) (see FIG. In step S505, YES), the focus movement control unit 303 determines the selected selection target object as an execution target (step S506), and the operation ends.

  On the other hand, if the thumb-up sign is not detected by the hand shape recognition unit 301 (NO in step S505), the operation returns to the process in step S502.

  In this operation, the first method of step S504 has been described as being executed when the fist has not moved more than the predetermined distance Lx from the base point after a predetermined time has elapsed from step S503. However, in practice, the first method in step S504 is executed even when the fist has not moved more than the predetermined distance Lx from the base point after a predetermined time has elapsed from step S501. Further, although step S505 has been described as being executed after step S504, in practice, it is executed at any timing after the grip fist is detected in step S501.

  FIG. 8 shows an example of screen transition accompanied by focus movement according to the present embodiment. As shown in screens 801 to 804 in FIG. 8, in this example, there are ten selection target objects represented by A to J.

  At the stage shown on the screen 801, the selection target object B is focused.

  Next, when the hand shape recognition unit 301 (FIG. 3) detects the fist via the camera 107 (FIG. 1) and further moves the fist to the right by a predetermined distance Lx (YES in step S502 of FIG. 5). ) As shown in the screen 802, the focus moves to the selection target object C.

  Similarly, when the fist is moved rightward, the focus moves to the selection target object D as shown in the screen 803. At this time, the value in the vertical direction (Y-axis direction) of the base point is reset to the value when the focus moves to the selection target object D (step S503 in FIG. 5).

  Accordingly, when the next fist is moved downward by a predetermined distance Ly (shown in FIG. 7B) or more (YES in step S502 in FIG. 5), the selection target is displayed as shown in the screen 804. The focus moves to the object I.

  As described above, the focus movement control unit 303 (FIG. 3) detects that the grip fist is moved in a direction of up, down, left, and right by a predetermined distance or more, so that the focus movement control unit 303 (FIG. 3) has exceeded the predetermined distance. As a result, focus movement corresponding to the detected direction occurs with respect to the selection target object displayed on the screen.

  FIG. 9 shows an example of a user interface (UI) 900 displayed on the screen when a hand shape (grip fist) photographed via the camera 107 is detected.

  When a fist is detected by the hand shape recognition unit 301 (FIG. 3) via the camera 107 and the focus movement control unit 303 (FIG. 3) receives a signal indicating the detection (YES in step S501 of FIG. 5), As shown in FIG. 9, a grip fist icon corresponding to the hand shape is displayed. When this icon is displayed for the first time, it is displayed superimposed on a predetermined selection target object (for example, the selection target object A in the case of the screen shown in FIG. 8). Further, this icon moves between the selection target objects according to the flowchart shown in FIG. Note that the arrow around the clenched fist icon indicates that the focus can be moved in that direction. Therefore, by using the hand-shaped icon shown in this example, smooth movement of the focus on the selection target object is realized without reacting to an unintended movement (blur) of the hand shape that is the recognition target object. It becomes possible.

  According to the present embodiment, when the hand shape is moved by a predetermined distance or more, the new base point is a point that is 1 to 2 times the predetermined distance from the immediately preceding base point in the direction in which the hand shape has moved. By resetting, it is possible to suppress the movement of the focus due to the shake caused by the poor recognition accuracy or the shake of the user's hand (according to step S503 in FIG. 5).

  Further, according to the present embodiment, when the focus is moved in the direction intended by the user, even if the focus is actually moved slightly in the vertical direction, the movement of the focus in another axis direction not intended by the user is suppressed. (According to step S503 in FIG. 5).

  Further, according to the present embodiment, even if the user moves the hand slowly without intention, the movement of the focus can be suppressed (according to step S504 in FIG. 5).

(Application form)
Below, the form which applied embodiment mentioned above is demonstrated, referring drawings.

  FIG. 10 shows a hand shape recognition unit 301 (FIG. 3) showing a predetermined hand shape (in this embodiment, a fist) photographed via the camera 107 when executing a predetermined application (AV application in this embodiment). 3 shows an example of the UI 1000 when the focus movement control unit 303 (FIG. 3) first receives a signal indicating the detection (YES in step S501 in FIG. 5). In the UI 1000 shown in FIG. 10, 1 × 6 matrix buttons are arranged, but in the displayed UI, N × M (N, M: natural numbers) two-dimensional matrix buttons are arranged. Can be done.

  In relation to the above, FIG. 13 shows a case where a fist is detected by the hand shape recognition unit 301 in FIG. 3 and the focus movement control unit 303 in FIG. 3 first receives a signal indicating the detection (step S501 in FIG. 5). YES), an example of a default UI displayed in accordance with the operation target application is shown.

  Here, FIG. 13A is a diagram showing a UI 1301 displayed by an AV application such as DVD or CD viewing, and FIG. 13B is a diagram showing a UI 1302 displayed by a television viewing application. For example, in the Windows (registered trademark) OS series of Microsoft Corporation, these applications and the corresponding UI to be displayed are associated with each other and stored in advance in a registry, for example.

  Next, FIG. 11 shows how the focus of the selection target object (button) moves when a predetermined hand shape (grip fist) is moved with respect to the UI 1000 shown in FIG.

  FIG. 11A shows a detection of a fist photographed through the camera 107 by the hand shape recognition unit 301 in FIG. 3 when the AV application is executed, and the focus movement control unit 303 in FIG. This is the UI 1101 when the signal is first received (YES in step S501 in FIG. 5). At this stage, the media is played and the fourth pause button from the left is focused.

  Next, when the detected fist is moved in the right direction and the movement distance becomes a predetermined distance or more (YES in step S502 in FIG. 5), as shown in FIG. The button is moved forward and the focus is on the button that advances the chapter in the DVD title or the song in the CD music by one.

  Similarly, when the detected fist is moved in the right direction, as shown in FIG. 11B, the focus is also moved right by one, and the fast-forward button is focused.

  FIG. 12 shows how the UI 1000 shown in FIG. 10 changes when the detected hand shape changes from the first hand shape (grip fist) to the second hand shape (thumb up sign). .

  FIG. 12A shows the detection of a fist photographed via the camera 107 by the hand shape recognition unit 301 in FIG. 3 when the AV application is executed, and the focus movement control unit 303 in FIG. This is the UI 1201 when the signal is first received (YES in step S501 in FIG. 5). At this stage, the media is played and the fourth pause button from the left is focused.

  Here, it is assumed that the second hand shape (thum-up sign) is detected by the hand shape recognition unit 301 in FIG. 3 via the camera 107 and the focus movement control unit 303 in FIG. 3 receives a signal indicating the detection. (YES in step S505 in FIG. 5). Then, the focus movement control unit 303 in FIG. 3 determines the execution target as the focused pause button. Next, it is indicated that the focused pause button (the hatched portion of the UI 1202 in FIG. 12) has been determined as an execution target because its color has been changed or its size has increased. In response to the determination, the CPU 201 (FIG. 2) pauses the media playback. Thereafter, the button at the position where the pause button exists is changed to a button (not shown) for releasing the pause and playing again, and the UI in a state where the button is focused is displayed.

  Further, the above-described UI 1000 or the like disappears from the screen after a predetermined time (for example, 1 second) has elapsed since the hand shape stored in the hand shape database 302 is no longer detected.

  According to the present embodiment, the operation target application and the displayed corresponding UI are associated with each other, so that an operation intended by the user can be performed quickly.

  In addition, according to the present embodiment, since the recognition target hand shape does not disappear until a predetermined time elapses, for example, even if the detection of the hand shape that the user has put out due to erroneous detection of the hand shape is interrupted for a while, The UI can be operated continuously.

  Further, according to the present embodiment, since the hand shape to be recognized does not disappear until a predetermined time elapses, for example, even when the user cannot fully move the focus by moving his hand, By returning to a position with a margin, the UI can be continuously operated.

  The preferred embodiment of the present invention has been described above, but the present invention is not limited to the above-described embodiment as it is, and can be embodied by modifying the constituent elements without departing from the scope of the present invention.

  For example, the present invention can be applied to various information processing apparatuses such as a desktop personal computer, a mobile phone, a personal digital assistant (PDA), a game machine, and a television in addition to the notebook personal computer described in this embodiment. can do.

  In addition, the processing described in the above-described embodiment is performed on a recording medium such as a magnetic disk (flexible disk, hard disk, etc.), an optical disk (CD-ROM, DVD, etc.), a semiconductor memory, etc. as a program that can be executed by a computer. It can be written and provided to various devices. It is also possible to transmit to a variety of devices by transmitting via a communication medium. The computer reads the program recorded on the recording medium or receives the program via the communication medium, and the operation is controlled by this program, thereby executing the above-described processing.

  As described above, according to the present invention, it is possible to provide an information processing apparatus, a focus control movement method, and a focus movement control program capable of smoothly performing an operation on a display screen by a user's gesture photographed by a camera. it can.

100 Computer 101 Main Unit 102 Display Unit 103 LCD
104 Power button 105 Keyboard 106 Touchpad 107 Camera 201 CPU
202 Main memory 203 North bridge 206 South bridge 210 HDD
211 ODD
212 BIOS-ROM
213 EC / KBC
230 BIOS
301 hand shape recognition unit 302 hand shape database 303 focus movement control unit

Claims (19)

An information processing apparatus having a display,
A camera connected to the information processing apparatus;
Storage means for storing one or more hand shapes;
A hand shape recognition unit for detecting whether or not a first hand shape that matches one or more hand shapes stored in the storage means is included in an image photographed by the camera;
The hand shape recognition unit detects that the first hand shape that matches one or more hand shapes stored in the storage means is included in the image taken by the camera. A focus movement control unit that controls the movement of the focus with respect to the plurality of selection target objects displayed on the display based on the movement of the first hand shape,
The focus movement control unit moves the focus in the moving direction when the first hand shape moves more than a predetermined distance around the base point in the horizontal axis or vertical axis direction in the image. A characteristic information processing apparatus.   Using the position of the first hand shape first detected by the hand shape recognition unit as a first base point, the focus movement control unit can determine whether the first hand shape is in the horizontal axis or vertical axis direction in the image. 2. The information processing apparatus according to claim 1, wherein the focus moves in the moving direction when the first reference point is moved more than the predetermined distance. 3.   The focus movement control unit moves from the position of the first base point by 1 to 2 times the predetermined distance with respect to a direction in which the focus is moved, and in a direction perpendicular to the direction in which the focus is moved. The information processing apparatus according to claim 2, wherein the position moved to the point of the first hand shape when the focus is moved is reset as the second base point.   After moving the focus, the focus movement control unit moves the focus from the position of the base point immediately before the first hand shape moves more than the predetermined distance to the direction in which the focus is moved. The position moved to the point of the first hand shape when the focus is moved with respect to the direction perpendicular to the direction in which the focus is moved is re-used as a new base point. The information processing apparatus according to claim 1, wherein the information processing apparatus is set.   A second hand shape that matches one or more hand shapes stored in the storage means by the hand shape recognition unit, and is different from the first hand shape, When it is detected that it is included in the image photographed by the camera, the focus movement control unit, among the plurality of selection target objects selected when the second hand shape is detected The information processing apparatus according to claim 1, wherein one of the two is determined as an execution target.   The focus movement control unit, after the second base point is reset, when the first hand shape has not moved more than the predetermined distance from the second base point after a first predetermined time, The position of the base point is moved to the position of the first hand shape before the first predetermined time has elapsed from the time elapsed after the second base point has been reset. Information processing device.   When the first base shape has not moved more than the predetermined distance from the new base point after the first predetermined time has elapsed after the new base point has been reset, 5. The position of the base point is moved to the position of the first hand shape before the first predetermined time has elapsed from the time elapsed after the new base point is reset. Information processing device.   The corresponding user interface associated with a target application is displayed on the display when the first hand shape is first detected by the hand shape recognition unit. Information processing device.   9. The information processing according to claim 8, wherein the user interface is not displayed on the display after a second predetermined time has elapsed since the first hand shape is no longer detected by the hand shape recognition unit. apparatus.   When the first hand shape is first detected by the hand shape recognition unit, an icon indicating that the first hand shape has been detected is selected from among the predetermined selection target objects. The information processing apparatus according to claim 1, wherein the information processing apparatuses are displayed on the display so as to overlap each other. A focus movement control method for controlling movement of a focus with respect to a plurality of selection target objects displayed on the display in an information processing apparatus having a display and a storage unit storing one or more hand shapes,
Obtaining an image via a camera connected to the information processing apparatus;
Detecting whether or not a first hand shape that matches one or more hand shapes stored in the storage means is included in the image taken by the camera;
When it is detected that the first hand shape that matches one or more hand shapes stored in the storage means is included in the image taken by the camera, the first hand shape is detected. Based on the movement of the hand shape, the focus is moved to the plurality of selection target objects,
A focus movement control method, wherein when the first hand shape moves more than a predetermined distance around a base point in the horizontal axis or vertical axis direction in the image, the focus is moved in the moved direction. .   With the position of the first hand shape first detected by the hand shape recognition unit as the first base point, the first hand shape is centered on the first base point in the horizontal axis or vertical axis direction in the image. The focus movement control method according to claim 11, wherein when the movement is greater than the predetermined distance, the focus is moved in the moved direction.   The focus is moved in the direction perpendicular to the direction in which the focus is moved from the position of the first base point by moving the focus by 1 to 2 times the predetermined distance. The focus movement control method according to claim 12, wherein the position moved to the point of the first hand shape at the time is reset as a second base point.   After the focus is moved, the first hand shape is moved from the position of the base point just before the predetermined distance moves more than the predetermined distance to 1 to 2 times the predetermined distance in the direction in which the focus is moved. The position moved to the first hand-shaped point when the focus is moved with respect to the direction perpendicular to the direction in which the focus is moved is reset as a new base point. The focus movement control method according to claim 11.   The second hand shape that matches one or more hand shapes stored in the storage means, and a second hand shape different from the first hand shape is taken by the camera. When it is detected that the second hand shape is detected, one of the plurality of selection target objects selected when the second hand shape is detected is determined as an execution target. The focus movement control method according to claim 11.   If the first hand shape has not moved more than the predetermined distance from the second base point after the first predetermined time has elapsed after the second base point has been reset, the position of the base point is changed to the first base point. 14. The focus movement control method according to claim 13, wherein the focus movement control method moves to the position of the first hand shape before the first predetermined time elapses after the time elapsed after the two base points are reset.   After the new base point is reset, if the first hand shape has not moved more than the predetermined distance from the new base point after a first predetermined time has elapsed, the position of the base point is changed to the new base point. 15. The focus movement control method according to claim 14, wherein the focus movement control method moves to the position of the first hand shape before the first predetermined time elapses after the time elapses after the reset base point is reset.   12. The focus movement control method according to claim 11, wherein when the first hand shape is first detected, a corresponding user interface associated with a target application is displayed on the display. In an information processing apparatus having a display and storage means storing one or more hand shapes,
A procedure for acquiring an image via a camera connected to the information processing apparatus;
Detecting whether or not a first hand shape that matches one or more hand shapes stored in the storage means is included in the image taken by the camera;
When it is detected that the first hand shape that matches one or more hand shapes stored in the storage means is included in the image taken by the camera, the first hand shape is detected. Controlling the movement of the focus with respect to the plurality of selection target objects displayed on the display based on the movement of the hand shape, and
A focus movement control program for moving the focus in the moving direction when the first hand shape moves more than a predetermined distance centering on a base point in the horizontal axis or vertical axis direction in the image .

Images