JP5295839B2 - Information processing apparatus, focus movement control method, and focus movement control program - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To provide an information processing apparatus, a focus movement control method and a focus movement control program for smoothly performing an operation on a display screen by the gesture of a user photographed by a camera. <P>SOLUTION: In an information processing apparatus, an image is obtained through a camera, and whether or not a first hand shape matched with one or more hand shapes stored in a storage means is included in the image is detected, and when it is detected that the first hand shape is included in the image, the movement of a focus to a plurality of selection target objects is controlled based on the movement of the first hand shape. Then, when the first hand shape is moved by a prescribed distance or more with the base point as a center in a horizontal axial or vertical axial direction in the image, the focus is moved to the moving direction. <P>COPYRIGHT: (C)2011,JPO&amp;INPIT

Description

  The present invention generally relates to an information processing apparatus, a focus movement control method, and a focus movement control program. More specifically, the present invention relates to an information processing apparatus, a focus movement control method, and a focus movement control program that recognize a user's gesture and perform control based on the recognized gesture.

  Conventionally, a method of operating an information processing apparatus such as a television or a personal computer (PC) by a user's gesture has been proposed. According to such a method, the information processing apparatus can be remotely operated without using an input device such as a mouse, a keyboard, or a remote controller.

  For example, there is a technique for detecting a movement of a recognition target object in a recognition area set in a shooting area by a camera, determining a movement category of the recognition target object based on the detected information, and executing a command corresponding to the category. It has been proposed (see Patent Document 1).

  In addition, by moving the user's specific hand shape, the cursor is moved on the display screen in correspondence with the specific hand shape photographed using the camera, and the virtual button displayed on the display screen is moved. The technique to select is proposed (refer patent documents 2 and 3).

JP 2007-34525 A JP-A-8-44490 JP 2004-78977 A

  In the above-described technique, it is necessary to recognize the user's gesture photographed by the camera, but due to the poor recognition accuracy, blurring occurs in the pointing (cursor on the display screen) due to the gesture, and the blur is caused for the user. It can be annoying.

  The present invention has been made in view of the above circumstances, and an information processing apparatus, a focus movement control method, and a focus that can smoothly perform an operation on a display screen by a user's gesture photographed by a camera. An object is to provide a movement control program.

In order to solve the above-described problem, according to one aspect of the present invention, a focus for controlling a display and a focus for selecting one of a plurality of selection target objects displayed on the display so as to move one object at a time. a movement control means, a camera connected to the information processing apparatus, based on the base position change at every predetermined time, the camera position detection means for detecting a continuous manner the position of the hand of the object in the image to be photographed When the position of the target hand moves more than a predetermined distance based on the detected position, the movement direction of the target hand is continuously determined every predetermined time, assuming that the target hand has moved. manner; and a moving direction detecting means for detecting a first detected position of the hand as the first base position of the base position by the position detection means, the focus Dynamic control means, when the position of the hand in the horizontal axis or the vertical axis direction in the image is moved the predetermined distance or more about said first base position, from the first base point position, and moving the focus A position moved to the point of the hand when the focus is moved with respect to a direction perpendicular to the direction in which the focus is moved, and the base point position An information processing apparatus that resets the second base position is provided.

According to another aspect of the present invention, an information processing apparatus having a display acquires an image via a camera connected to the information processing apparatus, and the position detection unit changes a base position every predetermined time. based on the camera is the position of the hand of the object in the image is detected continuous manner to shoot, movement direction detection means, based on the detected position, a distance position of the hand of the subject is given In the case of the above movement, assuming that the target hand has moved, the movement direction of the target hand is continuously detected every predetermined time, and the focus movement control means is based on the detected movement direction, while moving the focus to select one of a plurality of selection target objects displayed on the display for each object, the base point first detected position of the hand by the position detecting means When the position of the hand moves more than the predetermined distance around the first base position in the horizontal axis or vertical axis direction in the image as the first base position of the position, from the first base position, A position moved to the point of the hand when the focus is moved with respect to a direction perpendicular to the direction in which the focus is moved with respect to a direction perpendicular to the direction in which the focus is moved with respect to the direction in which the focus is moved, A focus movement control method for resetting as a second base point position among the base point positions is provided.

Furthermore, according to another aspect of the present invention, an information processing apparatus having a display is based on a procedure for acquiring an image via a camera connected to the information processing apparatus and a base point position that changes every predetermined time. , a step of the camera is the position of the target of the hand in the image detected in continuous manner to shoot, based on the detected position, if the position of the hand of the object has moved a predetermined distance or more, the target A plurality of selection target objects displayed on the display based on the procedure of continuously detecting the moving direction of the target hand every predetermined time and the detected moving direction. while moving the focus to select one for each object, the first detected position of the hand as the first base position of the base position by the position detecting means, wherein When the position of the hand in the horizontal axis or vertical axis direction in the image moves more than the predetermined distance around the first base point position, the predetermined position with respect to the direction in which the focus is moved from the first base point position. And a position moved to the point of the hand when the focus is moved with respect to a direction perpendicular to the direction in which the focus is moved as a second base point position of the base point positions. A focus movement control program for executing the resetting procedure is provided.

  According to the present invention, it is possible to provide an information processing apparatus, a focus movement control method, and a focus movement control program capable of smoothly performing an operation on a display screen by a user's gesture photographed by a camera.

1 is a perspective view showing an information processing apparatus according to an embodiment of the present invention. The block diagram which shows the system configuration | structure of the information processing apparatus of FIG. The block diagram which shows a part of system configuration | structure of the information processing apparatus of FIG. 1 in detail. The figure which shows the example of the hand shape used in this embodiment. 4 is a flowchart illustrating an example of the operation of the focus movement control unit in FIG. 3 in the information processing apparatus in FIG. 1. In the flowchart of FIG. 5, (a) illustrates a case where the first hand shape has not moved more than a predetermined distance with respect to the base point, and (b) illustrates a case where the first hand shape is predetermined with respect to the base point. The figure explaining the case where it has moved more than the distance. In the flowchart of FIG. 5, (a) is a diagram for explaining resetting of the base point in the first axial direction, and (b) is a diagram for explaining resetting of the base point in the second axial direction. The figure which shows an example of the transition of the screen accompanying a focus movement which concerns on this embodiment. The figure which shows an example of the user interface (UI) displayed on a screen when the hand shape image | photographed via the camera is detected. When a predetermined application is executed, a predetermined hand shape photographed via the camera 107 is detected by the hand shape recognition unit in FIG. 3, and the focus movement control unit in FIG. 3 first receives a signal indicating the detection. The figure which shows an example of UI when doing. The figure which shows a mode that the focus in a selection object (button) moves when a predetermined hand shape is moved. The figure which shows a mode that UI shown in FIG. 10 changes, when the detected hand shape changes from a 1st hand shape to a 2nd hand shape. When the predetermined hand shape is detected by the hand shape recognition unit in FIG. 3 and the focus movement control unit in FIG. 3 first receives a signal indicating the detection, the default UI displayed according to the application to be operated is displayed. It is a figure which shows an example, (a) is a figure which shows UI displayed by AV applications, such as DVD viewing, (b) is a figure which shows UI displayed by a television viewing application.

  Hereinafter, embodiments of the present invention will be described with reference to the drawings.

  First, with reference to FIG.1 and FIG.2, the structure of the information processing apparatus which concerns on one Embodiment of this invention is demonstrated. This information processing apparatus is realized as, for example, a notebook personal computer 100 that can be driven by a battery (hereinafter abbreviated as computer 100).

  FIG. 1 is a perspective view of the computer 100 with the display unit opened. The computer 100 includes a main unit 101 and a display unit 102.

  The display unit 102 incorporates a display device composed of an LCD (Liquid Crystal Display) 103. The display unit of the LCD 103 is disposed almost at the center of the display unit 102. A camera 107 is provided in the upper center portion of the display unit of the LCD 103.

  The display unit 102 is supported by the main unit 101, and is rotatably attached to the main unit 101 between an open position where the upper surface of the main unit 101 is exposed and a closed position covering the upper surface of the main unit 101. It has been.

  The main unit 101 has a thin box-shaped housing. On the upper surface of the main unit 101, a power button 104 for turning on / off the computer 100, a keyboard 105, a touch pad 106, and the like are arranged. ing.

  FIG. 2 is a block diagram illustrating a system configuration of the computer 100.

  As shown in FIG. 2, the computer 100 includes a CPU 201, a main memory 202, a north bridge 203, a graphics controller 204, an LCD 103, a VRAM 205, a south bridge 206, a USB controller 207, an IDE controller 208, a camera 107, a USB device 209, and a hard disk. Drive (HDD) 210, optical disk drive (ODD) 211, BIOS-ROM 212, power button 104, keyboard 105, touch pad 106, embedded controller / keyboard controller (EC / KBC) 213, power supply circuit 221, battery 222, and AC adapter 223 and the like.

  The CPU 201 is a processor that controls the overall operation of the computer 100. The CPU 201 executes the OS and various application programs loaded in the main memory 202. The OS and various application programs are stored in a magnetic disk storage medium (hard disk) or the like mounted on the HDD 210 and loaded from the storage medium into the main memory 202.

  The CPU 201 also executes a BIOS program 230 (hereinafter referred to as BIOS) stored in the BIOS-ROM 212. The BIOS-ROM 212 takes the form of a nonvolatile memory such as a flash EEPROM so that the program can be rewritten.

  The BIOS 230 is a program that controls various hardware components of the computer 100, and is read from the BIOS-ROM 212 when the computer 100 is activated.

  The north bridge 203 is a bridge device that connects the local bus of the CPU 201 and the south bridge 206. The north bridge 203 includes a memory controller that controls access to the main memory 202. The north bridge 203 has a function of communicating with the graphics controller 204 via an AGP (Accelerated Graphics Port) bus or the like.

  The graphics controller 204 is a controller that controls the LCD 103 used as a display monitor of the computer 100. The graphics controller 204 outputs a video signal corresponding to display data written in the VRAM 205 by the OS or application program to the LCD 103.

  The south bridge 206 controls each device on an LPC (Low Pin Count) bus and a PCI (Peripheral Component Interconnect) bus. The south bridge 206 includes a USB controller 207 for controlling the camera 107 and the USB device 209, and an IDE controller 208 for controlling the HDD 210 and the ODD 211.

  The camera 107 is configured by a CCD (Charge Coupled Device) image sensor or a CMOS (Complementary Metal Oxide Semiconductor) image sensor. The camera 107 captures an image (moving image) around the computer 100 and supplies the image to a hand shape recognition unit 301 described later. An image (moving image) captured by the camera 107 can be displayed on a display monitor of the computer 100. The frame rate of the image supplied by the camera 107 is, for example, 30 frames / second. In this embodiment, it is assumed that the camera 107 is a USB device compliant with the USB standard. The camera 107 may be an external type or a built-in type.

  The HDD 210 is a storage device having a hard disk controller and a magnetic disk storage medium. The magnetic disk storage medium stores various software including the OS and various data. The ODD 211 is a drive unit for driving a storage medium such as a DVD storing video content such as a DVD title or a CD storing music data.

  The EC / KBC 213 is a one-chip microcomputer in which an embedded controller (EC) for power management and a keyboard controller (KBC) for controlling the keyboard 105 and the touch pad 106 are integrated. The EC / KBC 213 is always turned on by the power from the power supply circuit 221 regardless of whether the computer 100 is turned on or off. The EC / KBC 213 turns on / off the computer 100 in cooperation with the power supply circuit 221 in response to the operation of the power button 104 by the user.

  The power supply circuit 221 uses the power from the battery 222 provided in the main unit 101 or the power from the external power source supplied via the AC adapter 223 under the control of the EC / KBC 213, Supply power to the device.

  FIG. 3 is a block diagram showing a part of the configuration of the computer 100 in more detail.

  As shown in FIG. 3, the image captured by the camera 107 is supplied to the hand shape recognition unit 301.

  The hand shape recognition unit 301 determines whether the image supplied from the camera 107 includes a hand shape that matches any one of hand shapes stored (registered) in advance in the hand shape database 302. . For example, the hand shape recognition unit 302 searches for one of hand shapes stored in advance in the hand shape database 302 from the image supplied from the camera 107.

  In the hand shape database 302, as shown in FIG. 4, a fist (five fingers are bent (FIG. 4A)) and a palm (five fingers are extended (FIG. 4B) )) And thumb-up signs (the state in which only the thumb is extended and the other four fingers are bent (FIG. 4C)) are stored in advance in association with corresponding identifiers.

  Here, the first hand shape (in this embodiment, the fist in FIG. 4A) is displayed on the display monitor and selects an icon (hereinafter referred to as a selection target object) that can be selected by the cross key. It is used for The second hand shape (in this embodiment, the thumb-up sign in FIG. 4C) is used to determine the selected selection target object as an execution target.

  The hand shape recognition unit 301 further calculates hand shape position information (coordinates) using a relative coordinate system two-dimensional coordinate system in which the base point position changes based on the passage of time or the movement of the hand shape. In the present embodiment, the hand shape recognition unit 301 uses the center of a rectangular area circumscribing the hand shape as position information (X, Y) of the hand shape in the relative coordinate system coordinate system. The position information of the hand shape is not limited to the center of the rectangular area circumscribing the hand shape, and may be, for example, the coordinates of the center of gravity of the hand shape.

  The focus movement control unit 303 receives the hand shape identifier detected by the hand shape recognition unit 301 and the position information of the hand shape from the hand shape recognition unit 301. Further, the focus movement control unit 303 selects a selection target object or determines the selected selection target object as an execution target according to the movement of the hand shape photographed by the camera. The operation of the focus movement control unit 303 will be described in detail below with reference to FIGS.

  In the present embodiment, the hand shape recognition unit 301 is realized by a dedicated hardware engine (LSI; not shown), and the focus movement control unit 303 is realized by software executed by the CPU 201 (FIG. 2). However, the hand shape recognition unit 301 and the focus movement control unit 303 may be realized by hardware or software. The software and hand shape database 302 are stored in, for example, the HDD 210 (FIG. 2).

  Next, the operation of the focus movement control unit 303 will be described with reference to FIGS. FIG. 5 is a flowchart illustrating an example of the operation of the focus movement control unit 303 in the computer 100. Here, the position of the hand shape is specified by the coordinates of the relative coordinate system two-dimensional coordinate system. That is, the hand shape moves in a two-dimensional space consisting of a horizontal direction (X-axis direction; first axial direction) and a vertical direction (Y-axis direction; second axial direction). Unless otherwise stated, attention is focused only on the movement in the lateral direction (X-axis direction).

  First, in step S <b> 501, the focus movement control unit 303 waits for the hand shape recognition unit 301 to detect the first hand shape (the fist of FIG. 4A) via the camera 107.

  When the fist is detected by the hand shape recognition unit 301 and the focus movement control unit 303 receives a signal indicating that the fist is detected (including an identifier corresponding to the fist and position information of the fist) (YES in step S501). The focus movement control unit 303 determines whether or not the hand shape has moved by a predetermined distance Lx (for example, 0.05 m) or more with respect to the base point of the two-dimensional coordinate system (step S502). In the following, it is assumed that the fist is continuously detected by the hand shape recognition unit 301 unless otherwise specified.

  If the fist has not moved more than the predetermined distance Lx with respect to the base point (NO in step S502), the operation returns to the process in step S502 again. On the other hand, if the fist has moved more than the predetermined distance Lx with respect to the base point (YES in step S502), the focus movement control unit 303 resets the base point (step S503). FIG. 6A shows the case where the fist has not moved more than the predetermined distance Lx with respect to the base point, and FIG. 6B shows the case where the fist has moved more than the predetermined distance Lx with respect to the base point. Yes.

  Here, the resetting of the base point in step S503 will be described with reference to FIG.

  FIG. 7A schematically shows a state of resetting the base point on the X axis when the fist moves more than a predetermined distance Lx in the X axis (first axis) direction. In this case, the focus movement control unit 303 sets, as a new base point in the X-axis direction, a second predetermined predetermined direction from the point immediately before the fist moves over the predetermined distance Lx in the X-axis to the direction in which the fist moves in the X-axis. Reset to a point that is a distance away. The second predetermined distance is preferably set to 1 × Lx or more and 2 × Lx or less, and is set to 1.8 × Lx in the present embodiment.

  On the other hand, FIG. 7B schematically shows a state of resetting the base point on the Y axis (second axis) when the fist moves more than a predetermined distance Lx in the X axis direction. In this case, the focus movement control unit 303 determines that the fist has moved more than the predetermined distance Lx in the X-axis direction from the point immediately before the fist moves more than the predetermined distance Lx in the X-axis as the new base point in the Y-axis direction. Reset to the point at which was detected.

  Returning to FIG. 5, next, the focus movement control unit 303 moves the base point as time elapses (step S504). The following two methods can be considered for the movement of the base point.

In the first method, when the gripping fist has not moved more than the predetermined distance Lx from the base point after a predetermined time (for example, 1 second) has elapsed from step S503, the focus movement control unit 303 determines from the elapsed time from step S503. Also, the point at which the fist is present before the predetermined time has elapsed is set as a new base point. That is, assuming that the time elapsed from step S503 is T seconds and the predetermined time is T ₀ seconds (T ≧ T ₀ ), the focus movement control unit 303 has detected that (T−T ₀ ) seconds have elapsed from step S503. The point that had a fist in the base is used as a new base point.

  In the second method, after step S503, the focus movement control unit 303 moves the X axis (first axis) in a direction in which the base point approaches the hand fist at a constant speed (eg, 0.002 m / sec) as time passes. ) Move the base point along. Therefore, the value of the base point Y-axis (second axis) does not change before and after step S504. When the value of the base point on the X-axis matches the value of the X-axis of the fist, the focus movement control unit 303 stops the movement of the base point.

  Next, in step S505, the focus movement control unit 303 waits for the hand shape recognition unit 301 to detect the second hand shape (the thumb-up sign in FIG. 4C) via the camera 107.

  When the thumb-up sign is detected by the hand shape recognition unit 301 and the focus movement control unit 303 receives a signal (including an identifier corresponding to the thumb-up sign and position information of the thumb-up sign) (see FIG. In step S505, YES), the focus movement control unit 303 determines the selected selection target object as an execution target (step S506), and the operation ends.

  On the other hand, if the thumb-up sign is not detected by the hand shape recognition unit 301 (NO in step S505), the operation returns to the process in step S502.

  In this operation, the first method of step S504 has been described as being executed when the fist has not moved more than the predetermined distance Lx from the base point after a predetermined time has elapsed from step S503. However, in practice, the first method in step S504 is executed even when the fist has not moved more than the predetermined distance Lx from the base point after a predetermined time has elapsed from step S501. Further, although step S505 has been described as being executed after step S504, in practice, it is executed at any timing after the grip fist is detected in step S501.

  FIG. 8 shows an example of screen transition accompanied by focus movement according to the present embodiment. As shown in screens 801 to 804 in FIG. 8, in this example, there are ten selection target objects represented by A to J.

  At the stage shown on the screen 801, the selection target object B is focused.

  Next, when the hand shape recognition unit 301 (FIG. 3) detects the fist via the camera 107 (FIG. 1) and further moves the fist to the right by a predetermined distance Lx (YES in step S502 of FIG. 5). ) As shown in the screen 802, the focus moves to the selection target object C.

  Similarly, when the fist is moved rightward, the focus moves to the selection target object D as shown in the screen 803. At this time, the value in the vertical direction (Y-axis direction) of the base point is reset to the value when the focus moves to the selection target object D (step S503 in FIG. 5).

  Accordingly, when the next fist is moved downward by a predetermined distance Ly (shown in FIG. 7B) or more (YES in step S502 in FIG. 5), the selection target is displayed as shown in the screen 804. The focus moves to the object I.

  As described above, the focus movement control unit 303 (FIG. 3) detects that the grip fist is moved in a direction of up, down, left, and right by a predetermined distance or more, so that the focus movement control unit 303 (FIG. 3) has exceeded the predetermined distance. As a result, focus movement corresponding to the detected direction occurs with respect to the selection target object displayed on the screen.

  FIG. 9 shows an example of a user interface (UI) 900 displayed on the screen when a hand shape (grip fist) photographed via the camera 107 is detected.

  When a fist is detected by the hand shape recognition unit 301 (FIG. 3) via the camera 107 and the focus movement control unit 303 (FIG. 3) receives a signal indicating the detection (YES in step S501 of FIG. 5), As shown in FIG. 9, a grip fist icon corresponding to the hand shape is displayed. When this icon is displayed for the first time, it is displayed superimposed on a predetermined selection target object (for example, the selection target object A in the case of the screen shown in FIG. 8). Further, this icon moves between the selection target objects according to the flowchart shown in FIG. Note that the arrow around the clenched fist icon indicates that the focus can be moved in that direction. Therefore, by using the hand-shaped icon shown in this example, smooth movement of the focus on the selection target object is realized without reacting to an unintended movement (blur) of the hand shape that is the recognition target object. It becomes possible.

  According to the present embodiment, when the hand shape is moved by a predetermined distance or more, the new base point is a point that is 1 to 2 times the predetermined distance from the immediately preceding base point in the direction in which the hand shape has moved. By resetting, it is possible to suppress the movement of the focus due to the shake caused by the poor recognition accuracy or the shake of the user's hand (according to step S503 in FIG. 5).

  Further, according to the present embodiment, when the focus is moved in the direction intended by the user, even if the focus is actually moved slightly in the vertical direction, the movement of the focus in another axis direction not intended by the user is suppressed. (According to step S503 in FIG. 5).

  Further, according to the present embodiment, even if the user moves the hand slowly without intention, the movement of the focus can be suppressed (according to step S504 in FIG. 5).

(Application form)
Below, the form which applied embodiment mentioned above is demonstrated, referring drawings.

  FIG. 10 shows a hand shape recognition unit 301 (FIG. 3) showing a predetermined hand shape (in this embodiment, a fist) photographed via the camera 107 when executing a predetermined application (AV application in this embodiment). 3 shows an example of the UI 1000 when the focus movement control unit 303 (FIG. 3) first receives a signal indicating the detection (YES in step S501 in FIG. 5). In the UI 1000 shown in FIG. 10, 1 × 6 matrix buttons are arranged, but in the displayed UI, N × M (N, M: natural numbers) two-dimensional matrix buttons are arranged. Can be done.

  In relation to the above, FIG. 13 shows a case where a fist is detected by the hand shape recognition unit 301 in FIG. 3 and the focus movement control unit 303 in FIG. 3 first receives a signal indicating the detection (step S501 in FIG. 5). YES), an example of a default UI displayed in accordance with the operation target application is shown.

  Here, FIG. 13A is a diagram showing a UI 1301 displayed by an AV application such as DVD or CD viewing, and FIG. 13B is a diagram showing a UI 1302 displayed by a television viewing application. For example, in the Windows (registered trademark) OS series of Microsoft Corporation, these applications and the corresponding UI to be displayed are associated with each other and stored in advance in a registry, for example.

  Next, FIG. 11 shows how the focus of the selection target object (button) moves when a predetermined hand shape (grip fist) is moved with respect to the UI 1000 shown in FIG.

  FIG. 11A shows a detection of a fist photographed through the camera 107 by the hand shape recognition unit 301 in FIG. 3 when the AV application is executed, and the focus movement control unit 303 in FIG. This is the UI 1101 when the signal is first received (YES in step S501 in FIG. 5). At this stage, the media is played and the fourth pause button from the left is focused.

  Next, when the detected fist is moved in the right direction and the movement distance becomes a predetermined distance or more (YES in step S502 in FIG. 5), as shown in FIG. The button is moved forward and the focus is on the button that advances the chapter in the DVD title or the song in the CD music by one.

  Similarly, when the detected fist is moved in the right direction, as shown in FIG. 11B, the focus is also moved right by one, and the fast-forward button is focused.

  FIG. 12 shows how the UI 1000 shown in FIG. 10 changes when the detected hand shape changes from the first hand shape (grip fist) to the second hand shape (thumb up sign). .

  FIG. 12A shows the detection of a fist photographed via the camera 107 by the hand shape recognition unit 301 in FIG. 3 when the AV application is executed, and the focus movement control unit 303 in FIG. This is the UI 1201 when the signal is first received (YES in step S501 in FIG. 5). At this stage, the media is played and the fourth pause button from the left is focused.

  Here, it is assumed that the second hand shape (thum-up sign) is detected by the hand shape recognition unit 301 in FIG. 3 via the camera 107 and the focus movement control unit 303 in FIG. 3 receives a signal indicating the detection. (YES in step S505 in FIG. 5). Then, the focus movement control unit 303 in FIG. 3 determines the execution target as the focused pause button. Next, it is indicated that the focused pause button (the hatched portion of the UI 1202 in FIG. 12) has been determined as an execution target because its color has been changed or its size has increased. In response to the determination, the CPU 201 (FIG. 2) pauses the media playback. Thereafter, the button at the position where the pause button exists is changed to a button (not shown) for releasing the pause and playing again, and the UI in a state where the button is focused is displayed.

  Further, the above-described UI 1000 or the like disappears from the screen after a predetermined time (for example, 1 second) has elapsed since the hand shape stored in the hand shape database 302 is no longer detected.

  According to the present embodiment, the operation target application and the displayed corresponding UI are associated with each other, so that an operation intended by the user can be performed quickly.

  In addition, according to the present embodiment, since the recognition target hand shape does not disappear until a predetermined time elapses, for example, even if the detection of the hand shape that the user has put out due to erroneous detection of the hand shape is interrupted for a while, The UI can be operated continuously.

  Further, according to the present embodiment, since the hand shape to be recognized does not disappear until a predetermined time elapses, for example, even when the user cannot fully move the focus by moving his hand, By returning to a position with a margin, the UI can be continuously operated.

  The preferred embodiment of the present invention has been described above, but the present invention is not limited to the above-described embodiment as it is, and can be embodied by modifying the constituent elements without departing from the scope of the present invention.

  For example, the present invention can be applied to various information processing apparatuses such as a desktop personal computer, a mobile phone, a personal digital assistant (PDA), a game machine, and a television in addition to the notebook personal computer described in this embodiment. can do.

  In addition, the processing described in the above-described embodiment is performed on a recording medium such as a magnetic disk (flexible disk, hard disk, etc.), an optical disk (CD-ROM, DVD, etc.), a semiconductor memory, etc. as a program that can be executed by a computer. It can be written and provided to various devices. It is also possible to transmit to a variety of devices by transmitting via a communication medium. The computer reads the program recorded on the recording medium or receives the program via the communication medium, and the operation is controlled by this program, thereby executing the above-described processing.

  As described above, according to the present invention, it is possible to provide an information processing apparatus, a focus control movement method, and a focus movement control program capable of smoothly performing an operation on a display screen by a user's gesture photographed by a camera. it can.

100 Computer 101 Main Unit 102 Display Unit 103 LCD
104 Power button 105 Keyboard 106 Touchpad 107 Camera 201 CPU
202 Main memory 203 North bridge 206 South bridge 210 HDD
211 ODD
212 BIOS-ROM
213 EC / KBC
230 BIOS
301 hand shape recognition unit 302 hand shape database 303 focus movement control unit

Claims (18)

Display,
Focus movement control means for controlling the focus for selecting one of a plurality of selection target objects displayed on the display so as to move for each object;
A camera connected to the information processing apparatus;
Based on the base position change at every predetermined time, a position detecting means for detecting a continuous manner the position of the hand of the object in the image which the camera is shooting,
Based on the detected position, when the position of the target hand moves more than a predetermined distance, the movement direction of the target hand is continuously changed every predetermined time, assuming that the target hand has moved. A moving direction detecting means for detecting,
The position of the hand detected in the horizontal axis or vertical axis direction in the image is determined by using the position of the hand first detected by the position detecting unit as the first base point position of the base point positions. When moving more than the predetermined distance with the first base point position as the center, the focus is moved from the first base position by more than the predetermined distance with respect to the direction in which the focus is moved, and is perpendicular to the direction in which the focus is moved. An information processing apparatus , wherein the position moved to the point of the hand when the focus is moved is reset as a second base point position of the base point positions . Storage means for storing one or more hand shapes;
Hand shape recognition means for detecting whether or not a first hand shape that matches one or more hand shapes stored in the storage means is included in an image photographed by the camera; And
The focus movement control means by the hand shape recognition means, the first hand shape that matches the one or more hand shape stored in said storage means, contained in the image captured by the camera If it is detected that the focus is detected, the moving direction detecting means detects the moving direction from the first base point position to the moving direction of the focus from 1 to 2 times the predetermined distance. the information processing apparatus according to claim 1, wherein Rukoto the moved. The position detection means continuously detects the position of the target hand in the image captured by the camera based on the base position that changes every predetermined time,
After the focus is moved, the focus movement control means is configured to reduce the predetermined distance by 1 with respect to a direction in which the focus is moved from the base point position immediately before the first hand shape moves more than the predetermined distance. The position moved to the point of the first hand shape when the focus is moved with respect to the direction perpendicular to the direction in which the focus is moved is doubled or doubled as a new base position. The information processing apparatus according to claim 2, wherein the information processing apparatus is set. A second hand shape that matches the one or more hand shapes stored in the storage means by the hand shape recognition means, and is different from the first hand shape, When it is detected that it is included in the image photographed by the camera, the focus movement control means is selected from the plurality of selection target objects selected when the second hand shape is detected. The information processing apparatus according to claim 2 , wherein one of the two is determined as an execution target. In the focus movement control means, after the second base point position is reset, the first hand shape has not moved more than the predetermined distance from the second base point position after a first predetermined time has elapsed. In this case, the base position is moved to the position of the first hand shape before the first predetermined time has elapsed from the time elapsed after the second base position is reset. Item 3. The information processing device according to Item 2 . The focus movement control means, after the new base position is reset, after the first predetermined time has passed, the first hand shape has not moved more than the predetermined distance from the new base position In this case, the base point position is moved to the position of the first hand shape before the first predetermined time has elapsed than the time elapsed after the new base point position is reset. 3. The information processing apparatus according to 3 .   The corresponding user interface associated with a target application is displayed on the display when the first hand shape is first detected by the hand shape recognition unit. Information processing device. 8. The information processing according to claim 7 , wherein the user interface is not displayed on the display after a second predetermined time has elapsed since the first hand shape is no longer detected by the hand shape recognition unit. apparatus.   When the first hand shape is first detected by the hand shape recognition means, an icon indicating that the first hand shape has been detected is selected from among the predetermined selection target objects. The information processing apparatus according to claim 2, wherein the information processing apparatus is superimposed on one and displayed on the display. An image processing apparatus having a display acquires an image via a camera connected to the information processing apparatus,
It said position detecting means, based on the base position change at every predetermined time, the camera is the position of the hand of the object in the image continues to detect communication to shoot,
Based on the detected position, the movement direction detection means determines that the movement direction of the target hand is the predetermined movement direction when the position of the target hand moves by a predetermined distance or more. Continuously detected every hour,
The focus movement control means moves the focus for selecting one of the plurality of selection target objects displayed on the display for each object based on the detected movement direction, and the position detection means firstly Using the detected hand position as the first base point position of the base point positions, the hand position moves more than the predetermined distance around the first base point position in the horizontal axis or vertical axis direction in the image. When the focus is moved from the first base point position by the predetermined distance or more with respect to the direction in which the focus is moved, and when the focus is moved in a direction perpendicular to the direction in which the focus is moved, focus moving system, characterized in that the position is moved to the point of the hand, and re-set as the second base position of said base position Method. The hand shape recognition means detects whether or not a first hand shape that matches one or more hand shapes stored in the storage means is included in the image taken by the camera;
The focus movement control means detects that the first hand shape that matches one or more hand shapes stored in the storage means is included in the image taken by the camera. and if, on the basis of the moving direction of movement direction detection means detects, from said first base position, and wherein Rukoto moved 1 to 2 times of the predetermined distance with respect to the direction of moving the focus The focus movement control method according to claim 10. The position detecting means continuously detects the position of the target hand in the image captured by the camera based on the base point position that changes every predetermined time,
After the focus is moved, the focus movement control means is configured to reduce the predetermined distance by 1 with respect to a direction in which the focus is moved from the base point position immediately before the first hand shape moves more than the predetermined distance. The position moved to the point of the first hand shape when the focus is moved with respect to the direction perpendicular to the direction in which the focus is moved is doubled or doubled as a new base position. The focus movement control method according to claim 11 , wherein the focus movement control method is set. The second hand shape that matches one or more hand shapes stored in the storage means, and a second hand shape different from the first hand shape is taken by the camera. When it is detected that the object is included in the object, the focus movement control means sets one of the plurality of selection target objects selected when the second hand shape is detected as an execution target. 13. The focus movement control method according to claim 11, wherein the focus movement control method is determined. After the second base point position is reset, the focus movement control means has not moved the first hand shape beyond the predetermined distance from the second base point position after a first predetermined time has elapsed. In this case, the base point position is moved to the position of the first hand shape before the first predetermined time has elapsed from the time elapsed after the second base point position is reset. 11. The focus movement control method according to 11 . The first hand shape has not moved more than the predetermined distance from the new base position after the first predetermined time has elapsed after the focus movement control means has reset the new base position. In this case, the base point position is moved to the position of the first hand shape before the first predetermined time has elapsed than the time elapsed after the new base point position is reset. 12. The focus movement control method according to 12 . 12. The focus movement control method according to claim 11, wherein when the first hand shape is first detected, a corresponding user interface associated with a target application is displayed on the display. In an information processing apparatus having a display,
A procedure for acquiring an image via a camera connected to the information processing apparatus;
Based on the base position vary from the predetermined time, the procedure in which the camera is the position of the hand of the object in the image is detected continuous manner to shoot,
Based on the detected position, when the position of the target hand moves more than a predetermined distance, the movement direction of the target hand is continuously changed every predetermined time, assuming that the target hand has moved. Steps to detect,
Based on the detected moving direction, the focus for selecting one of the plurality of selection target objects displayed on the display is moved for each object , and the hand first detected by the position detecting unit is moved . When the position of the hand moves more than the predetermined distance around the first base point position in the horizontal axis or vertical axis direction in the image with the position as the first base point position of the base point positions, Move from the base position to the point of the hand when the focus is moved with respect to a direction perpendicular to the direction of movement of the focus by moving the focus more than the predetermined distance with respect to the direction of movement of the focus. position of the focus movement control programming to be executed and a procedure for resetting and wherein the second base position of said base position Beam. Executing a procedure for detecting whether or not a first hand shape that matches one or more hand shapes stored in the storage means is included in the image photographed by the camera;
If it is detected that the first hand shape that matches one or more hand shapes stored in the storage means is included in the image taken by the camera, the detected movement 18. The focus movement control according to claim 17 , wherein the focus movement control is performed so as to move from the first base point position by 1 to 2 times the predetermined distance with respect to a direction in which the focus is moved based on a direction. program.

Images