JP2009251702A - Information processing unit, information processing method, and information processing program - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To provide: an information processing unit capable of achieving each kind of control easily by intuitive user's operation without any need for user's operation of an input device and fitting a sensor to a user; an information processing method; and an information processing program. <P>SOLUTION: An information input device S comprises: cameras 1, 2 for photographing prescribed photography regions 1a, 2a to output photography information 1b, 2b; and a computer 8 for acquiring the three-dimensional coordinate information of a user's fingertip U2 in the photography regions 1a, 2a based on the photography information 1b, 2b from the cameras 1, 2, and outputting a control command based on the three-dimensional coordinate information. <P>COPYRIGHT: (C)2010,JPO&INPIT

Description

  The present invention relates to an information processing apparatus, an information processing method, and an information processing program, and more particularly, to an information processing apparatus that outputs various control commands based on shooting information.

  Conventionally, in a personal computer (hereinafter referred to as a PC) or the like, a keyboard or a mouse is used as an input device. When a user uses a PC, character input, control command input, image and video operations, and the like are performed using a keyboard and a mouse.

  Specifically, the video output from the PC is displayed on a monitor (video display device) such as a liquid crystal or a CRT. A mouse pointer corresponding to the operation of the mouse is also displayed on the monitor, and when the user operates the mouse, the mouse pointer on the monitor also operates corresponding to the operation. In addition, when word processing software or mail software is started, a character input screen is displayed on the monitor. When the user performs a predetermined character input operation using the keyboard, the character corresponding to the input operation is input on the monitor. It is displayed on the screen.

  Such an input operation using a keyboard and a mouse can correspond to a large number of keys and a large number of operations and control commands, thereby enabling various command inputs and the like. On the other hand, in order to smoothly perform the input operation, it is necessary to learn the key arrangement and to acquire the click operation and the drag operation using the mouse. In addition, the operation itself is not intuitive and lacks interest.

  For example, Japanese Patent Application Laid-Open No. H10-228707 discloses a proposal for realizing a user's input operation more intuitively and facilitating the input operation. According to this, by using a pen and a tablet as an input device, an input operation that is more intuitive and easier to understand than a keyboard operation or a mouse operation is realized. There has also been a proposal that enables an intuitive input operation by detecting a user's operation itself using a sensor or a camera (see Patent Documents 2 and 3).

Japanese Patent Laid-Open No. 11-345071 JP 2004-187043 A JP 2006-155244 A

  However, according to the one disclosed in the above-mentioned Patent Document 1, it is necessary to use an input device such as a pen or a tablet, and although it is somewhat intuitive, the user needs to become familiar with the operation to some extent. There is no big difference from input operation with a keyboard or mouse. Moreover, in order to detect a user's operation with a sensor, it is necessary to attach a sensor to a part of the user's body (for example, an arm or a finger), and preparation is complicated or the user cannot move around freely. There are many equal restrictions.

  In Patent Document 3, a user's gesture is photographed using a camera, and control is performed based on the motion. However, the two-dimensional motion grasps until the user gets tired, and the variation of the input command is poor. In order to respond to many command inputs, it is necessary to prepare many gestures that can be selected two-dimensionally, and sometimes it is not an intuitive input operation by the user, such as requiring an unnatural action. Occurs.

  Therefore, it is possible to input a variety of control commands based on the user's intuitive operation, that is, natural movements, and it is not necessary to become familiar with how to use an input device such as a mouse. However, there has been a need for an input device that can easily perform an input operation.

  The present invention has been made in view of the above circumstances, and it is not necessary for the user to operate the input device or to attach a sensor to the user, and various controls are easily realized by intuitive user operations. It is an exemplary problem to provide an information processing apparatus, an information processing method, and an information processing program that can be performed.

  In order to solve the above problems, an information processing apparatus as an exemplary aspect of the present invention includes an imaging unit that images a predetermined imaging area and outputs imaging information, and three-dimensional coordinates of a detection target in the imaging area. The information processing apparatus is characterized by having control means for acquiring information based on photographing information from the photographing means and outputting a control command based on three-dimensional coordinate information.

  Since the control command is output based on the three-dimensional coordinate information of the detected object (for example, the user's hand), a variety of controls are realized as compared with the case where the control is performed based on the two-dimensional position of the detected object. be able to. In addition, since control based on the three-dimensional position of the object to be detected is possible, there is no need for the user to operate a separate input device, and the user does not need to install a separate detection means such as a sensor. The operation can be performed. Here, as a result of the input operation by the user, the operation information is input to the control unit, and the control command is output by the control unit. Therefore, in the present application, “input operation”, “input operation information”, and “ “Output of control command” may be used almost synonymously.

  For example, by using the information on the depth of field, it is possible to acquire the three-dimensional coordinate information of the object to be detected by one photographing unit. That is, the XY coordinate information of the detected object can be acquired from the image captured by the imaging unit, and the Z coordinate information can be acquired using the focus position information of the detected object.

  The imaging means is conceptually including a video camera using an image sensor such as a CCD or CMOS. Not only a visible light image can be captured, but also any imaging means that can grasp the contour shape of an object, such as an infrared image, an ultrasonic image, and an image using radio waves. For information input / output between the photographing means and the control means, for example, those using USB (Universal Serial Bus), RGB composite input / output, parallel bus input / output, serial bus input / output, IEEE1394 (so-called Fire Wire) input. Any input / output mode such as output, wireless (infrared wireless, Bluetooth, etc.) input / output can be applied.

  The imaging unit captures a first imaging unit that captures one region, and a second imaging that captures a second region at least partially overlapping the first region from a different direction from the first imaging unit. And the control means is configured to obtain the three-dimensional coordinate information of the detected object in the overlapping area where the first area and the second area overlap, and the imaging information from the first imaging means. And the imaging information from the second imaging means, and a control command may be output based on the three-dimensional coordinate information.

  Since the overlapping area is imaged from different directions by a plurality of imaging means, the position (coordinate information) of the detected object (for example, the user's hand) in the overlapping area can be easily grasped three-dimensionally.

  Since the position of the object to be detected is grasped by a plurality of photographing means and control according to the position is performed, the user does not need to operate a separate input device, and the user attaches a separate detection means such as a sensor. The input operation can be performed without having to.

  The control means may determine that the object is a detected object based on at least one of the movement information and the contour information of the object in the overlapping region.

  Since it is determined that an object having a relatively large movement with respect to the background or whose outline is conspicuous is the detected object, erroneous recognition of the detected object can be reduced. For example, even if other objects (for example, desks and chairs) exist so as to be included in the virtual area, they may not be mistakenly detected as a detected object when they are stationary. it can. In addition, an object having a distinct outline in a background with a similar hue (for example, a human hand in a black background) can be recognized as a detected object.

  Here, the determination based on the movement information refers to detecting the presence / absence of movement of the object in the overlapping area, the movement speed, the movement direction, and the like based on the photographing information, and whether or not the object is a detected object according to the result. It means to judge. In addition, the outline information typically includes hue information, lightness information (luminance information), and saturation information, and based on these information, the background or the object that can be identified with the background is distinguished from the detected object. Say.

  The control unit may set a virtual area having at least one virtual boundary plane in the imaging area, and determine that an object intruding into the virtual area via the boundary plane is a detected object. .

  Since it is determined that an object that has entered the virtual region via the boundary plane is a detected object, erroneous recognition of the detected object can be reduced. For example, even if other objects (for example, a desk or a chair) exist so as to be included in the virtual area, the other objects are not recognized as detected objects, and are outside the virtual area. When a standing user extends his / her arm forward and his / her hand enters the virtual area via the boundary plane, it can be determined that the hand is the detected object.

  It is determined that an object caused by the user (part of the user's body, a pointing stick or a pen held by the user) is an object to be detected accurately, and other objects that are not attributed to the user are not detected objects. Since the determination can be made accurately, the input operation by the user can be accurately grasped, and erroneous recognition of the input operation can be reduced.

  The control means may output a control command based on the three-dimensional coordinate information farthest from the boundary plane among the three-dimensional coordinate information of the detection target.

  When the user's hand enters the virtual area via the boundary plane, for example, a control command can be output based on the tip position of the index finger pointed by the user. Further, when the user performs an input operation with a pen in his hand, a control command can be output based on the position of the tip of the pen.

  The control means may output a control command based on the gesture of the detected object in the imaging area.

  Since the control command is output based on the gesture (motion) of the detected object in the imaging area, various controls can be realized, and the user's intuitive and natural motion can be used as the input operation. . For example, if you move the arm that the user has stretched forward in the overlap area from left to right, the mouse pointer moves from left to right or a straight line is drawn from left to right. It is possible to realize such control as to do. Also, for example, if the user knocks in the overlapping area, if the user swings his / her finger back and forth once, the action will be input as a click operation. You can also If you move your finger up, down, left, or right with your finger protruding forward, you can use that action as input for the drag operation.

  The control means determines that a plurality of three-dimensional coordinate information whose gestures substantially match in the shooting region are of one detected object, and a plurality of three-dimensionals whose gestures do not substantially match in the shooting region The coordinate information may be determined to belong to a plurality of different objects to be detected.

  By comprising in this way, it can grasp | ascertain how many to-be-detected bodies exist in an imaging | photography area | region. Therefore, it is possible to accurately grasp and accept a plurality of different input operations performed substantially simultaneously by the plurality of detection objects. For example, when it is determined that there is one object to be detected, a control command is output based on one gesture by the one object to be detected. On the other hand, for example, when it is determined that there are two objects to be detected, two control commands are output based on two gestures by each of the two objects to be detected.

  The information processing apparatus may further include video display means for displaying video on a predetermined video display surface, and the control means may output a control command for controlling video display based on the three-dimensional coordinate information.

  As the image display means, for example, a display device such as a CRT display, a liquid crystal display, a plasma display, and an EL display, as well as a projector that projects an image on a wall surface, a screen, or the like are included. Since the video display based on these is controlled based on the three-dimensional coordinate information of the detected object, for example, the movement of the detected object, which is a part of the user's body, and the video display, etc. are fused, and intuitive operation (input An interesting video expression based on the operation) can be realized.

  Since the input operation is grasped three-dimensionally rather than two-dimensionally, more interesting video display control can be performed. For example, if a user steps forward and holds a hand over the image of a flowing waterfall, the flowing water expresses the image so as to avoid the user's hand. It is possible to express images so as not to avoid them. By setting an overlapping area in front of the screen on which the image is projected by the projector, the user can control the image display by operating while standing in front of the screen and looking at the screen. In addition, by setting an overlapping area below the liquid crystal display and virtually setting a keyboard that does not actually exist in the overlapping area, the user can perform key input operations in an empty space below the liquid crystal display. Thus, an input result similar to that of a key input operation using an actual keyboard can be obtained.

  According to another exemplary aspect of the present invention, there is provided an information processing method in which imaging information from a first imaging unit that images a first area and a second area that at least partially overlaps the first area are first. 3D coordinate information of an object to be detected in an overlapping region where the first region and the second region overlap is acquired based on the photographing information from the second photographing unit that photographs from a direction different from the photographing unit. And a step of outputting a control command based on the three-dimensional coordinate information.

  Since the overlapping area is imaged from different directions by the plurality of imaging means, the position (coordinate information) of the detected object (for example, the user's hand) in the overlapping area can be grasped three-dimensionally. Since the control command is output based on the three-dimensional coordinate information of the detected object, a variety of controls can be realized as compared with the case where the control is performed based on the two-dimensional position of the detected object.

  Since the position of the object to be detected is grasped by a plurality of photographing means and control according to the position is performed, the user does not need to operate a separate input device, and the user attaches a separate detection means such as a sensor. The input operation can be performed without having to.

  An information processing program as still another exemplary aspect of the present invention acquires a three-dimensional coordinate information of a detected object in an imaging region based on imaging information from an imaging unit that images a predetermined imaging region. It functions as an information acquisition means and a control command output means for outputting a control command based on three-dimensional coordinate information.

  Since the control command is output based on the three-dimensional coordinate information of the detected object (for example, the user's hand), a variety of controls are realized as compared with the case where the control is performed based on the two-dimensional position of the detected object. be able to. In addition, since control based on the three-dimensional position of the object to be detected is possible, there is no need for the user to operate a separate input device, and the user does not need to install a separate detection means such as a sensor. The operation can be performed.

  Further objects and other features of the present invention will become apparent from the preferred embodiments described below with reference to the accompanying drawings.

  According to the present invention, it is not necessary for the user to operate the input device or to attach a sensor to the user, and it is possible to input a variety of control commands based on intuitive operations, that is, natural operations. There is no need to learn how to use an input device such as a mouse, and an intuitive, novel and interesting input operation can be easily performed even for a child or a physically handicapped person.

[Embodiment 1]
Hereinafter, an information input system (information processing apparatus) S according to Embodiment 1 of the present invention will be described with reference to the drawings. FIG. 1 is a schematic configuration diagram showing an outline of the overall configuration of the information input system S. The information input system S includes a first camera (first photographing means) 1, a second camera (second photographing means) 2, a projector (video display means) 6, and a computer (control means) 8. Has been.

  The first camera 1 is a camera for photographing the first region (first photographing region) 1a, and for example, a video camera using an image sensor such as a CCD or a CMOS is used. The first area 1a means an area in the imaging field of view schematically represented by a broken line L1 in FIG. In the first embodiment, the first camera 1 is disposed at an upper position in front of the screen 10, and the first region 1a is set so as to face the lower side. Therefore, the first area 1 a is set in front of the screen 10. In FIG. 1, the front direction when viewed from the screen 10 is the rear (−Z direction), and the opposite direction is the front (Z direction). Further, when viewing the screen 10 from the user U, the left direction is the X direction, the upper direction is the Y direction, and the opposite directions are the right direction (−X direction) and the lower direction (−Y direction), respectively.

  Similarly to the first camera, the second camera 2 uses a video camera or the like using an image sensor such as a CCD or a CMOS, and is typically a second area (first area) as an area in the field of view represented by a broken line L2. 2 shooting area) 2a. Similarly to the first area 1a, the second area 2a is also set in front of the screen 10, and the second camera 2 disposed at the right front position of the screen 10 moves the second area 2a in the left direction (- Shooting to face in the (X direction). Thus, the 1st camera 1 and the 2nd camera 2 image | photograph each area | region 1a, 2a from a mutually different direction.

  The first camera 1 is configured to input the shooting information 1b of the first area 1a to the computer 8, and the second camera 2 is configured to input the shooting information 2b of the second area 2a to the computer 8. The first area 1a and the second area 2a overlap at least partially, and a virtual area W is set in the overlapping area by the computer 8 described later. Therefore, the images in the overlapping area are taken from different angles and input from both the first camera 1 and the second camera 2 to the computer 8.

  The projector 6 is for displaying video on a screen (predetermined video display surface) 10. The projector 6 is connected to the computer 8 and can project a display video based on the video signal 3 from the computer 8 onto the screen 10. For example, as shown in FIG. 2A, the execution icons 12 and folders 14 of various application programs arranged on the OS (operating system) desktop are displayed, or as shown in FIG. The drawing surface 16 of the graphic software inside is displayed.

  The computer 8 is control means connected to the first camera 1, the second camera 2, and the projector 6. Hereinafter, a schematic configuration and detailed functions of the computer 8 will be described.

  As schematically shown in FIG. 3, the computer 8 is roughly configured with a storage device 4 and a CPU (arithmetic processing unit) 5 inside. The storage device 4 is configured by a memory, a hard disk, or the like, and stores and saves information. In the first embodiment, the information processing program P, the gesture database D, the virtual area are stored in the storage device 4. Setting information V is stored.

  The information processing program P is program software for causing the CPU 5 as a main part of the computer 8 to function as the information acquisition unit 5a, the gesture determination unit 5b, the control command output unit 5c, and the control command processing unit 5d. Details of each of the functions exhibited by the CPU 5 will be described later.

  The gesture database D is configured by associating a gesture (motion) pattern of the body U1 to be detected with a control command (command) corresponding to the gesture. For example, as shown in FIG. 4, a control command “mouse click” is associated with “one-time back-and-forth reciprocation in a short time” of the detected object U1, and “two-time back-and-forth reciprocation in a short time” ”Is associated with a control command“ double click ”. A control command of “dragging in the XY plane” is associated with “moving operation in the XY plane without moving backward after moving forward” of the detected body U1.

  Of course, other corresponding gestures may be associated with various control commands such as “scroll”, “delete”, “enlarge”, “reduce”, and “rotate”. In that case, even if the linked operation by the two detected objects (for example, “enlarge” when the two detected objects are separated from each other, “reduced” when the two detected objects are close to each other) is associated with each control command. Good.

  The virtual area setting information V stores three-dimensional coordinate information (position information, XYZ coordinate information) of the virtual area W set in the overlapping area, and demarcates the position of the virtual area W in the real space. That is, the three-dimensional coordinate information of the vertices t1 to t8 virtually stored in the real space shown in FIG. 1 is stored based on the shooting information 1b and 2b of the first camera 1 and the second camera 2. A rectangular parallelepiped region surrounded by the vertices t1 to t8 is set as the virtual region W. In the virtual region W, in the first embodiment, the rearmost plane defined by the vertices t1 to t4 is set as the boundary plane W1, and the vertices t5 to t8 are all set on the screen 10.

  Of course, the shape of the virtual region W is not limited to a rectangular parallelepiped, and various shapes such as a spherical shape, a cylindrical shape, and a triangular pyramid shape can be set. In the first embodiment, the vertices t5 to t8 are set on the screen 10, but t5 to t8 may be set in front of the screen 10 (−Z direction position).

  Next, each function performed by the CPU 5 will be described.

  The information acquisition unit 5a has a function of acquiring the respective pieces of shooting information 1b and 2b from the first camera 1 and the second camera 2. And it has a function which acquires the three-dimensional coordinate information of the to-be-detected body U1 in the virtual area W based on the imaging | photography information 1b and 2b.

  That is, the information acquisition unit 5 a acquires the shooting information 1 b of the first area 1 a by the first camera 1 and the shooting information 2 b of the second area 2 a by the second camera 2. Then, by performing image recognition processing using the photographing information 1b and 2b, it is determined whether or not the detected body U1 exists in the virtual region W, and if the detected body U1 exists, the three-dimensional Get coordinate information.

  The image recognition processing can be performed by the following procedure, for example, although a known image recognition processing technique can be used. That is, based on the luminance information, lightness information, saturation information, etc. in the photographing information 1b, 2b, the contour of the object existing in the overlapping area is extracted. Whether or not the object exists in the overlapping region can be determined based on whether or not the shape information resulting from the object exists in both the first region 1a and the second region 2a. .

  When it is determined that the object exists in the overlapping area, it is determined whether or not the object has entered the virtual area W via the boundary plane W1, and the virtual area W is determined via the boundary plane W1. If the object has entered, the object is determined to be the detected body U1. A virtual region W in which at least a part of the object whose contour shape is extracted by the image recognition process intersects the boundary plane W1 set by the vertices t1 to t4, and at least a part of the object is set by the vertices t1 to t8. If the object exists, the object is determined to be the detected body U1. In the first embodiment, the detected body U1 is the hand of the user U.

  When the information acquisition unit 5a determines that the hand (detected body) U1 of the user U exists in the virtual area W, the information acquisition unit 5a recognizes a part (farthest part) U2 farthest from the boundary plane W1 in the hand U1. . The three-dimensional coordinate information of the farthest portion U2 has the largest Z coordinate (ie, the closest to the screen 10) among the three-dimensional coordinate information of a plurality of places constituting the contour of the hand U1 existing in the virtual region W. ) Part three-dimensional coordinate information. In the first embodiment, the farthest portion U2 is the fingertip of the user U. The information acquisition means 5a always grasps the three-dimensional coordinate information of the fingertip U2 and constantly monitors the position, the operation direction, the operation speed, and the like.

  In the first embodiment, the position of the fingertip U2 corresponds to the position of the mouse pointer 7, and the computer 8 causes the projector 6 to display an image of the mouse pointer 7 based on the three-dimensional coordinate information of the fingertip U2. . That is, the mouse pointer 7 is displayed at the corresponding position on the screen 10 based on the two-dimensional coordinate information of the XY coordinates among the three-dimensional coordinate information of the fingertip U2. When the XY coordinate position of the fingertip U2 changes, the display position of the mouse pointer 7 changes correspondingly.

  The gesture determination unit 5b has a function of determining whether or not the operation of the fingertip U2 of the user U is an operation corresponding to a predetermined gesture pattern. Specifically, based on the three-dimensional coordinate information of the fingertip U2 acquired from the information acquisition means 5a, it is determined whether or not the operation of the fingertip U2 matches the gesture pattern stored in the gesture database D.

  When the movement of the fingertip U2 matches any gesture pattern in the gesture database D, the control command output unit 5c outputs a control command associated with the gesture pattern. For example, when it is determined that the operation of the fingertip U2 is “one-time back and forth reciprocation in a short time”, a control command “mouse click” is output from the control command output means 5c.

  On the other hand, when the operation of the fingertip U2 does not match any gesture pattern in the gesture database D, the gesture determination unit 5b causes the control command output unit 5c to output a control command for controlling video display. Based on this video display control command, the display position of the mouse pointer 7 changes corresponding to the change in the XY coordinate position of the fingertip U2.

  The control command output means 5c has a function of outputting a control command based on the three-dimensional coordinate information of the fingertip U2. Specifically, control commands such as “mouse click”, “double click”, and “drag” and a video display control command for controlling video display are output according to the determination result by the gesture determination means 5b.

  For example, when the fingertip U2 is at a position corresponding to the execution icon 12 on the screen 10, if the gesture determination means 5b determines "mouse click", the control command output means 5c provides a control command for "mouse click". To cause the control command processing means 5d to execute the application program corresponding to the execution icon 12. For example, when the fingertip U2 is at a position corresponding to the folder 14 on the screen 10 and the gesture determination means 5b determines “double click”, the control command output means 5c outputs a control command for “double click”. The control command processing means 5d outputs and causes a list display process of the files stored in the folder 14 to be performed.

  Further, for example, when the graphic software is already activated and the drawing surface 16 is displayed on the screen 10, the movement of the fingertip U <b> 2 moving from the left side to the right side in the virtual area W is in any gesture pattern. If it is determined by the gesture determination means 5b that the two do not match, the control command output means 5c outputs a video display control command for moving the mouse pointer 7 from the left side to the right side on the screen 10. Based on the video display control command, the control command processing means 5d draws a line from the left side to the right side on the drawing surface 16.

  The control command processing means 5d causes the computer 8 to execute predetermined calculation processing based on the control command from the control command output means 5c. When the control command is a control command corresponding to a mouse button operation such as “mouse click” or “double click”, a process corresponding to the control command (for example, application startup, folder content display, icon or (Move objects such as folders). On the other hand, when the control command is a video display control command, the video signal 3 is adjusted based on the video display control command, and the video is displayed on the screen 10 by the projector 6.

  Next, the information input procedure in the information input system S will be described with reference to the flowchart of FIG.

  The first camera 1 captures the first area 1a and outputs the photographing information 1b to the computer 8 (S.1). At the same time, the second camera 2 captures the second area 2a and outputs the captured information 2b to the computer 8 (S.2). A virtual area W is set in the overlapping area, and the three-dimensional coordinate information of the vertices t1 to t8 is stored in the virtual area setting information V (S.3).

  Based on the shooting information 1b and 2b, it is determined whether or not an object exists in the virtual area W (S.4), and if an object exists, whether or not the object is the hand U1 of the user U is determined. Judgment is made (S.5). Specifically, when the object has entered the virtual area W via the boundary plane W1, it is determined that the object is the hand U1.

  When the computer 8 grasps the hand U1, the position of the fingertip U2 is recognized (S.6). Specifically, of the hand U1 in the virtual region W, the part farthest from the boundary plane W1 (that is, the point with the largest Z coordinate) is recognized as the fingertip U2, and the three-dimensional coordinate information is obtained. get. Based on the XY coordinate information of the three-dimensional coordinate information of the fingertip U2, the projector 6 displays the mouse pointer 7 at a position on the screen 10 corresponding to the XY coordinate information of the fingertip U2 (S.7).

  The computer 8 grasps the operation of the fingertip U2 based on the three-dimensional coordinate information (S.8). When the movement of the fingertip U2 matches the gesture pattern in the gesture database D (S.9), the computer 8 outputs a control command corresponding to the gesture pattern (S.10), for example, mouse click or double Perform operations such as clicking. On the other hand, when the motion of the fingertip U2 does not match the gesture pattern in the gesture database D (S.9), the mouse pointer 7 is moved and displayed corresponding to the movement of the fingertip U2 in the XY plane (S.11). . Based on the movement display of the mouse pointer 7, for example, a line drawing operation or the like is realized.

  As mentioned above, although preferable embodiment of this invention was described, this invention is not limited to these, A various deformation | transformation and change are possible within the range of the summary.

  For example, in the first embodiment, the case where one detected object exists in the virtual area W has been described. However, a plurality of detected objects may exist in the virtual area W. In that case, a large number of three-dimensional coordinate information from a plurality of locations constituting the contours of the plurality of detection objects is acquired by the information acquisition means 5a. When the large number of three-dimensional coordinate information moves based on substantially the same operation, it can be determined that the large number of three-dimensional coordinate information is based on one detected object, and the virtual region It can be determined that one object to be detected exists in W.

  On the other hand, when a large number of three-dimensional coordinate information moves based on substantially different movements, that is, some of the large number of three-dimensional coordinate information moves substantially in the left direction (−X direction). Nevertheless, if the other three-dimensional coordinate information is substantially moving in the right direction (X direction), two detected objects (for example, the right hand of the user U in the virtual region W) And the left hand.) Can be determined to exist. Therefore, it can be configured to output two different control commands almost simultaneously based on the operations of the two detection objects. Further, based on the linked operation of the two detection objects, for example, a new control command such as “enlargement”, “reduction”, and “rotation” can be output.

[Embodiment 2]
FIG. 6 is a schematic configuration diagram showing an outline of the overall configuration of the information input system S ′ according to Embodiment 2 of the present invention. In the second embodiment, the information input system S ′ has a first camera 21, a second camera 22, a liquid crystal display 26, and a computer 28.

  The first camera 21 and the second camera 22 are simple cameras called a so-called USB camera or a Web camera, which are connected to the computer 28 by USB (Universal Serial Bus) using an imaging element such as a CMOS. In this information input system S ′, the first camera 21 and the second camera 22 are arranged on both the left and right sides of the liquid crystal display 26, and these include a virtual area W ′ set in front of the liquid crystal display 26. Shooting overlapping areas.

  The virtual area W ′ is formed so as to be surrounded by the vertices t21 to t28, and a boundary plane W1 ′ is set by the vertices t21 to t24 positioned above. A plurality of virtual key areas K are set in the virtual area W ′. The virtual key area K is virtually set based on the key arrangement corresponding to the keyboard, and is surrounded by a plurality of virtual vertices (for example, vertices k1 to k8), like the virtual area W. .

  The virtual key area K is associated with input control commands for different character information so as to correspond to the keys on the keyboard, respectively, and the user's fingertip U2 passes through the boundary plane W1 ′ in the predetermined virtual key area K. When entering, character information corresponding to the virtual key area K is input.

  According to the configuration according to the second embodiment, a virtual keyboard can be set without the need to separately prepare an input device such as a keyboard. For example, the user can perform a key input operation on the desk space in front of the liquid crystal display 26. If it does, the key input corresponding to the key input operation becomes possible. Since no separate input device is required, cost reduction and portability improvement of the entire system can be realized, and the operation space can be saved.

[Modification]
It is also possible to acquire the three-dimensional coordinate information of the detected object by one imaging means. For example, the YX coordinate information of the detected object can be obtained by the CCD camera, and the Z coordinate information of the detected object can be obtained based on the depth of field information of the detected object.

  As a specific example, for example, by moving the focus position of the CCD camera back and forth (± Z direction) and detecting at which position the object is focused, the position of the detected object in the Z direction is grasped. be able to. Further, the Z-direction position of the detected object can also be grasped by measuring the distance to the detected object in the imaging region using infrared rays or the like using a separately prepared autofocus device.

  As a connection method for connecting such a CCD camera to a computer, in addition to the above-described USB, IEEE 1394 (Fire Wire) connection method, wireless connection method using infrared, wireless using radio waves such as Bluetooth. Any connection method such as a connection method can be applied.

It is a schematic block diagram which shows the outline of the whole structure of the information input system which concerns on Embodiment 1 of this invention. FIG. 3 is an example of a display image displayed on the screen shown in FIG. 1, where (a) is a display example of execution icons and folders arranged on the desktop of the OS, and (b) is a graphic being activated. It is an example of a display of a software drawing surface. It is a schematic block diagram which shows the outline of the internal structure of the computer shown in FIG. It is a data block diagram of the gesture database stored in the memory | storage device of the computer shown in FIG. It is a flowchart which shows the procedure of the information input in the information input system which concerns on Embodiment 1 of this invention. It is a schematic block diagram which shows the outline of the whole structure of the information input system which concerns on Embodiment 2 of this invention.

Explanation of symbols

D: Gesture database K: Virtual key area L1, L2: Dashed line P: Information processing program S, S ′: Information input system (information processing apparatus)
t1 to t8, t21 to t28, k1 to k8: vertex U: user U1: hand (object to be detected)
U2: Fingertip (farthest part)
V: virtual area setting information W, W ′: virtual areas W1, W1 ′: boundary planes 21, 21: first camera (first photographing means)
1a: 1st area | region (1st imaging | photography area | region)
1b: shooting information 2, 22: second camera (second shooting means)
2a: second area (second imaging area)
2b: shooting information 3: video signal 4: storage device 5: CPU (arithmetic processing unit)
5a: Information acquisition means 5b: Gesture determination means 5c: Control command output means 5d: Control command processing means 6: Projector (video display means)
7: Mouse pointer 8, 28: Computer (control means)
10: Screen (predetermined video display surface)
12: Execution icon 14: Folder 16: Drawing surface 26: Liquid crystal display

Claims (10)

Photographing means for photographing a predetermined photographing region and outputting photographing information;
An information processing apparatus comprising: control means for obtaining three-dimensional coordinate information of a detection object in the photographing region based on photographing information from the photographing means and outputting a control command based on the three-dimensional coordinate information . The photographing means is
First imaging means for imaging the first region;
Second imaging means for imaging a second area at least partially overlapping the first area from a direction different from the first imaging means, and
The control means is
The three-dimensional coordinate information of the object to be detected in the overlapping region where the first region and the second region overlap, the photographing information from the first photographing unit and the photographing information from the second photographing unit. The information processing apparatus according to claim 1, wherein the information processing apparatus outputs the control command based on the three-dimensional coordinate information. The control means is
The information processing apparatus according to claim 1, wherein the information processing apparatus determines that the object is the detected object based on at least one of movement information and contour information of the object in the overlapping region. The control means is
A virtual area having at least one virtual boundary plane is set in the imaging area;
The information processing apparatus according to claim 1, wherein an object intruding into the virtual area via the boundary plane is determined as the detected object. The control means is
The information processing apparatus according to claim 3 or 4, wherein the control command is output based on three-dimensional coordinate information farthest from the boundary plane among the three-dimensional coordinate information of the detection target. The control means is
The information processing apparatus according to claim 1, wherein the control command is output based on a gesture of the detected object in the imaging region. The control means is
A plurality of the three-dimensional coordinate information with which the gesture substantially matches within the imaging region is determined to belong to one detected object;
The information processing apparatus according to claim 6, wherein a plurality of the three-dimensional coordinate information whose gestures do not substantially match in the imaging region are determined to belong to a plurality of different detection objects. A video display means for displaying video on a predetermined video display surface;
The information processing apparatus according to any one of claims 1 to 7, wherein the control unit outputs a control command for controlling the video display based on the three-dimensional coordinate information. Second information is obtained by photographing photographing information from first photographing means for photographing the first area and a second area at least partially overlapping with the first area from a direction different from that of the first photographing means. Acquiring three-dimensional coordinate information of the detected object in an overlapping region where the first region and the second region overlap based on imaging information from the imaging means;
Outputting a control command based on the three-dimensional coordinate information. Computer
Information acquisition means for acquiring three-dimensional coordinate information of the detected object in the imaging area based on imaging information from the imaging means for imaging a predetermined imaging area; and
An information processing program that functions as a control command output unit that outputs a control command based on the three-dimensional coordinate information.

Images