JP2015109111A - Gesture operation input processing device, three-dimensional display device and gesture operation input processing method - Google Patents

Abstract

PROBLEM TO BE SOLVED: To provide a technique of gesture interface for supporting operation input by user gesture.SOLUTION: A distance calculation unit 20 obtains, from a gesture image obtained by capturing an image of user gesture performed while watching a three-dimensional display, a distance of the user gesture in depth direction. A gesture recognition parameter adjustment unit 50 adjusts, on the basis of the resolution of distance measurements in depth direction, a parameter when recognizing operation input by user gesture. A gesture recognition processing unit 60 refers to, under an adjusted parameter, a distance in depth direction calculated by the distance calculation unit 20, to recognize operation input by user gesture.

Description

  The present invention relates to an apparatus and method for processing an operation input by a gesture.

  3D TVs that can view 3D images can now be used at home. In addition, a Blu-ray disc on which 3D video is recorded can be played back by a player and displayed on a 3D display for viewing high-quality 3D video. In a game machine, a game application using a 3D image can be played by connecting to a 3D display. Furthermore, a user's 3D image | video may be image | photographed using a camera provided with the distance sensor, and it may take in in a game application.

  Among game applications, there are an increasing number of types in which a user performs operations using the body or participates in a game. In the future, when the game application becomes three-dimensional, it is expected that more input operations using gestures will be used.

  Patent Document 1 discloses a portable game device having an interface through which a user can give an instruction from a position away from the screen.

International Publication No. 10/038822 Pamphlet

  There is a demand for a gesture interface technique for supporting operation input by a user's gesture. The system needs to control the detection sensitivity when recognizing the gesture to an appropriate level according to the performance of the system so that the user can input an operation through the gesture without feeling stressed in front of the display. .

  The present invention has been made in view of these problems, and an object of the present invention is to provide a technique that facilitates operation input by a gesture.

  In order to solve the above-described problem, a gesture operation input processing device according to an aspect of the present invention includes an instruction point extraction unit that extracts a user instruction point from an image obtained by photographing a user gesture performed while watching a display, and the instruction Based on at least one of the distance calculation unit for obtaining the distance in the depth direction to the point, the resolution of the distance measurement in the depth direction, and the three-dimensional display performance of the display, the depth direction when recognizing the operation input by the user's gesture A parameter adjustment unit that adjusts a parameter relating to detection sensitivity, and a distance in the depth direction to the indicated point calculated by the distance calculation unit based on the adjusted parameter, and an operation input by a user's gesture And a gesture recognition processing unit for recognizing.

  Another aspect of the present invention is a gesture operation input processing method. This method is a gesture operation input processing method in a three-dimensional gesture input system having an interface that supports an operation input by a user's gesture, and points to the user from an image obtained by photographing the user's gesture while looking at a display. Recognizing an operation input by a user's gesture based on at least one of a depth direction distance measurement resolution and a 3D display performance of the display. Adjusting a parameter related to detection sensitivity in the depth direction when performing the gesture, and referring to the distance in the depth direction to the indicated point calculated by the distance calculation unit based on the adjusted parameter, Recognize operation input by And a step.

  It should be noted that any combination of the above-described constituent elements and the expression of the present invention converted between a method, an apparatus, a system, a computer program, a data structure, a recording medium, etc. are also effective as an aspect of the present invention.

  According to the present invention, operation input by a gesture can be easily performed.

It is a block diagram of a three-dimensional gesture input system. It is a block diagram of a gesture operation input processing apparatus. FIGS. 3A and 3B are diagrams for explaining the distance measurement principle and distance resolution of a triangulation type camera. 4A and 4B are diagrams for explaining the distance measurement principle and distance resolution of a TOF type camera. It is a figure explaining the virtual screen set in the front surface of a display apparatus. 6A to 6E are diagrams illustrating an example of an operation input by a gesture. It is a figure explaining the parameter which adjusts the sensitivity of the motion detection of the depth direction of a gesture. 3 is a flowchart for explaining a method for adjusting a parameter related to sensitivity of motion detection in the Z direction by the gesture operation input processing device of FIG. 2.

  FIG. 1 is a configuration diagram of a three-dimensional gesture input system 300. The three-dimensional gesture input system 300 includes a game machine 200, a display device 210, and a camera 220.

  The game machine 200 executes content such as a game application and displays a video, an operation menu, and the like on the display device 210. Instead of the game machine 200, a player that reproduces content stored in a recording medium may be used.

  The display device 210 is a display capable of two-dimensional display and / or three-dimensional display. When the display device 210 can only display two-dimensionally, the content is displayed as a two-dimensional image. When three-dimensional display is also possible, the content is displayed in two dimensions according to an instruction from the user or an instruction from the application. Display as an image or a three-dimensional image.

  The camera 220 captures a gesture of the user 250. The camera 220 is provided on the display device 210 or the like. Consider a coordinate system in which the installation position of the camera 220 is the origin, the display surface is the XY plane, and the direction perpendicular to the display surface is the Z axis. The camera 220 has a distance measuring function for measuring the distance in the Z direction to the subject. For example, the camera 220 may have a distance sensor that measures the distance to the subject by projecting sound waves or light onto the subject and measuring the time from reflection to return, or two different viewpoints. You may have an image processing unit which calculates a depth value from the parallax image photoed from.

  The user 250 inputs an operation command for the game machine 200 using a gesture while viewing an image displayed on the display device 210. When the display device 210 has the dedicated glasses 230 for stereoscopic viewing like a stereo display, the user 250 wears the dedicated glasses 230 and views the screen of the display device 210.

  The user 250 gives an operation command to the game machine 200 by moving a finger of a hand or the like in the three-dimensional space as instruction points 240a to 240c. On the screen of the display device 210, the movement trajectories of the instruction points 240a to 240c detected by the game machine 200 are displayed in order to support the user's gesture input.

  The game machine 200 detects instruction points 240a to 240c such as fingers of the user 250's hand from the gesture image of the user 250 photographed by the camera 220, and obtains distances from the camera 220 to the instruction points 240a to 240c.

  In order for the game machine 200 to easily detect the instruction points 240a to 240c, the user may present only the index finger. Alternatively, the user 250 may attach a finger cap with a marker to a finger or make a gesture using an indicator bar with a marker so that the instruction point 240 can be easily recognized. Alternatively, the user 250 performs a gesture by holding a controller equipped with a position sensor capable of detecting three-dimensional coordinates, and transmits the three-dimensional coordinates detected by the position sensor to the game machine 200 by wireless communication or the like. Good.

  The game machine 200 recognizes the operation command intended by the user in accordance with the movement of the instruction point 240 of the user 250 in the three-dimensional space, and executes the operation command. In particular, the game machine 200 can provide flexibility to the gesture input interface by detecting the movement in the depth direction of the gesture of the user 250 to give the user 250 a degree of freedom.

  FIG. 2 is a configuration diagram of the gesture operation input processing device 100. A part or all of the functional configuration of the gesture operation input processing device 100 illustrated in FIG. 2 can be implemented in the game machine 200 by hardware, software, or a combination thereof. In addition to the game machine 200, it may be mounted on a personal computer, a portable device, a portable terminal, or the like.

  The gesture operation input processing device 100 includes an instruction point extraction unit 10, a distance calculation unit 20, a distance sensor performance acquisition unit 30, a distance sensor database 32, a display performance acquisition unit 40, a display database 42, a gesture recognition parameter adjustment unit 50, a virtual screen. A storage unit 52, a gesture recognition processing unit 60, a display control unit 70, and an application execution unit 80 are included.

  The instruction point extraction unit 10 acquires a gesture image of the user 250 taken by the camera 220 and extracts a user instruction point 240 from the gesture image. Since the instruction point 240 is a point to which the tip of the user's finger or a marker is attached, the instruction point 240 can be extracted from the image using a general image processing function such as edge extraction or image analysis.

  The distance calculation unit 20 calculates a distance in the depth direction to the indication point 240. When the camera 220 has a triangulation type distance measuring function, the distance in the depth direction from the parallax images captured from two different viewpoints to the instruction point 240 is obtained. In the case where the camera 220 has a time-of-flight (TOF) distance sensor such as an active laser distance meter or a radar, the distance calculation unit 20 projects light onto the instruction point 240 and then the light is reflected on the instruction point 240. The round trip time until the camera returns is acquired from the camera 220, and the distance to the indication point 240 is obtained by multiplying the round trip time by the speed of light.

  The distance sensor performance acquisition unit 30 refers to the distance sensor database 32, acquires information regarding the resolution of the distance sensor of the camera 220, and provides the information to the gesture recognition parameter adjustment unit 50. The resolution information of the distance sensor includes a range of measurable distance, identification information indicating whether the distance sensor distance measurement method is a triangulation type, or a TOF type.

  The display performance acquisition unit 40 refers to the display database 42, acquires information related to the display performance of the display device 210, and provides the display performance information of the display device 210 to the gesture recognition parameter adjustment unit 50. The display performance information includes information regarding the three-dimensional display performance. For example, the display device 210 is a 2D display that does not have a display capability in the Z direction as in a normal liquid crystal television display, or a stereo display that can be viewed in stereo (binocular stereoscopic view). Identification information indicating whether the 3D display is capable of displaying is included.

  The gesture recognition parameter adjustment unit 50 includes the resolution of the distance sensor provided from the distance sensor performance acquisition unit 30, the three-dimensional display performance of the display provided from the display performance acquisition unit 40, and the indication point 240 provided from the distance calculation unit 20. Based on the measured distance, the parameters for gesture recognition are adjusted. The parameters for the gesture recognition are particularly for adjusting the sensitivity of detecting the movement of the indication point 240 in the depth direction.

  If the resolution of the distance measurement in the depth direction is high, the gesture recognition parameter adjustment unit 50 increases the sensitivity by reducing the granularity for detecting the movement of the indication point 240 in the depth direction, and conversely, the resolution of the distance measurement in the depth direction is high. If it is low, the granularity for detecting the movement of the indication point 240 in the depth direction is coarsened to lower the sensitivity. Similarly, if the three-dimensional display performance of the display is high, the granularity for detecting the movement in the depth direction of the indication point 240 is made finer to increase the sensitivity. Conversely, if the three-dimensional display performance of the display is low, the indication point 240 Decrease the sensitivity by coarsening the granularity to detect the movement in the depth direction.

  In the present embodiment, a virtual screen is set on the front surface of the display device 210, and the instruction point 240 is activated when the instruction point 240 passes through the virtual screen. When the instruction point 240 is in front of the virtual screen as viewed from the user, the instruction point 240 is inactive, so that it is easy to distinguish when the user gives an operation input with a finger or the like and when not giving it. Like that. A plurality of virtual screens may be installed on the front surface of the display device 210 so that the user can input operation in stages by inserting a finger in stages. The virtual screen storage unit 52 holds information such as installation positions and installation intervals of such virtual screens.

  The gesture recognition parameter adjustment unit 50 adjusts the sensitivity for detecting the movement of the indication point 240 in the depth direction by adjusting parameters such as the installation position and installation interval of the virtual screen held in the virtual screen storage unit 52. can do.

  The gesture recognition processing unit 60 recognizes the user's gesture in the three-dimensional space with reference to the distance in the depth direction to the indication point 240 calculated by the distance calculation unit 20 under the adjusted parameters. Specify the operation command. The gesture recognition processing unit 60 determines whether or not the instruction point 240 has passed the virtual screen set on the front surface of the display device 210 based on the distance in the depth direction to the instruction point 240. The instruction point 240 is activated, and the operation input by the user's gesture is specified. The display control unit 70 displays the movement locus of the instruction point 240 on the display device 210 in order to indicate what kind of gesture the user has recognized by the gesture recognition processing unit 60.

  The application execution unit 80 executes the operation command specified by the gesture recognition processing unit 60 and reflects it in the application.

  FIGS. 3A and 3B are diagrams for explaining the distance measurement principle and distance resolution of the triangulation type camera 220. FIG. As shown in FIG. 3A, in the case of triangulation, two cameras 220a and 220b are used to capture parallax images from different viewpoints, or one camera 220 is moved to change the viewpoint position. By capturing the parallax image, the distance d to the instruction point 240 is measured. A stereo camera is an example of a ranging camera that uses the principle of triangulation.

  The graph of FIG. 3B shows the distance resolution R with respect to the absolute distance d in the triangulation type camera 220. The horizontal axis of the graph is the absolute distance d from the viewpoint to the subject, and the vertical axis is the distance resolution R by triangulation. As shown in FIG. 3B, in the case of triangulation, the distance resolution R decreases as the distance to the subject increases, and the measurement accuracy decreases. Conversely, the closer the distance to the subject, the higher the distance resolution R and the higher the measurement accuracy.

4A and 4B are diagrams illustrating the distance measurement principle and distance resolution of the TOF type camera 220. FIG. In the case of the TOF type camera 220, the distance to the subject is measured by projecting infrared light or ultrasonic waves onto the subject and measuring the reflected wave. As shown in FIG. 4A, in the TOF method, the light emitted from the sensor of the camera 220 is reflected by the indication point 240 and reflected, and the round-trip time t until it reaches the sensor is measured and multiplied by the speed of light c. By dividing by 2, the distance d can be obtained as follows.
d = t × c / 2

  The graph of FIG. 4B shows the distance resolution R with respect to the absolute distance d in the TOF type camera 220. As shown in FIG. 4B, in the case of the TOF method, the distance resolution R is constant regardless of the distance to the subject, and the measurement accuracy does not decrease even if the distance to the subject is far away.

  The measurement accuracy of the triangulation type camera 220 decreases as the distance to the subject increases, but the measurement accuracy can be increased by further widening the interval between the two viewpoints. Considering the installation of the camera 220 on the display device 210 in a general home, there is a limit to widening the viewpoint interval, so it is better to use the TOF type camera 220 whose measurement accuracy is constant regardless of the absolute distance. It is advantageous.

  The display device 210 also has a display capability difference that affects the operation input by the gesture. A two-dimensional display, such as an ordinary liquid crystal television display, does not have the ability to express in the Z direction. In the case of such a two-dimensional display, when a user draws a circle in the air while looking at his / her figure projected on the display or the movement locus of the instruction point 240, the distance in the depth direction is often a circle that is not constant. .

  A stereo display capable of stereo viewing (binocular stereoscopic viewing) has a Z-direction expression capability, and a user can view stereoscopically by viewing a set of left and right images using naked eyes or polarized glasses. For example, the left and right images are displayed in a time-division switching manner, and the left and right eyes are alternately input using the liquid crystal shutter glasses, or a parallax barrier or lenticular lens is used. Stereoscopic viewing is possible by making the left image / right image visible. In the case of such a stereo display, when a user draws a circle in the air while looking at the display, a circle parallel to the display surface can be drawn. However, the stereo display has a fixed viewpoint of the user for stereoscopic viewing, and can only perform stereoscopic viewing from one viewpoint in one direction, and often cannot follow changes in the user's viewpoint.

  A multi-view three-dimensional display such as an integral imaging method or a hologram method can produce a stereoscopic view from different viewpoint positions by projecting a plurality of different left and right images according to the viewpoint position. In the case of such a three-dimensional display, since the user can perform stereoscopic viewing from any viewpoint position, it looks as if there is a real thing in the three-dimensional space, and while viewing the displayed stereoscopic image, the depth direction Can be performed more accurately. The three-dimensional display can follow a change in the viewpoint position of the user, and enables more natural stereoscopic viewing. Further, if the hologram method is used, a substantially infinite set of left and right images can be shown, so that almost perfect stereoscopic viewing is possible.

  As the display device 210 changes from a two-dimensional display to a stereo display and a three-dimensional display, the operation accuracy in the Z-axis direction when the user performs an operation input with a gesture increases, and the operation input with a rich gesture in the Z direction occurs. It becomes possible to do.

  In the 3D gesture insertion system 300 according to the present embodiment, the user's operational feeling is optimized by adjusting the gesture recognition parameters according to the distance measurement resolution of the camera 220 and the 3D display performance of the display device 210. To do.

  FIG. 5 is a diagram for explaining a virtual screen set on the front surface of the display apparatus 210. In a coordinate system in which the camera 220 installed in the display device 210 is the origin, the display device 210 surface is the XY plane, and the depth direction perpendicular to the display device 210 surface is the Z axis, the distance d from the camera 220 is Z1, Z2. Virtual first and second virtual screens 270a and 270b are provided at positions (Z1 <Z2), respectively.

  When the user pushes his finger or the like toward the display device 210 and the indication point 240 breaks through these virtual screens, the indication point 240 is activated. The virtual screen has a distance switch function that activates the instruction point 240 according to the depth distance to the instruction point 240.

  When the user pushes the finger of the hand toward the display device 210 and the value of the Z coordinate at the position of the indication point 240 becomes smaller than Z2, it is determined that the indication point 240 has broken through the second virtual screen 270b, When the value of the Z coordinate at the position of the instruction point 240 becomes smaller than Z1, it is determined that the instruction point 240 has penetrated the first virtual screen 270a.

  For example, when the instruction point 240 breaks through the second virtual screen 270b at the position of the distance Z2, the gesture recognition processing unit 60 recognizes the operation as an operation for selecting a button or the like on the display, and the instruction point 240 Further pierce the first virtual screen 270a located at the distance Z1, the gesture recognition processing unit 60 may recognize the operation as a warp operation.

  Thus, by providing a two-stage virtual screen according to the distance in the depth direction, the gesture recognition processing unit 60 can specify different operation commands according to the change in the depth direction of the user's gesture. If the number of virtual screens is further increased, more different operation commands can be specified according to the number of virtual screens to be penetrated. In addition, if the number of virtual screens is increased, it is possible to detect a change in the acceleration in the depth direction and further vary the operation command.

  6A to 6E are diagrams illustrating an example of an operation input by a gesture. FIG. 6A is an example in which relative control is input by pushing a finger into a virtual screen and moving it up, down, left, or right from the origin, or drawing a circle. For example, when the finger breaks through the virtual screen, the instruction point 240 is activated, the object or icon displayed on the display device 210 is selected, and the object displayed on the screen is moved by moving the instruction point 240 up, down, left, or right. The icon can be moved up, down, left and right. When an image or document is displayed on the screen of the display device 210, the finger is moved up, down, left and right by moving the finger up, down, left and right, or a clockwise circle is drawn, and a counterclockwise circle is drawn. It may be possible to reduce the display.

  In FIG. 6B, when the finger is pushed into the virtual screen and moved to the left or right so as to cross the field of view of the cameras 220a and 220b, the gesture recognition processing unit 60 determines the user's gesture as a flick input, For example, the execution unit 80 moves to the next screen or executes continuous scrolling according to the determined operation input.

  FIGS. 6C and 6D show a state in which the user inputs a pinch operation using a gesture. Pinch is an operation of pinching with two fingers to widen (pinch out) or narrow (pinch in) between the fingers.

  In FIG. 6C, when a gesture is performed in which the left and right fingers are pushed into the virtual screen and the interval between the two fingers is narrowed, the gesture recognition processing unit 60 determines that the gesture is a pinch-in operation. The display control unit 70 executes processing for reducing the displayed screen.

  In FIG. 6D, when a gesture is performed in which two fingers on the left and right are pushed into the virtual screen and the distance between the two fingers is increased, the gesture recognition processing unit 60 determines that the gesture is a pinch-out operation. Then, the display control unit 70 executes a process for enlarging the displayed screen.

  In FIG. 6E, when the pointing point 240 is active and the finger is further protruded, the gesture recognition processing unit 60 determines that the operation is a warp operation. Further, for example, when only one finger is pushed out, the gesture recognition processing unit 60 determines that the selection / execution button is pressed, but when two fingers are pushed out, the gesture recognition processing unit 60 may determine that the cancel button is pressed.

  FIG. 7 is a diagram illustrating parameters for adjusting the sensitivity of motion detection in the depth direction of a gesture. Assume that virtual screens 270a, 270b, and 270c are provided at positions Z = Z1, Z2, and Z3 (Z1 <Z2 <Z3), respectively, where the display surface is the XY plane and the direction perpendicular to the display surface is the Z axis. The number of virtual screens to be set is N, the interval between adjacent virtual screens is D, and the width of the dead band (dead zone) of each virtual screen is d.

  The stage number N, the interval D, and the dead band width d are parameters related to the sensitivity of motion detection in the depth direction of the gesture, and these parameters are adjusted based on the distance measurement resolution and the three-dimensional display performance. Further, these adjustment parameters may be changed depending on the distance Z in the depth direction.

  As the display device 210 increases in 3D display performance from a 2D display to a stereo display and then to a 3D display, the gesture recognition parameter adjustment unit 50 increases the number of stages N and decreases the interval D and the dead bandwidth d. This is because the higher the three-dimensional display performance, the more the user can perform a rich operation in the Z direction, and there is no problem even if the detection sensitivity in the Z direction is increased.

  When the camera 220 is a triangulation type, the number of steps N, the interval D, and the dead band width d are changed according to the distance in the depth direction. In the triangulation method, the measurement accuracy decreases as the subject is farther from the camera 220. Therefore, when the distance from the camera 220 is smaller than a predetermined threshold, the number of steps N is increased, the interval D and the dead band width d are decreased, and the Z direction is reached. When the distance from the camera 220 is larger than a predetermined threshold, the number of steps N is reduced, the interval D and the dead band width d are reduced, and the detection sensitivity in the Z direction is lowered.

  When the camera 220 is of the TOF type, the measurement accuracy does not change even if the distance in the depth direction changes. Therefore, the interval D and the dead band width d are constant, but the TOF type has a high measurement accuracy. It can be increased compared to the triangulation type.

  FIG. 8 is a flowchart for explaining a method for adjusting a parameter related to the sensitivity of motion detection in the Z direction by the gesture operation input processing device 100. In the flowchart shown in FIG. 8, the processing procedure of each part is displayed by a combination of S (acronym for Step) meaning a step and a number. In addition, when a determination process is executed in the process displayed by the combination of S and a number, and the determination result is affirmative, Y (acronym for Yes) is added, for example (Y in S14) If the result of the determination is negative, N (the acronym for No) is added and (N in S14) is displayed.

  The display performance acquisition unit 40 acquires data relating to display performance, particularly 3D display performance, from the display database 42 (S10). When the display device 210 is connected to the game machine 200, the driver of the display device 210 is installed in the game machine 200. The display performance acquisition unit 40 makes an inquiry to the display database 42 based on information about the driver of the display device 210 and acquires data related to the three-dimensional display performance. Alternatively, if the driver of the display device 210 includes data related to 3D display performance, the display performance acquisition unit 40 may extract data related to 3D display performance from the driver.

  The gesture recognition parameter adjustment unit 50 sets initial values of the stage number N, the interval D, and the dead bandwidth d, which are adjustment parameters (S12). This initial value is a default value based on the premise that the display has high three-dimensional display performance.

  The display performance acquisition unit 40 determines whether or not the display device 210 is a three-dimensional display based on the data regarding the three-dimensional display performance (S14). If it is a three-dimensional display (Y of S14), it will progress to step S22, without changing the initial value of the stage number N, the space | interval D, and the dead band width d.

  If the display is not a three-dimensional display (N in S14), the gesture recognition parameter adjustment unit 50 decreases the number of stages N and increases the interval D and the dead bandwidth d (S16). Next, the display performance acquisition unit 40 determines whether or not the display device 210 is a stereo display (S18). If the display is a stereo display (Y in S18), the process proceeds to step S22 without changing the values of the stage number N, the interval D, and the dead band width d.

  When the display device 210 is not a stereo display, that is, a two-dimensional display (N in S18), the gesture recognition parameter adjustment unit 50 further decreases the number of stages N and increases the interval D and the dead bandwidth d (S20). .

  Next, the distance sensor performance acquisition unit 30 acquires data relating to the distance resolution of the camera 220 from the distance sensor database 32 (S21). When the camera 220 is connected to the game machine 200, a driver for the camera 220 is installed in the game machine 200. The distance sensor performance acquisition unit 30 makes an inquiry to the distance sensor database 32 based on the information of the driver of the camera 220 and acquires data related to the distance resolution. Alternatively, if the driver of the camera 220 includes data regarding distance resolution, the distance sensor performance acquisition unit 30 may extract data regarding distance resolution from the driver.

  The distance sensor performance acquisition unit 30 determines whether the camera 220 is a triangulation type based on the data related to the distance resolution (S22). If it is a triangulation type (Y in S22), the number of steps N, the interval D, and the dead band width d are varied according to the distance in the depth direction (S24). Specifically, the virtual screen level N is increased as the position is closer to the camera 220, and the virtual screen level N is decreased as the position is farther from the camera 220. Further, since the measurement accuracy of the virtual screen near the camera 220 is high, the interval D between adjacent screens is narrowed and the dead band width d is also narrowed. For the virtual screen far from the camera 220, the measurement accuracy decreases, so the interval D between adjacent screens is widened and the dead band width d is also widened.

  When the camera 220 is not the triangulation type (N in S22), the distance sensor performance acquisition unit 30 determines whether the camera 220 is the TOF type based on the data regarding the distance resolution (S26). In the case of the TOF type (Y in S26), the number N of virtual screen stages, the interval D, and the dead band width d are made constant regardless of the distance in the depth direction, and the total number N of virtual screen stages is increased (S28). If it is not the TOF type (N in S26), the stage number N, the interval D, and the dead band width d are not adjusted any more, and the process proceeds to step S30.

  The gesture recognition parameter adjustment unit 50 supplies the adjusted stage number N, interval D, and dead bandwidth d to the gesture recognition processing unit 60 (S30).

  As described above, according to the 3D gesture input system 300 of this embodiment, the detection sensitivity of the user's gesture, particularly in the depth direction, is appropriately set based on the measurement resolution of the distance sensor and the 3D display performance of the display. By adjusting, the user can input by gesture according to the performance of the system without feeling stress. Even if the combination of the distance sensor and the display changes, the detection sensitivity is automatically adjusted on the system side, so that a seamless and flexible gesture input interface can be provided.

  The present invention has been described based on the embodiments. The embodiments are exemplifications, and it will be understood by those skilled in the art that various modifications can be made to combinations of the respective constituent elements and processing processes, and such modifications are within the scope of the present invention. .

  DESCRIPTION OF SYMBOLS 10 Instruction point extraction part, 20 Distance calculation part, 30 Distance sensor performance acquisition part, 32 Distance sensor database, 40 Display performance acquisition part, 42 Display database, 50 Gesture recognition parameter adjustment part, 52 Virtual screen memory | storage part, 60 Gesture recognition process Unit, 70 display control unit, 80 application execution unit, 100 gesture operation input processing device, 200 game machine, 210 display device, 220 camera, 230 dedicated glasses, 240 instruction point, 250 user, 270 virtual screen, 300 3D gesture input system.

Claims (7)

A distance calculation unit for obtaining a distance in the depth direction of the user's gesture from a gesture image obtained by photographing the user's gesture performed while viewing a three-dimensional display capable of displaying a three-dimensional space that can be stereoscopically viewed;
A parameter adjustment unit that adjusts parameters when recognizing an operation input by a user's gesture based on resolution of distance measurement in the depth direction;
A gesture operation input comprising: a gesture recognition processing unit that recognizes an operation input by a user's gesture with reference to the distance in the depth direction calculated by the distance calculation unit under the adjusted parameter. Processing equipment.   The gesture operation input processing apparatus according to claim 1, wherein the resolution of the distance measurement in the depth direction is information indicating whether or not the gesture image is taken by a stereo camera. A display control unit for displaying a three-dimensional space that can be stereoscopically viewed;
A camera unit for photographing a user's gesture;
A three-dimensional display device, comprising: an output unit that outputs information on resolution of distance measurement in the depth direction of the camera unit necessary for adjusting a parameter when recognizing an operation input by a user's gesture.   The information on the resolution of the distance measurement in the depth direction is information indicating whether or not the three-dimensional display device is provided with a stereo camera for acquiring information on the depth direction of the user's gesture. The three-dimensional display device according to claim 3. A distance calculation step for obtaining a distance in the depth direction of the user's gesture from a gesture image obtained by photographing the user's gesture performed while viewing a three-dimensional display capable of displaying a stereoscopically viewable three-dimensional space;
A parameter adjustment step for adjusting a parameter for recognizing an operation input by a user's gesture based on the resolution of distance measurement in the depth direction;
And a gesture recognition processing step of recognizing an operation input by a user's gesture with reference to the distance in the depth direction calculated by the distance calculation step under the adjusted parameter. Processing method. A distance calculation function for obtaining a distance in the depth direction of the user's gesture from a gesture image obtained by photographing the user's gesture performed while viewing a three-dimensional display capable of displaying a stereoscopically viewable three-dimensional space;
A parameter adjustment function for adjusting parameters when recognizing an operation input by a user's gesture based on the resolution of distance measurement in the depth direction;
A gesture recognition processing function for recognizing an operation input by a user's gesture by referring to the distance in the depth direction calculated by the distance calculation function under the adjusted parameter is realized by the computer. program.   A recording medium storing the program according to claim 6.

Images