JP6102751B2 - Interface device and driving method of interface device - Google Patents

Description

  The present invention relates to an interface device and a driving method of the interface device.

  Conventionally, an interface between a machine and a person such as a computer and a digital device is a button, a keyboard, and a mouse. In recent years, the interface has evolved from the aforementioned to a touch interface that directly touches the screen, and a spatial operation type interface using gestures has also been developed. Examples of the spatial operation type interface include those of Non-Patent Documents 1 to 3.

  Non-Patent Document 1 discloses “SixthSense” developed at Massachusetts Institute of Technology. “SixthSense” recognizes the position of the color sack that the camera puts on the finger by using an interface that combines a small projector and camera, and can perform various interactive operations.

  Non-Patent Document 2 discloses “Omnitouch” developed by Microsoft Corporation. “Omnitouch” is an interface in which a device is placed on a person's shoulder, and is combined with a device called “Xtion” (see Patent Document 1) that performs three-dimensional measurement of a small projector and an object.

  Non-Patent Document 3 discloses “LuminAR” developed at Massachusetts Institute of Technology. “LuminAR” is a light stand type interface, and a small projector and a camera are provided in the light portion of the light stand, and a mechanically moving arm is provided.

US Patent Application Publication No. 2010/0118123

Pranav Mistry, SixthSense, MIT Media Lab, [Searched on December 16, 2011], Internet (URL: http://www.pranavmistry.com/projects/sixthsense) Hrvoje Benko and Scott Saponas, Omnitouch, Microsoft, [Search December 16, 2011], Internet (URL: http://research.microsoft.com/en-us/news/features/touch-101711.aspx) Natan Linder and Pattie Maes, LuminAR, MIT Media Lab, [Searched on December 16, 2011], Internet (URL: http://fluid.media.mit.edu/people/natan/current/luminar.html)

  “SixthSense” and “Omnitouch” require a person to control the position of an image to be projected for operation. In addition, “SixthSense” and “Omnitouch” must move their shoulders, hands, etc. so that an image is projected there, and a person must be wearing a device that performs the interface. “LuminAR” does not require a person to wear anything, but must hold his hand under the light of the light stand. These are traditional interfaces that humans match with machines, and are called ubiquitous types. In the future world, it is said that the interface between machines and people will change to an ambient type interface where machines are adapted to people. For example, machines from large ones such as digital signage to small ones such as mobile terminals and mobile phones will recognize humans and their movements and autonomously interact with the situation. .

  In order to realize an ambient type interface, a machine must have a wide recognition space in order to recognize people. This is because the recognition space is limited, which restricts human movement. You should also avoid using tools like color sack. In this regard, modern technology has many challenges. For example, “Xtion”, currently the most powerful gesture input device, has the following problems.

  FIG. 16 shows the basic principle and problem of “Xtion”. As shown in FIG. 16, “Xtion” includes an infrared light pattern generator 401 and a camera 403. The basic principle of “Xtion” is three-dimensional object authentication by so-called triangulation. The infrared light pattern 402 projected from the infrared light pattern generation device 401 hits an object (in this example, a hand) 404, and image distortion generated according to the shape of the hand 404 is read by the camera 403. A region where the projection range of the infrared light pattern generation device 401 overlaps with the angle of view of the camera 403 is a recognition space, and a region outside this space cannot be recognized. Thus, since the recognition space becomes narrower as it approaches the apparatus, it is likely that the recognition space is easily removed from the space as the distance approaches. As described above, the current technology has a narrow recognition space.

  SUMMARY OF THE INVENTION An object of the present invention is to provide an interface device capable of recognizing an object such as a person in a small and accurate wide recognition area and a driving method of the interface device in order to realize an ambient type interface. is there.

In order to achieve the above object, the interface device of the present invention comprises:
Including image projection means, imaging means, projection direction adjustment means, and control means,
The image projection means includes a light source that emits detection light for detecting an object,
The position of the object is recognized by the imaging means,
By the projection direction adjusting means, the projection direction of the image projected from the image projecting means can be moved according to the position of the recognized object,
The image projection means projects an image of the detection light onto the object,
The image of the detection light is recognized by the imaging means,
The control means controls the image projection means, the imaging means, and the projection direction adjustment means, and in advance according to the state of the detection light image recognized by the imaging means, with respect to the state. A signal for performing a predetermined operation is generated.

The driving method of the interface device of the present invention includes
Using an interface device including an image projection unit, an imaging unit, a projection direction adjustment unit, and a control unit,
An object recognition step of recognizing the position of the object by the imaging means;
A projection direction adjusting step for allowing the projection direction of the image projected from the image projection unit to be moved by the projection direction adjusting unit according to the position of the recognized object;
A detection image projection step of projecting an image of the detection light onto the object by the image projection means;
An image recognition step of recognizing an image of the detection light by the imaging means;
The control means controls the image projection means, the imaging means, and the projection direction adjustment means, and is predetermined for the state according to the state of the image recognized in the image recognition step. And an operation step for generating a signal for performing a predetermined operation.

  According to the present invention, it is possible to provide an interface device capable of recognizing an object such as a person in a small recognition area with high accuracy and a wide recognition area, and a driving method of the interface device, in order to realize an ambient type interface.

FIG. 1 is a block diagram illustrating a configuration of the interface apparatus according to the first embodiment. FIG. 2 is a flowchart of a driving method using the interface device according to the first embodiment. FIG. 3 is a diagram showing a configuration of an example of the projection direction adjusting means in the interface apparatus of the present invention. FIG. 4 is a diagram showing the configuration of another example of the projection direction adjusting means in the interface apparatus of the present invention. FIG. 5 is a diagram showing the configuration of still another example of the projection direction adjusting means in the interface apparatus of the present invention. FIG. 6 is a block diagram illustrating a configuration of the interface apparatus according to the second embodiment. FIG. 7 is an explanatory diagram of a driving method using the interface device of the second embodiment. FIG. 8 is a diagram showing a configuration of an embodiment of a laser projector which is an image projection means in the interface apparatus of the present invention. FIG. 9 is an explanatory diagram showing an example of image rotation means (rotation mechanism) in the interface apparatus of the present invention. FIG. 10 is an explanatory diagram showing an example of an image rotation method when the interface apparatus of the present invention does not have image rotation means. FIG. 11 is a diagram illustrating a case where the projected image is substantially circular. FIG. 12 is a block diagram illustrating a configuration of the interface apparatus according to the fifth embodiment. FIG. 13 is a block diagram illustrating a configuration of the interface apparatus according to the sixth embodiment. FIG. 14 is a block diagram illustrating a configuration of the interface apparatus according to the seventh embodiment. FIG. 15 is a block diagram illustrating a configuration of the interface apparatus according to the eighth embodiment. FIG. 16 is a diagram showing the basic principle and problems of Xtion. FIG. 17 is a diagram illustrating an example of a driving method of a related art interface device.

  Hereinafter, the interface apparatus of the present invention will be described in detail with examples. However, the present invention is not limited to the following embodiments. In the following drawings, the same portions are denoted by the same reference numerals. In the drawings, for convenience of explanation, the structure of each part may be simplified as appropriate, and the dimensional ratio of each part may be different from the actual one.

(Embodiment 1)
FIG. 1 is a block diagram showing the configuration of the interface device of this embodiment. FIG. 2 is a flowchart of a driving method using the interface apparatus according to the present embodiment. As shown in FIG. 1, the interface device 100 of the present embodiment includes an image projection unit 101, an imaging unit 106, a projection direction adjustment unit 105, and a control unit 107 as main components. The image projection unit 101 includes a light source 102. In the present embodiment, the light source 102 is a light source that emits detection light for detecting an object, and an infrared light source or the like can be preferably used. The control means 107 is connected to the image projection means 101, the imaging means 106 and the projection direction adjustment means 105. In FIG. 1, reference numeral 108 denotes the direction of detection light (detection light image) in which the projection direction of the image projected from the image projection unit 101 is changed by the projection direction adjustment unit 105, and reference numeral 109 denotes the detection light. The reflected light reflected by X is shown.

  The driving method using the interface apparatus of the present embodiment performs the following steps using the interface apparatus of FIG. 1 as shown in the flowchart of FIG.

  First, the position of the object is recognized by the imaging means 106 (object recognition process: step S10). The object is, for example, a part of a human body such as a person, a hand, a finger, a face, or a torso. The imaging means 106 includes, for example, a CCD (Charge Coupled Device) camera, a CMOS (Complementary Metal Oxide Semiconductor) camera, and the like. In recent years, image recognition technology has advanced, and face authentication has become commonplace even in general digital cameras. Recognition of the body part is possible as well. With image recognition, for example, once a hand is captured, it is possible to track even if an operation is performed so as to depart from the initial angle of view of the imaging means 106. It is also possible to periodically move the position of the angle of view of the imaging means 106 to find a person or a part of the person's body.

  Next, the projection direction adjustment unit 105 moves the projection direction of the image (detection light image) projected from the image projection unit 101 according to the position of the recognized object X (projection direction adjustment step: Step S20). If the object X is moving, tracking may be performed. The projection direction adjusting unit 105 will be described later.

  And the detection light image for detecting a detailed position and a motion is projected on the target object X from the image projection means 101 (detection image projection process: step S30). As the image projection unit 101, for example, a laser projector or the like can be used.

  Next, the imaging means 106 acquires an image (reflected light) of the detection light reflected by the object X, recognizes the image, and recognizes the state, such as the shape, operation, and position of the object X ( Image recognition process: Step S40). In the present embodiment (FIG. 1), the same imaging means 106 is used in the object recognition step S10 and the image recognition step S40, but the present invention is not limited to this. It is also possible to employ a configuration in which separate imaging means are used in the two steps.

  Operations and actions of the image projection unit 101, the imaging unit 106, and the projection direction adjustment unit 105 are controlled by the control unit 107. Then, the control means 107 issues a signal for performing a predetermined operation in advance with respect to the detected state of the object X such as shape, operation, position, etc. (operation process: step S50). By including the configuration of the present embodiment, the interface device that uses the state of the object X such as the shape, motion, and position as an input signal can appropriately respond to the movement of the object. The interface apparatus of the present invention can include means for outputting an operation signal for controlling an electronic device or the like according to the detected movement of the target object. In this case, the electronic apparatus can be accurately and in a wide operation range. It becomes possible to operate devices and the like.

  The projection direction adjusting unit 105 of the present embodiment is a unit that adjusts the projection direction of the image projected from the image projecting unit 101, and can be configured as follows.

  FIG. 3 is an example of the projection direction adjusting means in the interface apparatus of the present invention. The projection direction adjusting means of this example includes a lens, and the lens can adjust the tilt angle with respect to the image projecting means and can rotate. In FIG. 3, the lens 1201 is attached to the lens holder 1202. The lens holder 1202 is connected to driving means (for example, a linear motor) 1203 that can adjust the inclination angle of the lens 1201. The driving unit 1203 is connected to the support 1204. The support 1204 can be rotated by another driving means (for example, a motor) 1205. By driving the driving means 1203 and 1205, the lens 1201 can be tilted and rotated, and the lens 1201 can be moved in a wide range forward in the projection direction. By moving the lens 1201, the direction of the projection light 1206 from the projection lens 813 can be tracked and moved by the movement of the recognized object.

  FIG. 4 shows another example of the projection direction adjusting means in the interface apparatus of the present invention. The projection direction adjusting means of this example includes a lens, and the lens can adjust the tilt angle with respect to the image projecting means and can rotate. In FIG. 4, a lens 1301 is attached to a lens support 1302 having a spherical surface. The periphery of the lens support 1302 is supported by a ring-shaped support portion 1303. The support portion 1303 has at least a pair of drive means (for example, ultrasonic motors) 1304 and 1305 on the inner side so that the lens support 1302 can rotate. In this example, a substantially spherical lens support 1302 is driven by an ultrasonic motor. The lens support 1302 is preferably transparent. The ring-shaped support portion 1303 is preferably provided with a bearing or the like at a predetermined position inside. In FIG. 4, the driving means 1304 and 1305 are preferably ultrasonic motors that are formed between the support portion 1303 and the lens support 1302 and rotate the lens support 1302 in the directions of the arrows, respectively. By driving the driving means 1304 and 1305, the lens 1301 can be tilted and rotated, and the lens 1301 can be moved in a wide range forward in the projection direction. By moving the lens 1301, the direction of the projection light 1306 from the projection lens 813 can be tracked and moved by the movement of the recognized object. Further, according to this projection direction adjusting means, it is possible to make the mechanical system for changing the direction of the lens flat and further reduce the overall size. Further, the lens support 1302 is spherical, so that no axis is required. Moreover, since the motor is planarized, the volume increase caused by the projection direction adjusting means can be reduced, which is preferable.

  FIG. 5 shows still another example of the projection direction adjusting means in the interface apparatus of the present invention. The projection direction adjusting means of this example has a mirror, and the mirror can adjust the tilt angle with respect to the image projecting means and can rotate. In FIG. 5, the mirror 1401 has a rotating shaft that can rotate in the direction of the inclination angle, and a driving means (motor) 1402 is attached to the rotating shaft. The driving means (motor) 1402 is connected to another driving means (motor) 1404 via the support 1403. The drive unit (motor) 1404 is connected so that the mirror 1401 can rotate about the direction in which the image is projected from the image projection unit. By driving the driving means 1402 and 1404, the mirror 1401 can be tilted and rotated, and the mirror 1401 can be moved in a wide range forward in the projection direction. By moving the mirror 1401, the direction of the projection light 1405 from the projection lens 813 can be moved following the movement of the recognized object.

(Embodiment 2)
FIG. 6 is a block diagram illustrating a configuration of the interface apparatus according to the present embodiment. FIG. 7 is an explanatory diagram of a driving method using the interface device of the present embodiment. As shown in FIG. 6, the interface apparatus 600 of this embodiment includes an image projection unit 601, a light separation unit 604, a projection direction adjustment unit 605, an imaging unit 606, and a control unit 607 as main components. Yes. The image projection unit 601 includes a light source 602 and an image rotation unit 603. The light source 602 includes, for example, a visible light source that outputs visible light of red light, green light, and blue light, and an infrared light source that outputs infrared light, and the light source is, for example, a semiconductor laser. The infrared light source may be a light source that emits detection light for detecting an object. As the image projection unit 601, for example, a laser projector can be used. The light separation means 604 is a detection light separation optical system for image pickup means that separates the reflected light of the detection light out of the reflected light of the projection image and enters the image pickup means such as a camera. The projection direction adjusting unit 605 is an optical system having a direction variable structure. The control unit 607 is connected to the image projection unit 601, the imaging unit 606, and the projection direction adjustment unit 605. In FIG. 6, reference numeral 608 denotes the direction of detection light (detection light image) in which the projection direction of the image projected from the image projection unit 601 is changed by the projection direction adjustment unit 605, and reference numeral 609 denotes the detection light. The path of return light (reflected light) reflected by X is shown. In addition, in FIG. 6, illustration of the target object X located in the upper right part of the projection direction adjustment means 605 is abbreviate | omitted. Other configurations are the same as those of the interface apparatus described in the first embodiment.

  In the present embodiment, an image emitted from an image projection unit (laser projector) 601 having a light source 602 and an image rotation unit 603 therein is changed in direction and projected by a projection direction adjustment unit 605 through a light separation unit 604. . In order to perform three-dimensional measurement of the object X, the light source 602 emits infrared light. At this time, the light returned upon hitting the object X passes through the path 609 and enters the imaging unit 606 through the light separation unit 604. The control means 607 controls these series of operations.

  FIG. 7 shows the basic operation of the interface device of this embodiment. In FIG. 7, reference numeral 701 denotes an interface device (only a head portion is shown) of the present embodiment, 702 is an angle of view of an imaging unit (camera system) 606, 703 is a hand as an example of the object X, and 704 is infrared projection light. 705, a pattern of infrared light on the hand projected by the light, 706 rotating the image rotating means 603, and 707 a projected image (visible light).

  The operation is as follows. As shown in FIG. 7A, the imaging means (camera) 606 inside the interface device 701 recognizes the position of the hand or finger performing the gesture. The recognition may be performed, for example, by periodically scanning the projection direction adjusting unit 605. In addition, once a target such as a hand or a finger is recognized, the projection direction adjusting unit 605 may be operated so as to track the recognized target.

  Next, as shown in FIG. 7B, the infrared projection light 704 whose direction is changed by the projection direction adjusting means (an optical system having a direction variable structure) 605 as necessary is projected toward the hand 703. . As the recognition method, for example, in the case of a monocular camera, a time-of-flight method, a phase difference method, or the like can be used. A triangulation method can also be used in a method in which a camera is provided in another part described later.

  According to the position of the hand or finger read in this procedure, it is preferable to project an operation image in a direction that is easy to operate. Therefore, in FIG. 7C, the direction of the projection image 707 to be projected next is optimized by the image rotation means 603. Then, as shown in FIG. 7D, a projection image (visible light) 707 is projected. Some action is performed according to the gesture of the person with respect to this image. In this example, the operation is to select an image-projected option. Thus, according to the present embodiment, an interactive operation is possible. The above is an explanation of an example in which an action is performed using a projected image, but it is also possible to control other things simply by inputting a gesture.

  As described above, in the present embodiment, when the direction in which the user (the person who is going to perform the operation) accesses is different from, for example, the direction of the image initially set by the device, the position of the body or hand is recognized. By doing so, it is possible to rotate the direction of image projection so as to be directed to an optimum position. On the other hand, with the related technology, it is difficult to direct the projected image to the optimum position without causing a decrease in luminance. In FIG. 17, reference numeral 501 denotes a small projector, 502 denotes a projection image range, and 503 denotes a projection image using a part thereof. In this way, when trying to match the direction of the person, the related technology has to control the image using a part of the projection image. In the ambient type interface, since the shape and position of the projection target are not constant, it is necessary to change the projection image according to the shape and position, and this is realized by using a part of a larger projection range. The brightness of the projected image is inversely proportional to the projected area. Therefore, in the related technique, in such a usage pattern, the image becomes unnecessarily dark. The ability to perform the operation as in this embodiment is important for an ambient type interface in which a machine recognizes a person and its movement and works autonomously. Therefore, the interface device of this embodiment is suitable for an ambient type interface. It can be made to correspond.

(Embodiment 3)
FIG. 8 is a diagram showing an embodiment of a laser projector portion which is an image projection means in the interface apparatus of the present invention. As the image projection means in the interface apparatus of the present invention, for example, a small projector can be used. There are several types of projectors, but the projectors can be roughly classified into those using LEDs as light sources and those using lasers as light sources. The LED type is a reflective small liquid crystal display that uses DMD (digital micromirror device) or Si chip for image modulation because it cannot scan the light itself because of its size. Light is modulated using a surface element such as a certain LCOS (Liquid crystal on silicon). In addition to the above-described method using DMD or LCOS, the laser type includes a method of scanning a small mirror to scan the beam itself. The biggest difference between the LED type and the laser type is that the LED type needs to be focused by a projection lens, and it is impossible to focus on the entire surface when projecting to an oblique or curved surface. On the other hand, the laser type means that focus adjustment is not necessary within the practical range of the laser.

  As shown in FIG. 8, the laser projector of this embodiment includes a green semiconductor laser 801, a blue semiconductor laser 802, a red semiconductor laser 803, and an infrared semiconductor laser 804. Light emitted from these lasers is shaped by a collimator lens 805, synthesized by a mirror 806 and a dichroic mirror 807, and incident on a homogenizer 808 with a rotation mechanism. The role of the homogenizer 808 is to shape the beam in accordance with the shape of a modulation element to be described later, and to rotate the projection image by a rotation mechanism, and corresponds to the image rotation means of the present invention. The light exiting the homogenizer 808 enters the lens system 809 and then enters the polarization beam splitter 810. The light originally emitted from the laser is polarized in one direction, and most of the light is reflected here and enters the LCOS 811. The table 812 on which the LCOS 811 is mounted has a rotation mechanism. The light modulated and reflected by the LCOS 811 again enters the polarization beam splitter 810 and is projected through the projection lens 813. Reference numeral 814 represents the projection light.

  The LCOS 811 is an element that is driven in a time division manner. In this case, green, blue, red, and infrared light are projected at time intervals. In addition, since the light source is a point light source such as a laser, an image that does not require focus adjustment within a practical range can be obtained although there is a projection lens. In addition, unlike a method of using a scanning element to oscillate a laser beam, legal regulations on laser light are reduced, so that brighter light can be projected. Although depending on the conditions, it is possible to obtain a brightness of several hundreds of lumens and display an image that is about one digit brighter than the beam scanning method.

  The rotation mechanism that is the image rotation means in the present invention will be described in more detail with reference to FIG. FIG. 9A is a perspective view showing only a homogenizer 808 with a rotation mechanism, an LCOS 811, a table 812 with a rotation mechanism on which it is mounted, a polarization beam splitter 810, and a projection lens 813. In FIG. 9B, 901 indicates the initial position of the table 812 with a rotation mechanism, and 902 indicates the position of the illumination light projected by rotation by the homogenizer 808 with the rotation mechanism. When the homogenizer 808 with a rotation mechanism and the table 812 with a rotation mechanism rotate in a coordinated manner, the projection light also rotates while maintaining its shape in accordance with the rotation of the LCOS 811, and an image can be projected through the projection lens 813. As a result, the image is rotated.

(Embodiment 4)
FIG. 10 shows an example of a method that can rotate an image without providing a rotation mechanism. FIG. 10A is a perspective view showing only the homogenizer 1001, the LCOS 811, the stage 1002 on which the homogenizer 1001 is mounted, the polarization beam splitter 810, and the projection lens 813. The difference from the method shown in FIG. 9 is that the homogenizer 1001 forms substantially circular light and has no rotation mechanism, and the LCOS 811 base 1002 has no rotation mechanism. As a result, as shown in FIG. 10B, the illumination light 1003 on the LCOS 811 has a substantially circular shape, and the projection image necessarily has a substantially circular shape. As described above, although a region where partial irradiation is not performed appears, the light loss is small compared to the case where the projection image is a rectangle or a square, and bright projection is possible.

  Advantages when the projected image is substantially circular will be described with reference to FIG. As shown in FIG. 11A, even if the first projected image 1101 is oriented in an inappropriate direction with respect to the human hand 1102, only the image data is converted into an appropriate form without using mechanical rotation. By converting, it is possible to project an image in the optimum direction (projected image 1103). This is also a preferable aspect as an ambient type display.

(Embodiment 5)
FIG. 12 shows an interface apparatus according to the fifth embodiment. This embodiment is in accordance with the configuration of the interface device of the second embodiment shown in FIG. As shown in FIG. 12, the interface apparatus of this embodiment has an image projection means, an imaging means, a light separation means, and a projection direction adjustment means (lens 1513 with a direction variable mechanism). Reference numeral 1514 denotes projection light. The image projecting means includes a light source 1501 having a blue, green, red, and infrared laser as light source elements, a mirror 1502, and a despeckle 1503 including a diaphragm for reducing speckle noise, which is one of the problems of laser projectors. , A homogenizer 1504 with a rotation mechanism, a lens system 1505, a polarization beam splitter 1506, an LCOS 1507, and a table 1508 with a rotation mechanism. The imaging means includes a camera 1511 and a camera lens system 1510. A polarization beam splitter 1509 that guides reflected signal light from an object to imaging means (camera 1511, camera lens system 1510) is the light separation means. The position of the projection lens 1512 may be between the two polarizing beam splitters 1506 and 1509. The components are arranged so that the whole is compact. In this case, the camera 1511 is used for both the image recognition of the first object and the infrared pattern recognition of the object. Although the despecula 1503 is preferable, it is not an essential component, and this is the same in the following embodiments. Further, it is preferable that the image projection unit and the imaging unit are optically on the same axis. In the interface device of this embodiment, for example, the object may be recognized by scanning the lens 1513 with a direction variable mechanism periodically. In addition, once the object is recognized, the direction variable mechanism-equipped lens 1513 may be operated so as to track the recognized object.

(Embodiment 6)
FIG. 13 shows an interface apparatus according to the sixth embodiment. As shown in FIG. 13, the interface apparatus according to the present embodiment includes an image projection unit, an imaging unit, and a projection direction adjustment unit (lens 1613 with a direction variable mechanism). Reference numeral 1614 denotes projection light. The image projecting means includes a light source 1601 having blue, green, red, and infrared lasers as light source elements, a mirror 1602, and a despeckler 1603 including a diaphragm for reducing speckle noise, which is one of the problems of laser projectors. , A homogenizer 1604 with a rotation mechanism, a lens system 1605, a polarizing beam splitter 1606, an LCOS 1607, a stand 1608 with a rotation mechanism, and a projection lens 1609. The imaging means has a camera 1612 and a camera wide-angle lens 1610, and 1611 indicates an angle of view. The difference from the fifth embodiment is that the imaging means (camera system) is separated. In the case of the projection system, as described above, if the direction is not narrowed, it becomes very dark and cannot be made wide angle. However, in the camera system, it is possible to make wide angle. In this example, it is possible to perform a three-dimensional survey using triangulation using the difference in position between the projection system and the camera system.

(Embodiment 7)
FIG. 14 shows an interface apparatus according to the seventh embodiment. As shown in FIG. 14, the interface apparatus of the present embodiment includes an image projection unit, an imaging unit, and a projection direction adjustment unit (mirror 1713 with a direction variable mechanism). Reference numeral 1714 denotes projection light. The image projecting means is a despeckle 1703 comprising a light source 1701, a mirror 1702, and a diaphragm for reducing speckle noise, which is one of the problems of a laser projector, using blue, green, red, and infrared lasers as light source elements. , A homogenizer 1704 with a rotation mechanism, a lens system 1705, a polarization beam splitter 1706, an LCOS 1707, a stand 1708 with a rotation mechanism, and a projection lens 1719. The imaging means has a camera 1712 and a camera wide-angle lens 1710, and 1711 indicates an angle of view. The present embodiment is an example of a form in which the main element of the projection direction adjusting means (direction variable optical system) is a mirror. Other basic components are the same as those shown in FIG.

(Embodiment 8)
In the above embodiment, description has been made of using LCOS as an element for forming an image. However, a DMD (digital micromirror device) type can be used in the same manner as in the above embodiment. In addition, although the brightness is inferior as described above, the same projection direction adjusting means (direction variable optical system) as in the above-described embodiment can be applied to a scanning projector. FIG. 15 shows an example of the embodiment. As shown in FIG. 15, the interface device of the present embodiment includes an image projection unit, an imaging unit, and a projection direction adjustment unit (mirror 1806 with a direction variable mechanism). Reference numeral 1807 denotes projection light. The image projection means has a light source 1801 having blue, green, red, and infrared lasers as light source elements, and a scanning system 1802 having the ability to scan the beam horizontally and vertically, and at least a mechanism for rotating the image. have. The scanning system 1802 is formed of MEMS (Micro Electro Mechanical Systems) or the like. The imaging means includes a camera 1805 and a camera wide-angle lens 1803, and 1804 indicates a field angle. As described above, in the case of the scanning element type, since the scanning element is small, it is possible to provide a mechanism for changing the direction of the scanning element type. It is possible.

  In the above embodiments, the projectors are all laser projectors. However, if the projectors are operated only in a very narrow range, LED projectors are also practically used. Even in this case, the function that can change the direction of the image has an ambient type characteristic that the machine autonomously works on the person.

  As described above, the interface device of the present invention can widen the operating range, as described by exemplifying the embodiments. When the target object moves, for example, when the hand moves out of the angle of view of the camera, the projection direction adjusting means is controlled and directed in the direction of movement of the hand to track the movement in a wide area. Can be covered. In addition, the recognition pattern that is projected when recognizing a gesture by measuring the object three-dimensionally can be varied depending on the position of the object, so that an optimum pattern can always be projected, and the operating range in the depth direction is also Can be expanded.

  In addition, the device can be easily downsized. As the main part of the projection direction adjusting means, for example, a small and low-weight member such as a mirror or a lens can be used, so that the mechanical system for driving it can be very small. The difference is obvious when considering a mechanism for changing the direction of the combination of the projector and the camera as in the prior art.

  Furthermore, it is possible to improve the followability to human movement. It is easy to give quick response by the projection direction adjusting means described above. In particular, when a hand or the like that is an object is close to the apparatus, this high-speed response is very effective.

  It is also possible to improve recognition accuracy. In the present invention, it is possible to capture an object in front and to dynamically change the recognition pattern with respect to the depth, and it is possible to always project a pattern having an optimum density. As a result, the three-dimensional measurement accuracy of the object can be kept constant, and deterioration in accuracy with respect to the distance can be prevented.

  Furthermore, in the aspect which presents a projected image, it is possible to project a bright projected image in an appropriate direction. The projection direction adjusting means can be controlled and directed in the direction in which the object moves, and an image is not projected onto an unnecessary portion, so that the brightness can be increased. Further, by having the image rotation means, it is possible to project an image in an appropriate direction with respect to a person.

  As described above, the interface device and the driving method of the interface device of the present invention can provide an interface that is small and can accurately recognize an object such as a person in a wide recognition area. That is, according to the present invention, it is possible to provide an ambient type interface in which a machine recognizes a human motion and can autonomously and accurately convey information as a video. Applications of the interface device and the driving method of the interface device of the present invention are not limited to this, and examples include digital signage, PCs, tablet terminals, conference room and home appliance control. However, its use is not limited and can be applied to a wide range of fields.

  A part or all of the above embodiment can be described as in the following supplementary notes, but is not limited to the following.

(Appendix 1)
Including image projection means, imaging means, projection direction adjustment means, and control means,
The image projection means includes a light source that emits detection light for detecting an object,
The position of the object is recognized by the imaging means,
By the projection direction adjusting means, the projection direction of the image projected from the image projecting means can be moved according to the position of the recognized object,
The image projection means projects an image of the detection light onto the object,
The image of the detection light is recognized by the imaging means,
The control means controls the image projection means, the imaging means, and the projection direction adjustment means, and in advance according to the state of the detection light image recognized by the imaging means, with respect to the state. An interface device in which a signal for performing a predetermined operation is issued.

(Appendix 2)
The interface device according to appendix 1, wherein the image projection unit further includes a light source that emits visible light.

(Appendix 3)
The imaging means includes a light separating means for separating the reflected light of the detection light from the reflected light of the projection image from the image projecting means, and the image projecting means and the imaging means are optically on the same axis. The interface device according to Appendix 2, above.

(Appendix 4)
The interface apparatus according to appendix 1 or 2, wherein the imaging unit has a wide-angle lens system and is installed at a position separated from the image projection unit.

(Appendix 5)
The interface apparatus according to any one of appendices 1 to 4, wherein the image projection unit includes an image rotation unit.

(Appendix 6)
Furthermore, it has a modulation element for modulating an image,
The modulation element is a planar element,
The image rotation means has a homogenizer that guides irradiation light having an irradiation area substantially the same as the surface of the modulation element,
Synchronously with the rotation of the stage on which the modulation element is mounted, the homogenizer rotates so that the modulation element can be irradiated with irradiation light of the same area.
The interface device according to appendix 5.

(Appendix 7)
Furthermore, it has a modulation element for modulating an image,
The modulation element is a planar element,
The image projection means has a homogenizer capable of irradiating the modulation element with substantially circular irradiation light,
The image is rotated by converting the image signal.
The interface device according to any one of appendices 1 to 4.

(Appendix 8)
The projection direction adjusting means has a lens,
The lens is capable of adjusting an inclination angle with respect to the image projecting means and is rotatable.
The interface device according to any one of appendices 1 to 7.

(Appendix 9)
The lens is attached to a lens support having a spherical surface;
The lens support is supported by a ring-shaped support portion around the lens support,
The support portion has at least a pair of driving means on the inside thereof so that the lens support body can rotate.
The interface device according to appendix 8.

(Appendix 10)
The projection direction adjusting means has a mirror,
The mirror is capable of adjusting an inclination angle with respect to the image projection means and is rotatable.
The interface device according to any one of appendices 1 to 7.

(Appendix 11)
The light source includes a visible light source that outputs visible light of blue light, green light, and red light, and an infrared light source that outputs infrared light,
The visible light source and the infrared light source are semiconductor lasers;
The interface device according to any one of appendices 1 to 10.

(Appendix 12)
The image projection means is a scanning type by a scanning element,
The interface device according to any one of appendices 1 to 11.

(Appendix 13)
Using an interface device including an image projection unit, an imaging unit, a projection direction adjustment unit, and a control unit,
An object recognition step of recognizing the position of the object by the imaging means;
A projection direction adjusting step for allowing the projection direction of the image projected from the image projection unit to be moved by the projection direction adjusting unit according to the position of the recognized object;
A detection image projection step of projecting an image of the detection light onto the object by the image projection means;
An image recognition step of recognizing an image of the detection light by the imaging means;
The control means controls the image projection means, the imaging means, and the projection direction adjustment means, and is predetermined for the state according to the state of the image recognized in the image recognition step. An interface device driving method including an operation step of generating a signal for performing a predetermined operation.

(Appendix 14)
The object is part of a human part;
In the object recognition step, the imaging means recognizes the movement of the object,
In the object recognition step, the projection direction adjustment step may be configured to prevent the object from deviating from the angle of view of the imaging unit when the object is predicted to move away from the angle of view of the imaging unit. Including controlling and directing the projection direction adjusting means in the moving direction of
In the image recognition step, a three-dimensional measurement step of performing a three-dimensional measurement of the object by reading a reflection pattern from the object of an image of detection light projected onto the object by the imaging unit. Including
further,
An image rotation step of recognizing the state of the object, detecting the position of the person, and rotating the image in an optimal direction according to the position of the person;
The driving method of the interface device according to appendix 13, including a visible light image projecting step of projecting a visible light image from the image projecting unit.

(Appendix 15)
The object recognition step is performed by periodically scanning the projection direction adjusting means, and once the object is recognized, the object recognition process operates to track the recognized object.
15. A driving method of an interface device according to appendix 13 or 14.

(Appendix 16)
The object recognition step is performed by an imaging unit having a wide-angle lens.
16. A method for driving an interface device according to any one of appendices 13 to 15.

  Although the present invention has been described with reference to the exemplary embodiments and examples, the present invention is not limited to the above exemplary embodiments and examples. Various changes that can be understood by those skilled in the art can be made to the configuration and details of the present invention within the scope of the present invention.

  This application claims the priority on the basis of Japanese application Japanese Patent Application No. 2012-12159 for which it applied on January 24, 2012, and takes in those the indications of all here.

100, 600, 701 Interface device 101, 601 Image projection means 102, 602, 1501, 1601, 1701, 1801 Light source 105, 605, 1513, 1613, 1713, 1806 Projection direction adjustment means 106, 606 Imaging means 107, 607 Control means 108, 608, 1514, 1614, 1714, 1807 Projected light 109, 609 Reflected light 401 Infrared light pattern generator 402 Infrared light pattern 403 Camera 404, 703, 1102 Object (hand)
501 Small projector 502 Projected image range 503, 1101, 1103 Projected image (visible light)
603 Image rotation means 604 Light separation means 702 Angle of view 704 Infrared projection light 705 Infrared light pattern 706 Rotation of image rotation means 707 Projected image (visible light)
801 Green semiconductor laser 802 Blue semiconductor laser 803 Red semiconductor laser 804 Infrared semiconductor laser 805 Collimate lens 806, 1502, 1602, 1702 Mirror 807 Dichroic mirror 808, 1504, 1604, 1704 Homogenizer with rotation mechanism 809, 1505, 1605, 1705 Lens System 810, 1506, 1509, 1606, 1706 Polarizing beam splitter 811, 1507, 1607, 1707 LCOS
812, 1508, 1608, 1708 Tables with rotation mechanism 813, 1512, 1609, 1719 Projection lens 814 Projection light 902, 1003 Illumination light 1001 Homogenizer 1002 Base 1201, 1301 Lens (Projection direction adjusting means)
1202 Lens holder 1203, 1205, 1304, 1305, 1402, 1404 Driving means 1204, 1403 Support body 1206, 1306, 1405 Projection light 1302 Lens support body 1303 Support section 1401 Mirror (projection direction adjusting means)
1503, 1603, 1703 Despecra 1510 Camera lens system 1511, 1612, 1712, 1805 Camera 1610, 1710, 1803 Wide angle lens 1611, 1711, 1804 Camera angle 1802 Scanning system

Claims (11)

Including image projection means, imaging means, projection direction adjustment means, and control means,
The image projection means includes a light source that emits detection light for detecting an object,
The position of the object is recognized by the imaging means,
By the projection direction adjusting means, the projection direction of the image projected from the image projecting means can be moved according to the position of the recognized object,
The image projection means projects an image of the detection light onto the object,
The image of the detection light is recognized by the imaging means,
The control means controls the image projection means, the imaging means, and the projection direction adjustment means, and in advance according to the state of the detection light image recognized by the imaging means, with respect to the state. A signal to perform a predetermined operation is issued ,
Satisfy the following Symbol condition (2), the interface device.

Condition (2): further comprising a modulation element for modulating the image,
The modulation element is a planar element,
The image projection means has a homogenizer that can irradiate the modulation element with substantially circular irradiation light, and rotates the image by converting the image signal.
The interface device according to claim 1, wherein the image projection unit further includes a light source that emits visible light. The imaging means includes a light separating means for separating the reflected light of the detection light from the reflected light of the projection image from the image projecting means, and the image projecting means and the imaging means are optically on the same axis. The interface device according to claim 2, which is above. The interface apparatus according to claim 1, wherein the imaging unit has a wide-angle lens system and is installed at a position separated from the image projection unit. The projection direction adjusting means has a lens,
The lens is capable of adjusting an inclination angle with respect to the image projecting means and is rotatable.
The interface device according to any one of claims 1 to 4. The lens is attached to a lens support having a spherical surface;
The lens support is supported by a ring-shaped support portion around the lens support,
The support portion has at least a pair of driving means on the inside thereof so that the lens support body can rotate.
The interface device according to claim 5. The projection direction adjusting means has a mirror,
The mirror is capable of adjusting an inclination angle with respect to the image projection means and is rotatable.
The interface device according to any one of claims 1 to 4. Using an interface device including an image projection unit, an imaging unit, a projection direction adjustment unit, and a control unit,
An object recognition step of recognizing the position of the object by the imaging means;
A projection direction adjusting step for allowing the projection direction of the image projected from the image projection unit to be moved by the projection direction adjusting unit according to the position of the recognized object;
A detection image projection step of projecting an image of the detection light onto the object by the image projection means;
An image recognition step of recognizing an image of the detection light by the imaging means;
The control means controls the image projection means, the imaging means, and the projection direction adjustment means, and is predetermined for the state according to the state of the image recognized in the image recognition step. An operation step for generating a signal for performing a predetermined operation ,
Satisfy the following Symbol condition (2), the driving method of the interface device.

Condition (2): wherein the interface device further includes a modulation element for modulating the image,
The modulation element is a planar element,
The image projection means has a homogenizer that can irradiate the modulation element with substantially circular irradiation light, and rotates the image by converting the image signal.
The object is part of a human part;
In the object recognition step, the imaging means recognizes the movement of the object,
In the object recognition step, the projection direction adjustment step may be configured to prevent the object from deviating from the angle of view of the imaging unit when the object is predicted to move away from the angle of view of the imaging unit. Including controlling and directing the projection direction adjusting means in the moving direction of
In the image recognition step, a three-dimensional measurement step of performing a three-dimensional measurement of the object by reading a reflection pattern from the object of an image of detection light projected onto the object by the imaging unit. Including
further,
An image rotation step of recognizing the state of the object, detecting the position of the person, and rotating the image in an optimal direction according to the position of the person;
The interface device driving method according to claim 8, further comprising: a visible light image projecting step of projecting a visible light image from the image projecting unit. Including image projection means, imaging means, projection direction adjustment means, and control means,
The image projection means includes a light source that emits detection light for detecting an object,
The position of the object is recognized by the imaging means,
The projection direction of the image projected from the image projection unit by the projection direction adjustment unit, It can be moved according to the position of the recognized object,
The image projection means projects an image of the detection light onto the object,
The image of the detection light is recognized by the imaging means,
By the control means, the image projection means, the imaging means, and the projection direction adjustment means are In accordance with the state of the image of the detection light that is controlled and recognized by the imaging means, A signal is issued to perform a predetermined action for the state,
The projection direction adjusting means has a lens,
The lens is capable of adjusting an inclination angle with respect to the image projection means and is rotatable. Yes,
The lens is attached to a lens support having a spherical surface;
The lens support is supported by a ring-shaped support portion around the lens support,
The support portion includes at least a pair of inner portions so that the lens support body can rotate. Having drive means,
An interface device that satisfies the following condition (1).

Condition (1): The image projection means has an image rotation means.
An interface including image projection means, imaging means, projection direction adjustment means, and control means Using the device,
An object recognition step of recognizing the position of the object by the imaging means;
The projection direction adjusting unit determines the projection direction of the image projected from the image projecting unit. A projection direction adjustment step that allows movement according to the position of the identified object;
A detection image projection step of projecting an image of the detection light onto the object by the image projection means; ,
An image recognition step of recognizing an image of the detection light by the imaging means;
The control means causes the image projection means, the imaging means, and the projection direction adjustment means to In accordance with the state of the image that is controlled and recognized in the image recognition step, An operation step for generating a signal for performing a predetermined operation for the
The projection direction adjusting means has a lens,
The lens is capable of adjusting an inclination angle with respect to the image projection means and is rotatable. Yes,
The lens is attached to a lens support having a spherical surface;
The lens support is supported by a ring-shaped support portion around the lens support,
The support portion includes at least a pair of inner portions so that the lens support body can rotate. Having drive means,
A driving method of an interface device that satisfies the following condition (1).

Condition (1): The image projection means has an image rotation means.