JP6390736B2 - Spatial input device - Google Patents

Description

  The present invention relates to a spatial input device.

  Conventionally, a spatial input device that realizes a virtual user interface is known.

  For example, in Patent Document 1, when an image is displayed by a display element arranged for each unit region corresponding to each microlens of the microlens array unit, the displayed image is displayed on a plane in space by the microlens array unit. An input / output device that forms an image is disclosed. In this input / output device, light from an object for instruction such as a finger is collected by a microlens array unit and then received by an image sensor arranged in the same plane as the display element, and image data of the object is obtained. To be acquired. Based on the imaging data, the position of the object is detected.

JP 2010-55507 A

  However, although the input / output device of Patent Document 1 can detect the three-dimensional position of an object, there is a problem that the structure is complicated because a microlens array unit and a plurality of imaging elements are required.

  Therefore, an object of the present invention is to provide a spatial input device capable of detecting a depth position with high accuracy and high speed with a simple structure.

  In order to achieve the above object, a spatial input device according to an aspect of the present invention includes a light beam scanning unit that emits a predetermined light beam toward an image projected onto space and performs two-dimensional scanning, and a detection. A light detection unit which is set so that one end of the range is parallel to or coincides with the image, and the detection range is provided on the back side of the one end of the detection range when viewed from the direction in which the operation article enters the image; The number of scanning lines, which is the number of the light beams scanned by the beam scanning unit and reflected by the operation object and detected by the light detection unit, is counted, and the depth position is detected based on the counted number of scanning lines. And a depth detection unit.

  According to such a configuration, it is possible to detect the depth position of the operation article that has entered deeper than the image with high accuracy and high speed using one light detection unit. Therefore, it is possible to detect the depth position with high accuracy and high speed with a simple structure.

  In the above configuration, in the scanning by the light beam scanning unit, the light at one end of the detection range is based on the timing at which the light beam is reflected by the operation object at the end or first and detected by the light detection unit. It is good also as providing the touch coordinate detection part which detects the irradiation position coordinate of a beam as a touch coordinate.

  According to such a configuration, in addition to the depth position, the position where the image is touched with the operation article can be detected using one light detection unit, and the cost is reduced.

  In the above configuration, the correspondence between the number of scanning lines and the depth position may be changed according to the touch coordinates.

  According to such a configuration, even when the density of the scanning line changes according to the scanning position, variation in the depth detection result due to the touch position can be suppressed, and a sense of discomfort to the user can be suppressed.

  Further, in any one of the above-described configurations, it may further include a processing unit that switches processing depending on whether the detected depth position is deeper than a predetermined depth position after the touch coordinates are detected. Good.

  According to such a configuration, it is possible to change the processing depending on whether the operation article is inserted deeper than the image and the operation article is further pushed in or when the operation article is stopped at a predetermined depth position.

  In the above configuration, when the detected depth position is behind the predetermined depth position, the processing unit may move a predetermined image to the detected touch coordinates.

  According to such a configuration, when the operation article is moved deeper than the image and the operation article is further pushed in, it is possible to shift to a drag mode in which a predetermined image follows the position of the operation article.

  In any of the above-described configurations, a detection unit may be further provided that determines that a pull-out operation has been performed when the plurality of touch coordinates are detected and is no longer detected by the light detection unit.

  According to such a configuration, it is possible to detect an operation of pulling a plurality of operation articles deeper than the image and then pulling them out to the front as a pulling operation.

  Further, in the above configuration, the second light detection unit having a detection range that is set on the near side when viewed from the direction in which the operation article enters the image with respect to the detection range of the light detection unit, and the extraction operation And a calculation unit that calculates a drawing amount based on a detection result by the second light detection unit when it is determined that the detection has been performed.

  According to such a configuration, when a drawer operation is performed, the drawer amount can be calculated, and an operation according to the drawer amount can be performed.

  According to the present invention, the depth position can be detected with high accuracy and high speed by a simple structure.

1 is an overall schematic perspective view of an image display device according to an embodiment of the present invention. 1 is a schematic side view of an image display device according to an embodiment of the present invention. It is a block diagram which shows the structure of an infrared laser unit. It is a perspective view which shows the structure inside the housing | casing of a photodetector. It is a side view which shows the example of a setting of the detection range of a photodetector. It is a timing chart which shows an example of the light reception level of a vertical synchronizing signal, a horizontal synchronizing signal, and a photodetector. It is a side view showing an example in which the depth is not set parallel to the touch plane. It is a side view which shows the example which set the depth in parallel with the touch plane. It is a flowchart which shows an example of a detection process of a touch coordinate and depth. It is a figure which shows an example of a depth conversion table. It is a figure which shows the mode of the drag operation of an icon. It is a flowchart which shows the transfer process to drag mode. It is a flowchart which shows the process in drag mode. It is a side view which shows the Example which provides two photodetectors. It is a figure which shows the mode of extraction operation | movement. It is a flowchart which shows a drawer operation | movement determination process.

  An embodiment of the present invention will be described below with reference to the drawings. An overall schematic perspective view of an image display apparatus according to an embodiment of the present invention is shown in FIG. 1A, and a schematic side view is shown in FIG. 1B.

  An image display device 10 shown in FIGS. 1A and 1B includes a two-sided corner reflector array substrate (reflective element assembly substrate) 1, a liquid crystal display unit 2, an infrared laser unit 3, and a photodetector 4. .

  The two-sided corner reflector array substrate 1 forms an optical image S of an image displayed by the liquid crystal display unit 2 in the air so that an observer can see it.

  The double-sided corner reflector array substrate 1 has a structure in which square through holes that penetrate the main surface of the substrate in the vertical direction are arranged in a staggered manner in plan view with respect to the substrate. Mirror surfaces are formed as two-surface corner reflectors on two orthogonal inner wall surfaces of the through-holes.

  The light beam emitted from the point light source disposed in the space on the one main surface F2 side (FIG. 1B) of the substrate is reflected twice by the two-surface corner reflector and passes through the substrate while being bent. Each light beam emitted from the point light source and directed to the different two-surface corner reflector is reflected twice by the two-surface corner reflector, and then the main surface opposite to the main surface F2 of the substrate on which the point light source is disposed. An image is formed at one point in the space on the F1 side.

  The liquid crystal display unit 2 that emits surface light of the image can be regarded as a set of point light sources. Therefore, the light beam of the image light surface-emitted by the liquid crystal display unit 2 disposed in the space on the one main surface F2 side of the substrate (the space below the main surface F2 in FIG. 1B) is reflected by the two-surface corner reflector. Due to the reflection, an image is formed as an optical image S at a position symmetrical to the liquid crystal display unit 2 in the space on the other main surface F1 side (the space above the main surface F1 in FIG. 1B). Thereby, the user who is an observer can obtain the feeling that the image is displayed in the air.

  In order to make the image display device 10 function as a virtual input interface, there is provided an infrared laser unit 3 that irradiates the optical image S imaged by the two-sided corner reflector array substrate 1 as described above with infrared laser light. It is done.

  The block configuration of the infrared laser unit 3 is shown in FIG. As shown in FIG. 2, the infrared laser unit 3 includes an infrared LD (laser diode) 3A, a collimator lens 3B, a beam splitter 3C, a photodetector 3D, and a horizontal MEMS (Micro Electro Mechanical Systems) mirror 3E. , A vertical MEMS mirror 3F, a laser control circuit 3G, a mirror servo unit 3H, and an actuator 3I.

  The infrared LD 3A emits infrared laser light with the power controlled by the laser control circuit 3G. The emitted infrared laser light is collimated by the collimator lens 3B and enters the beam splitter 3C. A part of the light entering the beam splitter 3C is reflected and received by the photodetector 3D, and part of the light is transmitted to the horizontal MEMS mirror 3E. Based on the detection signal detected by the photodetector 3D, the laser control circuit 3G controls the output power of the infrared LD 3A.

  The laser light incident on and reflected by the horizontal MEMS mirror 3E that can be deflected so as to scan the laser light in the horizontal direction is reflected by being incident on the vertical MEMS mirror 3F that can be deflected so as to scan the laser light in the vertical direction, The light is emitted outside the casing of the infrared laser unit 3.

  Since the infrared laser unit 3 is disposed in a space on the main surface F2 side (FIG. 1B) on which the liquid crystal display unit 2 of the two-sided corner reflector array substrate 1 is disposed, the infrared laser emitted from the infrared laser unit 3 The light passes through the through-hole of the two-sided corner reflector array substrate 1 and is emitted to the optical image S.

  The laser light emitted from the infrared laser unit 3 is two-dimensionally scanned with respect to the optical image S by the deflection of the horizontal MEMS mirror 3E and the vertical MEMS mirror 3F.

  The mirror servo unit 3H drives the actuator 3I in accordance with the horizontal synchronization signal from the control unit 5 to deflect the horizontal MEMS mirror 3E, and drives the actuator 3I in response to the vertical synchronization signal from the control unit 5 to generate vertical MEMS. The mirror 3F is deflected.

  The photodetector 4 is for detecting laser light emitted from the infrared laser unit 3 and reflected by an operation object such as a finger, and the liquid crystal display unit 2 of the two-sided corner reflector array substrate 1 is disposed. It arrange | positions in the space by the main surface F2 side (FIG. 1B).

  FIG. 3 shows a schematic perspective view of the configuration inside the housing of the photodetector 4. As shown in FIG. 3, inside the housing of the photodetector 4, a light receiving element 4A for detecting the irradiation of the reflected laser light and the reflected laser light incident from the window of the housing are condensed to receive the light receiving element 4A. And a plate-shaped masking member 4C having a height which is disposed between the light receiving element 4A and the light collecting lens 4B and covers a part of the light receiving element 4A.

  As shown in FIG. 2, the light receiving element 4 </ b> A is connected to the control unit 5, and a detection signal from the light receiving element 4 </ b> A is sent to the control unit 5.

  The masking member 4C has a width in the same direction as the width direction of the light receiving element 4A. The masking member 4C has a curved shape in which both ends thereof are closer to the condenser lens 4B side than the central portion, and shields the reflected laser light in accordance with the incident angle to the light receiving element 4A so as to cover the light receiving element 4A. Regulates irradiation.

  The reflected laser light from the end in the width direction of the light receiving element 4A is condensed by the condenser lens 4B and the spot diameter irradiated onto the light receiving element 4A is larger than that at the center. Therefore, the portion to be shielded by the masking member is received by the light receiving element 4A without being completely shielded by the enlargement of the spot diameter, and there is a fear of erroneous detection. Therefore, by making the masking member 4C into a curved shape, the reflected laser beam at the end portion with the large spot diameter can be shielded while the spot diameter is small.

  The detection range of the photodetector 4 can be adjusted by adjusting the dimensions of the masking member 4C. An example of setting the detection range of the photodetector 4 is shown in FIG. The broken line in FIG. 4 represents the limit (end) of the detection range of the photodetector 4. As shown in FIG. 4, one end of the detection range of the photodetector 4 is set to be parallel to the optical image S in the vicinity of the optical image S, and this one end is optically imaged by an operation article (finger in FIG. 4). It becomes the touch plane T for detecting the position where S is touched. Note that the touch plane T may coincide with the optical image S.

  And the detection range of the photodetector 4 is provided in the back | inner side of the touch plane T seeing from the approach direction to the optical image S of the operation thing.

  An example of the vertical synchronizing signal and the horizontal synchronizing signal is shown in FIG. In the example of FIG. 5, the vertical synchronization signal is a stepped signal, and the horizontal synchronization signal is a triangular wave signal that increases / decreases for each step of the vertical synchronization signal. The horizontal MEMS mirror 3E and the vertical MEMS mirror 3F are deflected by such a synchronization signal, so that the laser light emitted from the infrared laser unit 3 is reciprocated in the horizontal scanning direction (FIG. 4) while being reciprocated in the vertical scanning direction (FIG. 4). ) Is scanned on the optical image S.

  In the case of scanning in the vertical scanning direction from top to bottom, in the example of FIG. 4, the laser light is first reflected by the fingertip and detected by the photodetector 4, and the laser light is sequentially reflected by the bottom surface of the finger as the scanning proceeds. Then, the light is detected by the light detector 4, and finally, the laser light is reflected by the bottom surface of the finger in the vicinity of the touch plane T and detected by the light detector 4. At this time, as shown in FIG. 5, a group of received light levels of the photodetector 4 appears along the time series of the synchronization signal. The control unit 5 (FIG. 2) determines the irradiation position coordinates of the laser light on the touch plane T based on the vertical and horizontal synchronization signal values (that is, the position of the MEMS mirror) corresponding to the light reception level detected last in the group. Are detected as touch coordinates indicating the position where the optical image S is touched by the operation article. Touch coordinates are two-dimensional coordinates composed of vertical coordinates and horizontal coordinates.

  When scanning in the vertical scanning direction from bottom to top, the touch coordinates may be detected based on the first detected timing among the group of light receiving levels.

  In addition, the control unit 5 counts the number of detections of the received light level excluding the last in the group, that is, the number of scanning lines that is the number of laser beams reflected by the finger, and the depth based on the counted number of scanning lines. A position is detected (note that all detection times in a group may be used). For example, a table defining the depth layer level corresponding to the number of scanning lines may be prepared in advance, and the depth layer level may be detected based on this table. For example, if the number of scanning lines is 0 to 5, the layer level 0 including the touch plane T, and if the number of scanning lines is 6 to 10, the layer level 1 deeper than the layer level 0 and the number of scanning lines are If it is 11-15, it can prescribe | regulate with the layer level 2 etc. of the back | inner side from the layer level 1.

  As described above, according to the present embodiment, it is possible to detect touch coordinates and detect multiple levels of depth using a single photodetector 4, thereby reducing the number of components and realizing cost reduction.

  Here, when scanning in the vertical direction of the laser beam by the infrared laser unit 3 at a constant speed, as shown in FIG. 6, the lower side of the touch plane T (vertical) where the distance between the infrared laser unit 3 and the touch plane T becomes short. The lower the scanning direction), the denser the scanning lines. Therefore, if the depth is defined by a fixed number of scanning lines, the depth cannot be set at a position parallel to the touch plane T as shown by the one-dot chain line in FIG. That is, although the operation article is entered at the same depth, the depth detection result varies depending on the position of the optical image S in the vertical direction, and thus the user may feel uncomfortable.

  Therefore, depth detection by the process shown in FIG. 8 may be performed. When the control unit 5 detects the touch coordinates and the number of scanning lines in step S1 of FIG. 8 (Y in step S1), the process proceeds to step S2. In step S <b> 2, the control unit 5 acquires the layer level of the depth from the depth conversion table stored in advance in the control unit 5 based on the vertical coordinates of the detected touch coordinates and the counted number of scanning lines.

  An example of the depth conversion table is shown in FIG. As shown in FIG. 9, the correspondence between the number of scanning lines and the layer level is defined for each vertical coordinate. 9, the larger the value of the vertical coordinate, the lower the touch plane T in FIG. It is specified that the number of scanning lines increases as the vertical coordinate value increases even at the same layer level. As a result, the depth can be set parallel to the touch plane T as indicated by the alternate long and short dash line in FIG. Therefore, when the operation article is entered at the same depth, the same depth detection result is obtained regardless of the vertical position, and the user's uncomfortable feeling can be suppressed.

  After step S2, in step S3, the control unit 5 outputs the detected touch coordinates and the acquired depth layer level. Then, the process of FIG. 8 ends.

  In the above description, only the vertical coordinate is referred to, but only the horizontal coordinate or both the vertical coordinate and the horizontal coordinate may be referred to.

  Next, an example of an application using touch coordinate detection and depth detection described above will be described with reference to FIGS. The application described here is a drag mode by pressing an operation article.

  The flowchart shown in FIG. 11 is started. First, in step S11, an operation object such as a finger is touched on the touch plane T (state (A) in FIG. 10), and the control unit 5 detects touch coordinates (in step S11). Y), proceed to step S12.

  When the control unit 5 detects the depth layer level in step S12, determines that the detected layer level is equal to or higher than the predetermined layer level, and detects pushing of the operation article (Y in step S12), step S13 is performed. (The state shown in FIG. 10B). In step S13, the control unit 5 shifts to the drag mode.

  On the other hand, when the control unit 5 determines in step S12 that the detected layer level does not reach the predetermined layer level and detects that the operation article is not pushed in (N in step S12), the process proceeds to step S14. move on. In step S14, the control unit 5 performs processing assuming that a tap operation has been performed. For example, when the finger is stopped while the icon I is touched with the finger as shown in FIG. 10A, an operation as a tap operation is performed.

  When transitioning to the drag mode, the flowchart of FIG. 12 is started. If the control unit 5 detects the touch coordinates in step S15 (Y in step S15), the process proceeds to step S16. In step S16, when it is determined that the layer level detected together with the touch coordinates in step S15 is equal to or higher than the predetermined layer level (Y in step S16), the process proceeds to step S17. In step S17, the control unit 5 performs display control of the liquid crystal display unit 2 so as to move the icon to the position of the touch coordinates detected in step S15. The icon to be moved is an icon including the touch coordinates detected in step S11 (FIG. 11). After step S17, the process returns to step S15.

  As shown in FIG. 10B, when the finger is pushed into the icon I and the mode is changed to the drag mode, the icon I follows and moves when the finger is pushed by repeating the steps S15 to S17. As shown in FIG. 10C.

  In step S15, the control unit 5 cannot detect the touch coordinates (N in step S15), or the touch coordinates can be detected in step S15, but the predetermined layer level is not reached in step S16 (step S16). N), the process proceeds to step S18. In step S18, the control unit 5 controls the display of the liquid crystal display unit 2 so as to fix the icon.

  Thus, when the icon I is moved by dragging as shown in FIG. 10C, when the finger is pulled out from the icon I as shown in FIG. 10D, the icon I is fixed in step S18. (Drop icon).

  As described above, a user interface capable of dragging an icon on the optical image S can be realized by detecting the touch coordinates and detecting the depth according to the present embodiment. Therefore, the user can operate the icon more intuitively.

  Next, as an example of an application, a drawing operation by an operation article will be described. In this embodiment, as shown in FIG. 13, a detection range (see FIG. 13) that is set on the near side as viewed from the approaching direction of the operation article to the optical image S with respect to the detection range of the photodetector 4 (broken line in FIG. 13). 13 is provided separately. Similar to the photodetector 4, the photodetector 15 includes a light receiving element 15 </ b> A, a condenser lens 15 </ b> B, and a masking member (not shown) disposed therebetween. The detection signal from the light receiving element 15A is sent to the control unit 5 (FIG. 2).

  With reference to FIGS. 14 and 15, the drawing operation processing with such a configuration will be described. The flowchart of FIG. 15 is started. First, in step S21, when the control unit 5 detects two touch coordinates (Y in step S21), the process proceeds to step S22. For example, as shown in FIG. 14A, when two fingers touch the optical image S, two touch coordinates are detected.

  If the control unit 5 determines in step S22 that the distance between the two detected touch coordinates is within a predetermined distance (Y in step S22), the process proceeds to step S23. For example, after (A) in FIG. 14, when two fingers are brought close as shown in (B), the distance between the two touch coordinates is within a predetermined distance.

  In step S23, when the control unit 5 continues to detect the touch coordinates and determines that the reflected laser beam from the operation article is no longer detected by the photodetector 4 (Y in step S23), the control unit 5 proceeds to step S24.

  In step S24, the control unit 5 touches each laser beam based on the values of the vertical and horizontal synchronization signals corresponding to the timing at which the laser beams reflected by the tips of the two manipulation objects are detected by the photodetector 15. A position coordinate on the plane T is detected. If the distance between the detected position coordinates is within a predetermined distance (Y in step S24), the process proceeds to step S25. In step S25, the control unit 5 determines that the drawing operation has been performed.

  For example, after (B) in FIG. 14, when two fingers are pulled close to each other as shown in (C), it is determined that the pulling operation has been performed in step S25 after the processing in steps S23 and S24. The

  In step S25, the control unit 5 determines the laser light on the touch plane T based on the values of the vertical and horizontal synchronization signals corresponding to the timing at which the laser beam reflected by the tip of the operation article is detected by the photodetector 15. Detect position coordinates. Then, based on the detected position coordinates and the touch coordinates detected at the end when the detection of the touch coordinates is continued in step S23, the drawing amount of the operation article is calculated.

  The control unit 5 controls the display of the liquid crystal display unit 2 so as to enlarge / reduce the image in the optical image S according to, for example, the amount of extraction.

  As described above, according to the present embodiment, it is possible to realize a user interface that can be operated by an operation that is pulled out by two operating objects, and to give a more sensuous operation feeling to the user.

  The embodiment of the present invention has been described above, but the embodiment can be variously modified within the scope of the gist of the present invention.

  For example, the light beam scanned two-dimensionally is not limited to infrared light, and may be visible light. According to this, when visible light is reflected by an operation object such as a finger, the user can see the color, so that the user can be informed that the operation object is reliably located in the scanning range of the light beam.

DESCRIPTION OF SYMBOLS 1 Two-surface corner reflector array board | substrate 2 Liquid crystal display part 3 Infrared laser unit 3A Infrared laser diode 3B Collimator lens 3C Beam splitter 3D Photodetector 3E Horizontal MEMS mirror 3F Vertical MEMS mirror 3G Laser control circuit 3H Mirror servo part 3I Actuator 4 Light Detector 4A Light receiving element 4B Condensing lens 4C Masking member 5 Control unit 10 Image display device 15 Photo detector 15A Light receiving element 15B Condensing lens S Optical image T Touch plane I Icon

Claims (8)

A scanning unit that emits a light beam toward an image projected in space and scans the light beam two-dimensionally with respect to the image ;
A first light detection unit for detecting the light beam reflected by an operation article within a detection range;
The number of scanning lines, which is the number of the light beams detected by the first light detection unit , is counted, and the first depth position of the operation article in the direction perpendicular to the image is detected based on the number of scanning lines. A first position detector;
Equipped with a,
A detection plane that is one end of the detection range coincides with or is parallel to the image;
The detection range is a spatial input device provided on the back side of the detection plane when viewed from the direction in which the operation article enters the image .   The spatial input device according to claim 1, wherein the detection range is a region having a predetermined distance on the scanning unit side with respect to the projection region from a plane parallel to the projection region of the image. The first position detector on the basis of the table defining the number of scanning lines of the relationship between the position of the vertical scanning direction of the scanning unit, according to claim 1 or claim 2 for detecting the first depth position Spatial input device.   4. The spatial input device according to claim 1, further comprising a second position detection unit configured to detect an irradiation position of the light beam as an instruction position based on a vertical scanning direction of the scanning unit. 5. The spatial input device according to claim 4, further comprising a processing unit that switches processing based on whether the first depth position is deeper than a predetermined second depth position. The spatial input device according to claim 5, wherein when the first depth position is deeper than the predetermined second depth position, the processing unit moves a predetermined image to the detected indication position.   The space according to claim 4, further comprising a determination unit that determines that a predetermined operation has been performed based on the fact that the designated position is no longer detected by the second position detection unit. Input device. A second light detection unit having a predetermined distance in a direction opposite to the scanning unit with respect to the projection region from a plane parallel to the projection region of the image;
The spatial input device according to claim 7, further comprising: a calculation unit that calculates a movement distance based on a detection result by the second light detection unit.

Images