JP4549213B2 - Multi-scene imaging optical element and line-of-sight detection input device - Google Patents

Description

The present invention relates to a multi-scene imaging optical element capable of simultaneously guiding a plurality of scenes (scenes) in different directions to a single imaging surface, and a line-of-sight detection input device configured using the same.

  An optical system for simultaneously guiding the light of two scenes in front to the observer's eyes is described in Patent Document 1 below. What was shown by the literature 1 is comprised like FIG. 5, for example. That is, a lens 51 having lens portions 51a and 51b at two adjacent locations, and a holographic plate (glass plate) 52 for reflecting the light passing through them and changing the path are arranged at positions close to each other. Yes. An input holographic optical element 53 and an output holographic optical element 54 are attached to the holographic plate 52.

Incident light from the scene A enters the holographic plate 52 from the lens portion 51a through the input holographic optical element 53, and repeats reflection to reach the observer's eye 55 from the output holographic optical element 54. On the other hand, incident light from the scene B enters the holographic plate 52 from the lens portion 51b at a substantially right angle and reaches the eyes 55 of the same observer as it is. Thus, both the lights of the scenes A and B reach the observer's eyes 55.
JP-T-2001-514864

  Further, a spectacle-type eye gaze detection apparatus that measures an object captured by grasping the direction of the eye gaze by taking an image of the eye has a configuration shown in FIG. Are listed. The line-of-sight detection device has an illumination unit 62, an imaging unit 63, and an image display unit 66 mounted on a spectacle frame 61 having transparent plates 64 and 65.

In the apparatus of FIG. 6, while the illumination unit 62 illuminates the user's eye e, the imaging unit 63 (and a control unit (not shown) connected thereto) determines the direction of the user's line of sight. The display unit 66 displays an image including several options to the user by a hologram optical element (display) 67. Therefore, this line-of-sight detection device (the control unit thereof) grasps which option the user sees from the direction of the line of sight determined as described above, and uses the signal to control various devices. Use.
JP 2003-230539 A

  The optical system described in Patent Document 1 can guide light from two scenes to one imaging surface at the same time, but it is large and heavy due to its complicated structure such as combining a lens and a holographic plate. It becomes a thing. Therefore, it is expected that it is actually quite difficult to use it in the same manner as glasses by incorporating it in a glasses frame or the like. Further, since the light is allowed to pass through a plurality of optical elements and the light is reflected many times, there is a problem in that the image is distorted and it is necessary to make an appropriate adjustment to the optical system.

  On the other hand, the line-of-sight detection device described in Patent Document 2 detects only the direction of the line of sight (the position of the black eye), and requires another method for measuring an object captured by the line of sight. In the example of Patent Document 2, in order to measure which option is being viewed on the display, the position of the black eye when the position of the sample point on the display is viewed in advance is examined, and measurement is performed based on that information. I do. However, in order to do this, the number of measuring instruments to be used increases, and it is indispensable to perform a considerably complicated calculation or calibration. In addition, when the range that can be measured for the object that the line of sight is captured is limited to the choices on the display, it is impossible to grasp what the line of sight is directed to from any arbitrary scene in real space, Must be quite limited.

The present invention aims to solve such problems. In other words, it provides a simple optical imaging device for multiple scene imaging, and further includes an optical element that can be detected directly and can detect the direction of the line of sight in real space. The gaze detection input device which can be provided is provided.

The multi-scene imaging optical element according to the invention is for guiding a plurality of scenes (scenes) in different directions to one imaging surface at the same time, and images each light of the plurality of scenes on the one imaging surface. It is characterized by comprising a single holographic optical element that can be made to operate.
The “holographic optical element” as used herein refers to an element having an optical function based on a holographic technique. A “single holographic optical element” means an element constituted by an integral element without combining a lens, a mirror, or the like. The “input surface” and “imaging surface” are not limited to planes, and may be arbitrary curved surfaces. An optical element 1 shown in FIGS. 1 and 2 is the above-described holographic optical element, and is an example of an optical element for multiple scene imaging according to the invention .

Such an optical element for imaging multiple scenes can be widely used when two or more scenes are to be simultaneously guided to an observer's eyes or a camera. However, unlike the conventional one, it is composed of a single holographic optical element, so that it can be made compact and light, and the distortion of the image is reduced without optical adjustment. There are significant advantages. Such advantages are not present in the optical system (see FIG. 5) described in Patent Document 1 (Japanese Patent Publication No. 2001-514864).

  The holographic optical element is a planar hologram having a refractive index distribution for imaging the light of each scene on the one imaging surface, and the refractive index distribution for each light of a plurality of scenes. It is good to use what was integrated. By doing so, the above-described multiple-scene imaging optical element can be manufactured and embodied without any particular difficulty.

In the holographic optical element described above, it is preferable that a scene on both sides sandwiching the element is imaged on one imaging surface on one side. “Those that form an image of a scene on both sides of an element on one imaging surface on one side” means a holographic optical element in which a transmission type and a reflection type are integrated.
If such a holographic optical element is used, the optical system of Patent Document 1 in which multiple scene imaging optical elements are used to form images on a single imaging surface in a direction separated by about 180 °. It is possible to demonstrate functions that are not included.

The gaze detection input device according to the invention is:
-Arranging one of the above-described multiple-scene imaging optical elements so that a scene in a person's field of view and an image of the person's eye are simultaneously imaged on one imaging surface,
In addition, an imaging element is provided at the position of the imaging surface so as to simultaneously capture a scene in the human field of view and an image of the human eye. Visual axis detecting input device constructed by the optical element 1 and the imaging device c, etc. in FIGS. 2 and 3, etc. is an example of the visual line detecting input device.

According to such a line-of-sight detection input device, both an eye part (black eyes and the surroundings) and a scene in the field of view are imaged by one image sensor. Therefore, no other optical equipment is required, and the apparatus can be configured simply and compactly. In addition, since both images are not captured by separate image sensors, it is possible to accurately and easily grasp the target captured by the line of sight without special calculations or calibrations using a measuring device. it can.
Whereas the technique of the above-mentioned Patent Document 2 (Japanese Patent Laid-Open No. 2003-230539) picks up only the eye part and determines to which part of the image on the display the line of sight is directed, the apparatus of the invention The real part and the real space are imaged to detect where the line of sight is directed in the real space. That is, the gaze direction that can be detected is not limited to a specific display. In addition, despite the imaging of the eye and the real space, separate imaging elements and the like are not necessary as described above, and calibration and the like are not necessary.

In the above gaze detection input device,
・ A holographic optical element is attached to the lens frame corresponding part of the spectacle frame as a planar object that forms a scene in front of the element and an image of the eye part behind the element on one imaging surface,
-It is preferable to attach the above image sensor to the same eyeglass frame ------.

  By doing so, the user can use this line-of-sight detection input device as a pair of glasses and use it as an apparatus for inputting a signal by line-of-sight detection instead of a conventional personal computer mouse or the like. Since the holographic optical element and the imaging element are mounted on the eyeglass frame, it becomes a new hands-free input device. In addition, since a single holographic optical element forms an image on a common imaging element with a front scene and a rear eye part, it can be configured to be compact and lightweight.

In the above gaze detection input device,
・ In addition to attaching the above holographic optical element, the above image pickup element, and transmission means for transmitting gaze detection information emitted by the image pickup element to the outside,
-It is preferable to attach a receiving means for receiving information transmitted from the outside and an information presenting means for visually or audibly expressing the information received by the receiving means.
The line-of-sight detection input device 20 shown in FIG. 4 is an example of the line-of-sight detection input device referred to here, to which the transmission unit 2 and the reception unit 8 are attached.

  Since such a line-of-sight detection input device includes a holographic optical element, an image pickup element, and a transmission unit, it is possible to detect and transmit an object to which the user is pointing the line of sight. Moreover, since it has a receiving means and an information presentation means, the information from the outside can be provided visually or audibly to the user. That is, the user can perform bidirectional information transmission using the line-of-sight detection input device.

It is further preferable that the transmission means and the reception means are connected to a database or a terminal device of an information provider or work instructor.
If it does so, the information from an image pick-up element can be transmitted to a database and an expert (information provider and work directions person), and the information from them can be displayed on an information presentation means, and can be provided to a user. In other words, it is possible to transmit information to and from a remote database or expert via a network or the like using this line-of-sight detection input device as a wearable input / output terminal.

In the line-of-sight detection input device,
・ Attach the above holographic optical element to the lens part of the eyeglass frame as a plane that forms the scene in front of the element and the image of the eye behind the element on a single imaging surface. While mounting the above image sensor on the same glasses frame,
-It is recommended to attach information presentation means that visually represents the information received by the reception means to the lens part equivalent part of the same eyeglass frame ------.
The line-of-sight detection input device 20 in FIG. 4 also illustrates such a configuration.
By doing so, the line-of-sight detection input device can be configured as a device that can be worn on the head like ordinary glasses, and it becomes a hands-free and easy-to-use bidirectional information transmission means.

According to the optical element for multi-scene imaging by the invention, while a configuration is simple lightweight and compact, it can be guided at the same time one of the imaging surface a plurality of scenes (scene) in different directions, yet the strain image is Get smaller.

According to the line- of- sight detection input device according to the invention , both the eye part and the scene in the field of view are imaged by one image sensor. Therefore, it is possible to accurately and easily grasp the object captured by the line of sight without performing calibration or performing special calculations. There is also an advantage that it is possible to detect not only a limited scene in a specific display but also a scene in a real space. In addition to the conventional mouse for personal computers, this device can be used as a device for inputting signals by eye gaze detection, and can also be configured as a wearable input / output terminal capable of bidirectional information transmission. .

  1A to 1C show a multi-scene imaging optical element 1 as an embodiment of the invention and a manufacturing procedure thereof.

First, FIG. 1A is a diagram showing a manufacturing procedure of a transmission type holographic optical element (hologram) 1a. And light from the light source O ₁ at the position P1 of the input surface 1, incident from the same side as the light from the light source O ₁ of the input surface 1 with respect to the optical element 1a, and focused on the position P2 of the observation plane light Are recorded on the optical element 1a. Then, the scene at the position P1 on the input surface 1 is imaged on the position P2 on the observation surface by the interference / diffraction action of light by the interference fringes in the optical element 1a.

FIG. 1B shows a procedure for producing the reflection type holographic optical element (hologram) 1b. The light from the light source O _{3 at} the position P3 on the input surface 2 and the light source on the input surface with respect to the optical element 1b. Interference fringes with light incident from the opposite side and imaged at the position P4 are recorded on the optical element 1b placed in the same direction away from the positions P3 and P4. When such an optical element 1b is used, the scene at the position P3 on the input surface 2 is imaged on the position P4 on the observation surface on the same side as viewed from the optical element 1b.

  The multi-scene imaging optical element 1 shown in FIG. 1C is manufactured by integrating the transmissive and reflective holographic optical elements 1a and 1b formed by FIGS. 1A and 1B. The optical elements 1a and 1b can be integrated, for example, by performing both exposures shown in FIGS. 1A and 1B on one photoconductor for recording interference fringes and developing the photoconductor. Although it is possible, a refractive index distribution based on interference fringe information by computer simulation may be produced by electron beam drawing or the like. In any case, the multiple scene imaging optical element 1 is configured as a single holographic optical element called a hologram or the like.

  According to the produced optical element 1, a scene such as the object m at the position of the input surface 1 is imaged on the observation surface, and a scene such as the eye part e at the position of the input surface 2 is also on the same observation surface. Form an image. In other words, the multiple-scene imaging optical element 1 has a lens function for forming an image of light such as the object m on the observation surface, a mirror function for reflecting light of the eye portion e and the like to form an image on the observation surface, and With lens function. Therefore, if the imaging device c is arranged with the position of the imaging surface s aligned with the observation surface, two scenes (the object m and the eye portion e) are formed as one image on the imaging device c. Become. Note that if the number of accumulations described above is increased, it is possible to form three or more scenes as the same image. Note that, in the above-described manufacturing method, the input surface and the observation surface are not limited to flat surfaces, and may be arbitrary curved surfaces.

  FIG. 2 is a diagram conceptually illustrating a method for performing human eye-gaze detection input using an optical element for multiple scene imaging. The multi-scene imaging optical element 1 shown in FIG. 1 (c) produced as described above is placed in front of the user's eye e as shown in FIG. 2 (a), and the CCD is placed next to the user's head. An image sensor c such as a camera is disposed. Then, as shown in FIG. 2B, the image sensor c captures an image m ′ of the object m in front of the user and an image e ′ of the eye part e of the user as one image. (The method of overlapping images can be set arbitrarily). In the image, a scene that appears in front is reflected, and the position of the user's black eyes is displayed. Therefore, the user can easily know from what the user is looking at and what the user is looking at.

  According to such a principle, the apparatus 10 shown in FIG. 3 can be configured as a wearable eyeglass-type line-of-sight detection input apparatus for a computer. In the illustrated line-of-sight detection input device 10, the multiple-scene imaging optical element 1 is attached to the left and right lens portions corresponding to the eyeglass frame 11, and the imaging element c is attached to the left and right portions of the same frame 11 close to the ear hook. .

Such a line-of-sight detection input device 10 can be used as a computer input device in place of a mouse or the like after the user wears it on the head like glasses. In other words, the direction of the line-of-sight detection input device 10 is determined so that the scene to be viewed is in the field of view, and the line of sight is directed to a specific scene (such as the object m) in the field of view. From the image m ′ such as the object m captured in one image and the image e ′ of the user's eye part e, it is possible to detect the user's intention by the line of sight. If it is determined that the intention is fixed, such as staring for a certain period of time or blinking a specific number of times, an input function similar to that of a mouse can be realized completely hands-free. Needless to say, even if the user moves, the gaze detection is not hindered.
If the multiple-scene imaging optical element 1 is attached to the left and right lens portions, the human visual field can be obtained from the image of the right eye and the image of the right eye, and the image of the left eye and the image of the left eye. Three-dimensional information can be acquired.

  In FIG. 4, a glasses-type in which a display 6 (that is, an information presentation means that visually conveys information to the user) is mounted on the glasses frame 21 together with the above-described multiple-scene imaging optical element 1 and imaging element c. The line-of-sight detection input device 20 etc. are shown. The optical element 1 is attached to the right (or left) portion of the lens portion, and the display 6 is attached to the opposite portion of the lens portion. The image sensor c was attached in the vicinity of the ear hooking portion toward the rear surface of the optical element 1. The image sensor c is connected to a transmitting means 2 for transmitting the image information to the outside, and the display 6 is connected to a receiving means 8 for taking image information from the outside into the display 6.

  According to the line-of-sight detection input device 20, not only can the input information based on the user's line of sight be transmitted to the outside by the action of the optical device 1 for multiple scene imaging, the image sensor c, and the transmission means 2, but also the reception means 8 And the display 6 make it possible to visually provide information from the outside to the user. That is, by using this line-of-sight detection input device 20, the user can perform bidirectional information transmission with the outside. Instead of the display 6 or together with the display 6, an auditory information presentation means such as an earphone 7 can be provided. In such a case, similar bidirectional information transmission is possible.

  The line-of-sight detection input device 20 may be connected to a net (information transmission network) 30 as shown in FIG. 4 and may form an information transmission / reception system with the database 41 or the like similarly connected to the net 30. The user of the line-of-sight detection input device 20 accesses specific information in the database 41 by a signal (input information by line of sight) sent from the transmission unit 2 and receives presentation of the information in the database 41 through the reception unit 8. . In that case, the line-of-sight detection input device 20 detects what the user sees among the information displayed on the display 6 by recognizing the movement of the user's black eye by the image sensor c and transmits it to the database 41. If so, it is particularly easy to select and collect information.

In the transmission / reception system described above, there is a merit when the line-of-sight detection input device 20 is connected to the terminal device 42 used by an expert (a person skilled in a specific work and capable of providing work instructions and information). . For example,
1) The scene that the user of the line-of-sight detection input device 20 is viewing and the object that the user's line of sight is capturing are transmitted to the terminal device 42 of the expert at a remote location and displayed on the screen.
2) The expert looks at the screen of the terminal device 42, knows the user's work status, etc., teaches the work target part and the part to be noticed, presents necessary data and manuals, etc. Give work instructions to the user via information presentation means such as 7.
It is also possible to use it like ------.
In the same way, the transmission / reception system is configured as a system for providing information to doctors during surgery, a guided navigation system such as instructing a route that matches the encounter situation, and a dictionary system that translates the foreign language of the part being read It is also possible to use it preferably. Depending on the application, a line-of-sight detection input device (for example, the device 10 in FIG. 3) that does not have an information presentation unit may be used instead of the device 20 in FIG.

FIGS. 1A to 1C are diagrams showing a multiple scene imaging optical element 1 as an embodiment of the invention and a manufacturing procedure thereof. FIGS. 2A and 2B are diagrams conceptually showing a method for performing human eye-gaze detection input using an optical element for multiple scene imaging. 1 is a perspective view showing a glasses-type line-of-sight detection input device 10 configured using a multi-scene imaging optical element 1. FIG. It is a conceptual diagram which shows the spectacles type eyes | visual_axis detection input device 20 and the transmission / reception system containing it. It is principal part sectional drawing which shows the conventional optical system for guide | inducing the light of two scenes ahead ahead to an observer's eyes simultaneously. It is a perspective view which shows the conventional spectacles type gaze detection apparatus.

Explanation of symbols

DESCRIPTION OF SYMBOLS 1 Optical element for multiple scene imaging c Imaging element m Object e Eye part 2 Transmission means 6 Display (information presentation means)
7 Earphone (information presentation means)
8 Receiving means 10.20 Line-of-sight detection input device 30 Net 41 Database

Claims (5)

A multi-scene imaging optical element composed of a single holographic optical element and configured to form an image of a scene on both sides of the element on one imaging surface on one side is in the human field of view Attach the spectacle and the person's eye image at the same time to the lens part of the spectacle frame so as to form an image on one imaging surface at the same time,
An imaging device is provided at the position of the imaging surface so as to simultaneously capture a scene in the human field of view and an image of the human eye, and is attached to the same glasses frame.
A line-of-sight detection input device. In addition to the above-mentioned holographic optical element and the above-described imaging element, and transmission means for transmitting the line-of-sight detection information emitted by the imaging element to the outside,
The line-of-sight detection input device according to claim 1 , further comprising: a receiving unit that receives information transmitted from outside; and an information presentation unit that visually or audibly represents the information received by the receiving unit. The line-of-sight detection input device according to claim 2 , wherein the transmission unit and the reception unit are connected to a database or a terminal device of an information provider or a work instructor. The information presenting means for visually representing the information received by the receiving means is attached to the same eyeglass frame at a position corresponding to the lens unit different from the side where the optical element for multiple scene imaging is attached. The line-of-sight detection input device according to claim 2 or 3 . A transmission / reception system including the line-of-sight detection input device according to claim 3 or 4,
The imaging device transmits the line-of-sight detection information to the terminal device via the transmission unit, whereas the terminal device receives the line-of-sight detection information and receives the necessary instruction or information accordingly. Provided to the information presenting means through the means
A transmission / reception system including a line-of-sight detection input device.

Images