JP2023094539A - Inclination detector, visual line detector, retina projection display device, head attachment type display device, eye examination device, virtual reality display device, user state estimation device, drive support system, inclination detection method of solid object, and visual line detection method - Google Patents

Abstract

To provide an inclination detector with a simple configuration.SOLUTION: An inclination detector in one embodiment comprises: a plurality of light emission parts; light reception means which is caused to emit light by the plurality of light emission parts, receiving light reflected by objects and outputting an electric signal based on the light intensity of the received light; and output means which outputs, on the basis of the electric signal output from the light reception means, a detection result of inclinations of the objects. The light reception means is caused to emit light by the plurality of light emission parts, in a state in which inclinations of objects are almost equal, and receives a plurality of pieces of light reflected by the objects.SELECTED DRAWING: Figure 1

Description

The present invention provides a tilt detection device, a line-of-sight detection device, a retinal projection display device, a head-mounted display device, an eye examination device, a virtual reality display device, a user state estimation device, a driving support system, a three-dimensional object tilt detection method, and a line-of-sight detection. Regarding the method.

2. Description of the Related Art Tilt detection devices are known that detect the tilt of an object based on reflected light of light irradiated to the object. Such a tilt detection device includes a line-of-sight detection device that detects the line-of-sight direction from the tilt of a person's eyeball as an object, a retinal projection display device having the line-of-sight detection device, a head-mounted display device, an optometric device, and the like. Used in various applications.

In order to detect the tilt of the eyeball with high accuracy, the above-mentioned tilt detection device scans the laser light irradiated to the eyeball with a MEMS (Micro Electro Mechanical Systems) mirror, and detects the reflected light of the scanned light from the eyeball. A configuration is disclosed.

A tilt detection device is required to have a simple configuration.

SUMMARY OF THE INVENTION An object of the present invention is to provide a tilt detection device having a simple configuration.

A tilt detection device according to an aspect of the present invention includes a plurality of light emitting units, light emitted by the plurality of light emitting units, receiving light reflected by an object, and outputting an electric signal based on the light intensity of the received light. and an output means for outputting a detection result of the tilt of the object based on the electrical signal output from the light receiving means, wherein the light receiving means detects that the tilt of the object is approximately In the same state, a plurality of lights emitted from the plurality of light emitting units and reflected by the object are received.

According to the present invention, it is possible to provide a tilt detection device with a simple configuration.

It is a figure which shows the whole structural example of the sight line detection apparatus which concerns on embodiment. 1 is a diagram showing a configuration example of a VCSEL array according to a first embodiment; FIG. It is a figure which shows an example of laser-beam incidence to the photodiode which concerns on embodiment. It is a figure which shows the laser beam incidence to the photodiode based on a comparative example. 4 is a block diagram of a hardware configuration example of a processing unit according to the embodiment; FIG. 3 is a block diagram of a functional configuration example of a processing unit according to the first embodiment; FIG. 4 is a flowchart of an example of processing by a processing unit according to the first embodiment; FIG. 3 is a first diagram of the position of reflected light by the eye of light from the VCSEL array of FIG. 2; FIG. 3 is a second diagram of the reflected light position by the eye of light from the VCSEL array of FIG. 2; FIG. 3 is a third diagram of the position of reflected light by the eye of light from the VCSEL array of FIG. 2; FIG. 4 is a 4th view of the reflected light position by the eye of light from the VCSEL array of FIG. 2; FIG. 5 is a fifth diagram of the reflected light position by the eye of light from the VCSEL array of FIG. 2; FIG. 6 is a diagram 6 of the reflected light position by the eye of light from the VCSEL array of FIG. 2; FIG. 7 is a diagram of the reflected light position by the eye of light from the VCSEL array of FIG. 2; FIG. 1 is a first diagram showing an example of a simulation result of line-of-sight direction detection according to the first embodiment; FIG. 2 is a second diagram showing an example of a simulation result of line-of-sight direction detection according to the first embodiment; FIG. 10 is a diagram showing the arrangement of VCSEL arrays according to a comparative example; FIG. 18 is an eye-reflected light position diagram of light from the VCSEL array of FIG. 17; FIG. 1 is a first diagram showing a simulation result of line-of-sight direction detection according to a comparative example; FIG. 2 is a second diagram showing a simulation result of line-of-sight direction detection according to a comparative example; FIG. 11 is a diagram showing an arrangement example of a VCSEL array according to a modification; FIG. 22 is a first diagram of the reflected light position by the eye of light from the VCSEL array of FIG. 21; FIG. 22 is a second view of the reflected light position by the eye of light from the VCSEL array of FIG. 21; FIG. 22 is a third diagram of the position of reflected light by the eye of light from the VCSEL array of FIG. 21; FIG. 22 is a fourth diagram of the reflected light position by the eye of light from the VCSEL array of FIG. 21; FIG. 22 is a fifth diagram of the reflected light position by the eye of light from the VCSEL array of FIG. 21; FIG. 22 is a sixth diagram of the reflected light position by the eye of light from the VCSEL array of FIG. 21; FIG. 22 is a seventh diagram of the reflected light position by the eye of light from the VCSEL array of FIG. 21; FIG. 22 is an eighth diagram of the reflected light position by the eye of light from the VCSEL array of FIG. 21; FIG. 1 is a first diagram showing an example of a detection result by a line-of-sight detection device according to a modification; FIG. 2 is a second diagram showing an example of a detection result by the line-of-sight detection device according to the modification; It is a figure which shows the structural example which does not have a concave-surface mirror. It is a figure which shows the structural example which does not have a concave mirror and a lens. FIG. 10 is a block diagram of an example functional configuration of a processing unit according to the second embodiment; 9 is a flowchart of an example of processing by a processing unit according to the second embodiment; FIG. 11 is a block diagram of an example of the functional configuration of a processing unit according to the third embodiment; 10 is a flowchart of an example of processing by a processing unit according to the third embodiment; FIG. 11 is a diagram showing a configuration example of a retinal projection display device according to a fourth embodiment;

BEST MODE FOR CARRYING OUT THE INVENTION Hereinafter, embodiments for carrying out the invention will be described with reference to the drawings. In each drawing, the same constituent parts are denoted by the same reference numerals, and overlapping descriptions are omitted as appropriate.

The embodiments shown below are examples of tilt detection devices for embodying the technical idea of the present invention, and the present invention is not limited to the embodiments shown below. The shapes of components, their relative positions, parameter values, etc. described below are intended to be illustrative rather than limiting the scope of the present invention, unless specifically stated otherwise. is. Also, the sizes and positional relationships of members shown in the drawings may be exaggerated for clarity of explanation.

In the diagrams shown below, directions may be indicated by the X-axis, Y-axis, and Z-axis. The Y direction along the Y axis indicates a predetermined direction in the light emitting surface, and the Z direction along the Z axis indicates a direction orthogonal to the predetermined direction in the light emitting surface. .

The direction in which the arrow points in the X direction is indicated as +X direction, the direction opposite to +X direction is indicated as -X direction, the direction in which the arrow points in Y direction is indicated as +Y direction, and the direction opposite to +Y direction is indicated as -Y direction. , and the direction in which the arrow points in the Z direction is denoted as +Z direction, and the direction opposite to +Z direction is denoted as -Z direction. In the embodiment, as an example, it is assumed that the plurality of light emitting units emit light in the -X direction. However, these do not limit the orientation of the tilt detection device during use, and the orientation of the tilt detection device during use is arbitrary.

A tilt detection device according to an embodiment includes a plurality of light emitting units, and light receiving means for receiving light emitted by the plurality of light emitting units and reflected by an object, and for outputting an electric signal based on the light intensity of the received light. and output means for outputting the detection result of the inclination of the object based on the electric signal output from the light receiving means. The light receiving means receives a plurality of lights emitted from the plurality of light emitting parts and reflected by the object in a state where the inclination of the object is substantially the same.

The tilt detection device according to the embodiment detects the tilt of an object based on the light emitted by a plurality of light emitting parts, so it includes a movable part such as a MEMS (Micro Electro Mechanical Systems) mirror and drives the movable part. The inclination of the object can be detected without using a driving means, and the configuration of the inclination detection device is simplified. Further, since the tilt detection device receives a plurality of lights emitted from a plurality of light emitting parts and reflected by an object, a light receiving means such as a photodiode is used to output an electric signal based on the light intensity of the received light. can track the movement of an object. Such a light receiving means has a simpler configuration than other light receiving means such as a PSD (Position Sensitive Detector) that outputs an electric signal based on the position of the received light, and the driving means for driving the light receiving means is simple. The configuration is also simple. As described above, the present embodiment provides a tilt detection device with a simple configuration.

The target object is, for example, a human eyeball. The eyeballs tilt in the direction the human gaze is directed. The tilt detection device is, for example, a line-of-sight detection device that outputs the detection result of the tilt of the eyeball as human line-of-sight direction information. Note that the eyeball tilt information includes not only information directly indicating the tilt angle, but also information related to the tilt angle in addition to the tilt angle of the eyeball.

The line-of-sight direction information output from the line-of-sight detection device is used in, for example, an eye tracking device or an optometry device. Alternatively, the line-of-sight direction information can be used to project an image onto the retina or the like in a retinal projection display device or a head-mounted display device such as an HMD (Head Mounted Display). It is used, for example, to correct the position or the content of the image.

Embodiments will be described below using the line-of-sight detection device as an example of the tilt detection device. An eyeball is an example of an object and an example of a three-dimensional object. In the drawings of the embodiments, the eyeball of the right eye of a human being is exemplified, but the functions and effects of the line-of-sight detection device are the same even for the eyeball of the left eye. Further, even when two sight line detection devices are applied to the eyeballs of both eyes, the functions and effects of the sight line detection devices are the same.

[Embodiment]
<Configuration example of line-of-sight detection device 10>
(overall structure)
FIG. 1 is a diagram illustrating an example of the overall configuration of a line-of-sight detection device 10 according to an embodiment. As shown in FIG. 1, the line-of-sight detection device 10 has a VCSEL array 1, a lens 2 and a concave mirror 3 as incident position control means, a photodiode 4 as detection means, and a processing section 100. FIG.

The line-of-sight detection device 10 outputs the drive signal Dr from the processing unit 100 to the VCSEL array 1 to cause the VCSEL array 1 to emit laser light L1. A laser beam L1 emitted from the VCSEL 1 is once condensed by the lens 2 and then enters the concave mirror 3 as a laser beam L2 that spreads as it approaches the concave mirror 3 .

The laser light L2 incident on the concave mirror 3 is reflected by the concave mirror 3 toward the eyeball 30, and then enters the cornea 31 of the eyeball 30 as the laser light L3. The laser light L3 incident on the cornea 31 is reflected by the cornea 31 and enters the photodiode 4 as laser light L4. The photodiode 4 receives the laser beam L4 reflected by the cornea 31 of the eyeball 30 and outputs an electrical signal Se based on the light intensity of the received laser beam L4 to the processing unit 100 .

The processing unit 100 detects the inclination of the eyeball 30 based on the electrical signal Se output from the photodiode 4, and outputs line-of-sight direction information Ed as the detection result to an external device. Note that the external device is an eye tracking device, optometry device, retinal projection display device, HMD, or the like that uses the line-of-sight direction information Ed. Alternatively, the external device may be a storage device that stores the line-of-sight direction information Ed, a display device that displays the line-of-sight direction information Ed, a transmission device that transmits the line-of-sight direction information Ed, or the like.

The laser beams L1, L2, L3, and L4 differ in the position of the optical path along which the laser beam propagates and the spread angle of the propagated laser beam. the nature is the same. Therefore, they are collectively referred to as laser light L when they are not distinguished from each other.

The VCSEL array 1 is an example of multiple light emitting units. For example, the VCSEL array 1 includes a plurality of VCSELs (vertical cavity surface emitting lasers; Vertical Cavity Surface Emitting LASER) element.

A plurality of VCSEL elements included in the VCSEL array 1 each correspond to a light emitting section. For example, the VCSEL array 1 can emit laser light L1 having directivity and a finite spread angle in the -X direction from a plurality of VCSEL elements.

The laser light L1 corresponds to light emitted by the light emitting section. The line-of-sight detection device 10 individually and independently drives the plurality of VCSEL elements included in the VCSEL array 1 according to the drive signal Dr from the processing unit 100, and selectively emits laser light from the plurality of VCSEL elements included in the VCSEL array 1. Illuminate L1.

For example, the line-of-sight detection device 10 can arbitrarily emit laser light L1 from a plurality of VCSEL elements included in the VCSEL array 1 . Especially in this embodiment, the line-of-sight detection device 10 emits a plurality of laser beams L1 from a plurality of VCSEL elements included in a plurality of VCSEL elements. Further, the line-of-sight detection device 10 selectively sequentially switches each of the plurality of laser beams L1.

The wavelength of the laser light L is preferably near-infrared light, which is non-visible light, so as not to impede the human's vision when the line of sight is detected. For example, the wavelength of near-infrared light is 700 [nm] or more and 2500 [nm] or less. However, the wavelength of the laser light L is not limited to invisible light, and may be visible light. For example, the wavelength of visible light is 360 [nm] or more and 830 [nm] or less. The VCSEL array 1 will be separately described in detail with reference to FIG.

The lens 2 and the concave mirror 3 function as incident position control means for controlling the incident position of the laser light L4, which is the laser light L1 emitted from the plurality of VCSEL elements and reflected by the eyeball 30, onto the photodiode 4. The lens 2 and the concave mirror 3 control the position at which the laser light L4 enters the photodiode 4 by changing the divergence angle of the laser light L1.

The lens 2 is a condensing element with positive refractive power. Lens 2 is provided such that the focal point of lens 2 is located between lens 2 and concave mirror 3 . The laser light L2 transmitted through the lens 2 is once condensed at the focal position of the lens 2, and then enters the concave mirror 3 while spreading as it approaches the concave mirror 3. FIG. The lens 2 can cause the laser light L2 to enter a wide area within the concave mirror 3, and the laser light L3 reflected by the concave mirror 3 to enter a wide area of the eyeball 30. The detection range can be expanded. The lens 2 refracts the laser light L1 from the plurality of VCSEL elements to control the incident position of the laser light L2 on the concave mirror 3, thereby expanding the detection range of the line-of-sight direction by the line-of-sight detection device 10. FIG.

However, the incident position of the laser beam L on the concave mirror 3 can also be controlled by adjusting the positional relationship between the VCSEL elements in the VCSEL array 1 . The adjustment of the positional relationship between the VCSEL elements can control the incident position of the laser light L on the concave mirror 3 without using the lens 2, so that the sight line detection device 10 can be downsized and the number of parts can be reduced. 10 is preferable in terms of simplifying the configuration. On the other hand, the use of the lens 2 makes it possible to control the incident position of the laser beam L on the concave mirror 3 by adjusting the focal length of the lens 2 and the position where the lens 2 is arranged, thereby simplifying the production of the line-of-sight detection device 10. It is preferable in terms of

The concave mirror 3 has a function as converging reflecting means that reflects the incident laser beam L2 and converges the reflected laser beam L3 as it approaches the eyeball 30 . The concave mirror 3 has a center of curvature on the side on which the laser beam L2 is incident.

The laser beam L3 is incident near the cornea 31 of the eyeball 30 . The center of curvature of the concave mirror 3 is located off the optical axis of the laser light L between the VCSEL array 1 and the concave mirror 3 . With this arrangement, the VCSEL 1 and the concave mirror 3 form a so-called off-axis optical system. The optical axis of the laser light L is, in other words, the central axis of the laser beam as the laser light L. As shown in FIG.

In this embodiment, the concave mirror 3 is exemplified. A diffractive optical element or the like may be provided instead of the concave mirror 3 . The concave mirror 3 is preferable in that the number of parts can be reduced and the line-of-sight detection device 10 can be miniaturized. Since the concave mirror 3 has a high reflectance, it is preferable in that the loss of light quantity can be reduced. Since the wavefront control element and the diffractive optical element are easy to align, they are preferable in terms of simplifying the manufacturing of the line-of-sight detection device 10 .

If the concave surface of the concave mirror 3 is an anamorphic aspherical surface with different curvatures in the two orthogonal directions that intersect the optical axis of the laser beam L3, the diameter of the laser beam L4 on the light receiving surface of the photodiode 4 is reduced and the laser beam The beam of L4 is shaped isotropically. For example, if the radius of curvature of the concave mirror 3 in the Y direction in FIG. 1 is smaller than the radius of curvature of the concave mirror 3 in the Z direction, an isotropic beam is shaped. Note that the laser beam L4 is an example of light reflected by the object.

The angle of incidence of the laser beam L3 on the eyeball 30 is adjusted so that the laser beam L3 is incident on the cornea 31 at a predetermined angle when the eyeball 30 is in normal vision. The angle of incidence of the laser beam L3 on the eyeball 30 is the angle between the optical axis of the laser beam L3 and the tangential plane of the eyeball 30 including the point where the optical axis of the laser beam L3 intersects the surface of the eyeball 30 .

It is preferable that the predetermined angle of incidence of the laser light L3 on the eyeball 30 is large so that the concave mirror 3 is not included in the human visual field whose line of sight is detected. The viewing angle of a person on the side where the nose of the person whose line of sight is detected is generally about 60 degrees, with the standard viewing direction being 0 degrees. Therefore, it is preferable that the angle of incidence of the laser light L3 on the eyeball 30 in the line-of-sight detection device 10 is approximately 60 degrees or more with respect to the normal vision direction.

At least some laser beams L among the plurality of laser beams L emitted from each of the plurality of VCSEL elements in the VCSEL array 1 are incident at different positions on the cornea 31 of the eyeball 30 at different angles with respect to the other laser beams L. Incident at

The surface of the cornea 31 of the eyeball 30 is a transparent body containing water, and generally has a reflectance of about 2[%] or more and 4[%] or less. The laser light L3 incident on the cornea 31 of the eyeball 30 is reflected by the surface of the cornea 31 of the eyeball 30 and propagates toward the photodiode 4 as laser light L4.

The photodiode 4 is an example of light receiving means for receiving the laser light L4 emitted by the VCSEL array 1 and reflected by the eyeball 30 and for outputting an electric signal Se based on the light intensity of the laser light L. FIG. Photodiode 4 is preferably a silicon photodiode, which has high sensitivity to visible light. A PN type, a PIN type, or an APD (Avalanche photodiode) can be applied to the photodiode 4 . From the viewpoint of simplifying the configuration of the line-of-sight detection device 10, the PN type is particularly preferable. The electrical signal Se is, for example, a current signal, but may be a voltage signal. The photodiode 4 has, for example, only one anode electrode and one cathode electrode, and is capable of outputting an electrical signal based on light intensity, and has two anode electrodes and one cathode electrode. It does not include a configuration in which an electrical signal based on the light position can be output by a configuration having two or more anode electrodes and two or more cathode electrodes. Image sensors such as PSD, CMOS (Complementary Metal Oxide Semiconductor), CCD (Charge Coupled Device), and the like are examples of configurations capable of outputting electrical signals based on such light positions. From the viewpoint of simplifying the configuration of the line-of-sight detection device 10, a photodiode that consumes less power than an imaging element is particularly desirable.

Particularly in this embodiment, the photodiode 4 receives a plurality of laser beams L4 emitted from the plurality of VCSEL elements included in the VCSEL array 1 and reflected by the eyeball 30 in a state where the inclination of the eyeball 30 is substantially the same. do.

A state in which the tilts of the eyeballs 30 are substantially the same is, for example, a state in which the tilts of the eyeballs 30 are ±3.0 degrees or less. For example, a state in which the tilt of the eyeball 30 is almost the same is a state of a fixational eye movement in which the eyeball 30 has a minute amplitude.

The processing unit 100 is an example of output means for outputting the detection result of the tilt of the eyeball 30 based on the electrical signal Se output from the photodiode 4 . The configuration of the processing unit 100 will be separately described in detail with reference to FIGS. 10 and 11. FIG.

(Configuration example of VCSEL array 1)
FIG. 2 is a diagram showing an example of the configuration of the VCSEL array 1. As shown in FIG. FIG. 2 shows the VCSEL array 1 viewed from the -X direction side, which is the direction in which the VCSEL array 1 emits laser light L1.

VCSEL array 1 includes VCSEL elements 11-16. VCSEL element 11 has a light emitting surface 11A, VCSEL element 12 has a light emitting surface 12A, and VCSEL element 13 has a light emitting surface 13A. VCSEL element 14 has a light emitting surface 14A, VCSEL element 15 has a light emitting surface 15A, and VCSEL element 16 has a light emitting surface 16A. Each of the light emitting surfaces 11A to 16A is a surface from which the laser light L1 is emitted. The light emitting surface 1A of the VCSEL array 1 is a surface including all of the light emitting surfaces 11A to 16A. Note that the laser light L1 is a generic term for laser light emitted from each of the VCSEL elements 11 to 16. FIG.

The VCSEL elements 11 to 16 are arranged side by side along two different directions along the light emitting surface 1A. The two different directions along the light emitting surface 1A are the Y direction and the Z direction in FIG. The VCSEL elements 11 to 16 are two-dimensionally arranged along each of the Y and Z directions. Both the Y direction and the Z direction are examples of predetermined directions along the light emitting surface included in the light emitting section.

Especially in this embodiment, the distances D between adjacent VCSEL elements in the VCSEL elements 11 to 16 along the Y direction and the Z direction are all equal. However, this "equal" allows a difference that is generally recognized as an error, for example, a difference of 1/10 or less of the interval D. Even when this difference is included, the effects of the line-of-sight detection device 10 can be obtained.

The array of the VCSEL elements 11 to 16 may be a one-dimensional array, but a two-dimensional array is preferable in terms of expanding the detection range of the line-of-sight direction by the line-of-sight detection device 10 .

The plurality of light emitting units are not limited to the VCSEL array 1, and may include, for example, a plurality of LDs (semiconductor lasers; Laser Diodes) or LEDs (Light Emitting Diodes). .

In this embodiment, the VCSEL array 1 including the VCSEL elements 11 to 16 is exemplified as the plurality of light emitting units. It may be a light source. Also, the plurality of light emitting units may be configured by combining a plurality of types of light sources as light emitting units. For example, the plurality of light emitting units can be configured by combining an LD that continuously emits the laser light L1 and a pulsed laser light source that emits the laser light L1 in pulses. It can be configured by

The VCSEL array 1 is preferable from the viewpoint of downsizing the line-of-sight detection device 10 because it facilitates in-plane integration of a plurality of light emitting units. In addition, since the VCSEL array 1 can stabilize the light amount of the laser light L even when the light amount of the laser light L is small, the light amount of the laser light L incident on the eyeball 30 is suppressed, and the line-of-sight direction can be accurately determined. It is preferable in that it can be detected in

Each of the laser beams L1 emitted by the VCSEL elements 11 to 16 is diffracted at the respective apertures of the VCSEL elements 11 to 16 in the VCSEL array 1 and spreads as it approaches the lens 2 . The spread angle of the laser light L1 can be controlled by the aperture shapes of the VCSEL elements 11 to 16, respectively. The smaller the opening area of each of the VCSEL elements 11 to 16, the smaller the power consumption.

A relative positional relationship between the VCSEL elements 11 to 16 is determined in advance, and is referred to when creating an estimation model used when detecting the line-of-sight direction by the processing unit 100 . The line-of-sight detection device 10 detects the line-of-sight direction based on the relative positional relationship between the VCSEL elements 11 to 16 shown in FIG. 2 used to create the estimation model.

(Example of incidence of laser light L4 on photodiode 4)
Next, the incidence of the laser light L4 on the photodiode 4 will be described with reference to FIGS. 3 and 4. FIG. FIG. 3 is a diagram showing an example of incidence of the laser light L4 on the photodiode 4 according to the embodiment. FIG. 4 is a diagram showing laser light incident on a photodiode according to a comparative example.

As shown in FIG. 3( a ), the laser light L 4 is incident on the light receiving surface 41 of the photodiode 4 when the line-of-sight direction 50 of the eyeball is the normal viewing direction. The laser beam L4 includes a laser beam L41 and a laser beam L42. In addition, the laser beam L4 is a generic notation when the laser beam L41 and the laser beam L42 are not particularly distinguished. Both laser beams L41 and L42 diverge as they approach the light receiving means. In FIG. 3A, the spread angle is φ, and all of the spread laser light L4 is incident on the light receiving surface 41 of the photodiode 4. In FIG. However, the laser light incident on the light-receiving surface 41 may not be all of the spread laser light L4. For example, part of the laser beam L42 may protrude from the light receiving surface 41 in the -Y direction.

Laser light L41 is emitted from one of VCSEL elements 11 to 16, for example, VCSEL element 11, passes through lens 2 and concave mirror 3, and is reflected by eyeball 30 to light receiving surface 41 of photodiode 4. Incident. Laser light L42 is emitted from a VCSEL element other than the one VCSEL element among VCSEL elements 11 to 16, for example, VCSEL element 12, passes through lens 2 and concave mirror 3, is reflected by eyeball 30, and becomes a photo. It is incident on the light receiving surface 41 of the diode 4 . The VCSEL array 1 emits laser light L41 and laser light L42.

When the laser light L41 and the laser light L42 are incident on the light receiving surface of the photodiode 4, the photodiode 4 outputs an electrical signal based on the received light intensity. Here, when the tilt of the eyeball changes as shown in FIG. 3B, the positions and angles at which the laser light L41 and the laser light L42 enter the surface of the eyeball change. By changing the incident position on the eyeball, the landing position of the light reflected from the eyeball on the photodiode 4 changes. For example, when the eyeball tilts to the left as shown in FIG. 's landing position shifts in the -Y direction.

A change in the landing position causes a change in the amount of light that lands on the light receiving surface of the photodiode 4 , thereby changing the magnitude of the electrical signal output by the photodiode 4 . More specifically, in FIG. 3B, part of the laser beam L41 and the laser beam L42 protrudes from the light receiving surface of the photodiode 4 because the landing position on the photodiode 4 is changed. Therefore, the electrical signals P41 and P42 are smaller than in the case of FIG. 3(a).

FIG. 3(c) shows the state of the laser L4 when the light receiving surface of the photodiode 4 is viewed from the YZ plane. In this embodiment, the center-to-center distance d between the laser beams L41 and L42 satisfies the following equation.
|h−w/2|≦d≦h+w/2
However, h is the beam radius and w is the length of the light receiving surface. When the center-to-center distance d exceeds the upper limit, a plurality of laser beams do not enter the light receiving surface of the photodiode 4, so the tilt of the eyeball cannot be properly estimated. On the other hand, when the lower limit is not reached, there is no change in the amount of received light for the laser beams L41 and L42 with respect to the change in the inclination of the eyeball, and in this case also appropriate estimation cannot be performed.

In this embodiment, by modeling the correlation between the inclination of the eyeball and the output signal intensity of the photodiode 4 based on a plurality of laser beams, the inclination of the eyeball can be determined from the signal intensity of the photodiode 4 based on the plurality of laser beams. can be estimated.

FIG. 4 shows a comparative example. Unlike FIG. 3, even if the tilt of the eyeball changes, the output signal intensity of the photodiode 4 based on the laser L42 does not change. can't get Therefore, in this case, the tilt of the eyeball cannot be estimated.

In this embodiment, the distance D (see FIG. 2) between adjacent VCSEL elements in the VCSEL elements 11 to 16 is Y It is shorter than the width w, which is the length along each of the direction and the Z direction.

Furthermore, at least a portion of the laser beams L41 and L42 overlap each other on the light receiving surface 41. FIG. An overlap region e in FIG. 3 represents the length along the Y direction of the region where the laser beams L41 and L42 overlap each other on the light receiving surface 41. FIG.

In the line-of-sight detection device 10, the distance along the light emitting surface 1A between the VCSEL elements, the curvature radius and installation position of the lens 2, and the curvature radius and An installation position is predetermined. The line-of-sight detection device 10 can increase the beam size of the laser light L4 on the light receiving surface 41 of the photodiode 4 by making the laser light L4 incident on the photodiode 4 . As a result, the photodiode 4 can receive the laser beams L41 and L42 emitted from the VCSEL elements 11 and 12 of the VCSEL array 1 and reflected by the eyeball 30 .

If the beam diameter of the laser light L4 on the light receiving surface 41 of the photodiode 4 is increased, there is a high probability that a part of the laser light L4 will deviate from the light receiving surface when the tilt of the eyeball changes. The correlation with the output signal value of the photodiode 4 is strengthened. Therefore, it is desirable that the beam diameter of the laser light L4 on the light receiving surface 41 of the photodiode 4 is large. If the beam diameter of the laser light L4 is equal to or greater than the length along the predetermined direction of the light receiving surface of the photodiode 4, the amount of change in the output signal of the photodiode 4 with respect to the inclination of the eyeball is increased, which is preferable.

However, the incidence of the laser beam L4 on the photodiode 4 is not limited to the state shown in FIG. The line-of-sight detection device 10 is configured so that the laser light L4 is in the state shown in FIG. 3 radius of curvature and installation position can also be determined.

(Hardware configuration of processing unit 100)
Next, the processing unit 100 will be described. FIG. 2 is a block diagram illustrating an example of the hardware configuration of the processing unit 100. As illustrated in FIG.

As shown in FIG. 5, the processing unit 100 includes a CPU (Central Processing Unit) 101, a ROM (Read Only Memory) 102, a RAM (Random Access Memory) 103, and an SSD (Solid State Drive) 104. . The processing unit 100 also includes a light source driving circuit 105, an I/V (Intensity of current/Volt) conversion circuit 106, an A/D (Analog/Digital) conversion circuit 107, an input/output I/F (Interface) 108, and , has These are electrically connected via a system bus B so as to be able to communicate with each other.

The CPU 101 is an arithmetic device that reads out programs and data from a storage device such as the ROM 102 and the SSD 104 onto the RAM 103 and executes the programs, thereby controlling the entire processing unit 100 and realizing functions described later.

The ROM 102 is a non-volatile semiconductor memory (storage device) capable of retaining programs and data even when power is turned off. The ROM 102 stores a BIOS (Basic Input/Output System) executed when the processing unit 100 is started, programs such as OS settings and network settings, and data. A RAM 103 is a volatile semiconductor memory (storage device) that temporarily holds programs and data.

The SSD 104 is a nonvolatile memory that stores programs for executing processing by the processing unit 100 and various data. Note that the SSD may be an HDD (Hard Disk Drive).

The light source drive circuit 105 is an electric circuit that is electrically connected to the VCSEL array 1 and outputs a drive signal Dr to the VCSEL array 1 according to a control signal to drive the VCSEL array 1 to emit light. In this embodiment, the light source drive circuit 105 includes a demultiplexer circuit that switches the output destination of the drive signal Dr. The drive signal Dr is a current signal, but may be a voltage signal.

The I/V conversion circuit 106 is an electric circuit that is electrically connected to the photodiode 4 and outputs an analog voltage signal obtained by I/V converting the analog electric signal Se output from the photodiode 4 .

The A/D conversion circuit 107 is an electric circuit that outputs a digital voltage signal obtained by A/D converting the analog voltage signal output from the I/V conversion circuit 106 .

The input/output I/F 108 is an interface that enables communication between an external device and the processing unit 100 .

(Functional configuration of processing unit 100)
FIG. 6 is a block diagram showing an example of the functional configuration of the processing unit 100. As shown in FIG.

As shown in FIG. 6, the processing unit 100 includes a selection unit 111, a light source driving unit 112, a switching unit 113, an I/V conversion unit 114, an A/D conversion unit 115, and a first storage unit 116. , a determination unit 117 , an estimation unit 118 , and an output unit 119 .

The processing unit 100 realizes the functions of the selection unit 111 and the estimation unit 118 by executing a predetermined program stored in the ROM 102 by the CPU 101, and selects the light source driving unit 112 and the switching unit 113 by the light source driving circuit 105 and the like. Realize each function. Further, the processing unit 100 realizes each function of the I/V conversion unit 114 and the A/D conversion unit 115 by the I/V conversion circuit 106, the A/D conversion circuit 107, etc., and the first accumulation unit 116 by the RAM 103, etc. , and the function of the output unit 119 is realized by an input/output I/F or the like.

Note that the processing unit 100 may realize at least part of each of the functions described above by an electronic circuit such as an ASIC (application specific integrated circuit) or an FPGA (Field-Programmable Gate Array). In addition, the processing unit 100 may implement at least part of each of the functions described above by distributed processing of a plurality of functional configuration units, or by using an external device.

The selection unit 111 selects a VCSEL element to emit light from among the VCSEL elements 11 to 16 included in the VCSEL array 1 and outputs information indicating the selected VCSEL element to the switching unit 113 . Note that the number of 6 VCSEL elements included in the VCSEL array 1 is an example. The number of VCSEL elements may be any number as long as it is plural.

The light source driving section 112 outputs a driving signal Dr for causing the VCSEL array 1 to emit light to the VCSEL element selected by the selecting section 111 via the switching section 113 .

The switching unit 113 switches such that the driving signal Dr from the light source driving unit 112 is output to the VCSEL element selected by the selecting unit 111 .

The I/V converter 114 converts the electrical signal Se output from the photodiode 4 into an analog voltage signal, and outputs the converted voltage signal to the A/D converter 115 .

The A/D conversion unit 115 converts the analog voltage signal input from the I/V conversion unit 114 into digital voltage data Da, and then outputs the digital voltage data Da to the first storage unit 116 .

The first accumulation unit 116 accumulates all the digital voltage data Da input from the A/D conversion unit 115 for the number of VCSEL elements. The first storage unit 116 can store the digital voltage data Da and the VCSEL elements 11 to 16 in association so that they can be referenced in the processing by the determination unit 117 and the estimation unit 118 .

The determination unit 117 determines whether or not all of the VCSEL elements 11 to 16 are caused to emit light. For example, the determination unit 117 determines whether or not all the VCSEL elements 11 to 16 have been caused to emit light based on whether or not all the digital voltage data Da accumulated in the first accumulation unit 116 have been accumulated for the number of VCSEL elements. I can judge.

Based on the digital voltage data Da accumulated in the first accumulation unit 116, the estimation unit 118 uses an estimation model created in advance to calculate line-of-sight direction information Ed. The estimation unit 118 outputs the calculated line-of-sight direction information Ed to an external device via the output unit 119 .

<Example of Processing by Processing Unit 100>
FIG. 7 is a flowchart showing an example of processing by the processing unit 100. As shown in FIG. The processing unit 100a starts the processing of FIG. 7, for example, in response to a start operation input signal by the user of the line-of-sight detection device 10 (hereinafter simply referred to as the user).

First, in step S71, the processing unit 100 selects which of the VCSEL elements 11 to 16 is to emit light by the selection unit 111, and outputs information indicating the selected VCSEL element to the switching unit 113. .

Subsequently, in step S72, the processing unit 100 outputs the driving signal Dr from the light source driving unit 112 via the switching unit 113 to the selected VCSEL element. The VCSEL element to which the drive signal Dr is applied emits laser light L1 according to the drive signal Dr. After passing through the lens 2 and the concave mirror 3, the laser beam L1 is reflected by the eyeball 30 and enters the photodiode 4 as the laser beam L4. The photodiode 4 outputs an electric signal Se to the processing unit 100 according to the light intensity of the received laser light L4.

Subsequently, in step S73, the processing unit 100 causes the I/V conversion unit 114 to convert the electrical signal Se into an analog voltage signal, and then the A/D conversion unit 115 converts the analog voltage signal into digital voltage data Da. do. The processing unit 100 causes the first storage unit 116 to store the digital voltage data Da in association with the VCSEL element that emits light.

Subsequently, in step S74, the processing unit 100 causes the determination unit 117 to determine whether or not all the VCSEL elements 11 to 16 have been caused to emit light.

When it is determined in step S74 that all the VCSEL elements are not caused to emit light (step S74, No), the processing unit 100 performs the processing after step S71 again. In this case, the processing unit 100 selects a VCSEL element that has not yet emitted light using the selection unit 111, and causes this VCSEL element to emit light.

On the other hand, when it is determined in step S74 that all the VCSEL elements have been caused to emit light (step S74, Yes), in step S75, the processing unit 100 causes the estimation unit 118 to use an estimation model created in advance. Then, based on the six digital voltage data Da corresponding to the VCSEL elements 11 to 16 accumulated in the first accumulation unit 116, the line-of-sight direction information Ed is obtained by calculation.

Subsequently, in step S<b>76 , the processing unit 100 outputs the line-of-sight direction information Ed acquired by the estimation unit 118 to the external device via the output unit 119 .

Subsequently, in step S77, the processing section 100 determines whether or not to end the processing. For example, the processing unit 100 can determine whether or not to end according to a user's end operation input signal or the like.

In step S77, when it is determined to end (step S77, Yes), the processing unit 100 ends the process. On the other hand, when it is determined not to end (step S77, No), the processing section 100 performs the processing after step S71 again.

In this manner, the processing unit 100 can output line-of-sight direction information Ed based on the tilt of the eyeball 30 .

(Example of estimation model by estimation unit 118)
Next, the estimation model used in the estimation calculation by the estimation unit 118 will be described. The estimation model is created by, for example, Canonical Correlation Analysis (for example, non-patent document "Shotaro Ako, Journal of Japanese Neural Network Society, Vol.20, No.2 (2013), p62-72) ”, etc.).

Canonical correlation analysis is a kind of multivariate analysis, and is a method of extracting common components from two sets of high-dimensional data x and y. For example, it is possible to express the following equations (1) and (2), where high-dimensional data x is line-of-sight direction data and high-dimensional data y is digital voltage data Da.

In equations (1) and (2), θx is the horizontal viewing direction [degrees], and θz is the vertical viewing direction [degrees]. P1 to P6 are digital voltage data Da corresponding to each light emitting unit. x is two-dimensional data and y is six-dimensional data.

The estimating unit 118 can calculate scalar quantities u and v represented by the following equation (3) by calculating the inner product of the linear transformation vectors a and b for the two high-dimensional data x and y. The number of dimensions of the linear transformation vector a is equal to the number of dimensions of the high-dimensional data x, and the number of dimensions of the linear transformation vector b is equal to the number of dimensions of the high-dimensional data y.

In equation (3), linear transformation vectors a and b are obtained so that the correlation coefficients of the scalar quantities u and v are large. The estimating unit 118 can obtain the scalar quantities u and v in order of the largest correlation coefficient, the next largest correlation coefficient, the third largest correlation coefficient, and so on. Note that the maximum number of pairs of scalar quantities u and v that can be obtained by the canonical correlation analysis is equal to the number of dimensions of data with a small number of dimensions among the original high-dimensional data. For example, as a result of analyzing the high-dimensional data represented by equations (1) and (2) by canonical correlation analysis, two sets of scalar quantities u and v are obtained.

（４）式における係数ｃおよびｄは、例えば最小二乗法で求めることができる。 Since there is a strong correlation between the scalar quantities u and v obtained by the canonical correlation analysis, the relationship between the scalar quantities u and v can be approximated by a linear curve represented by Equation (4).

The coefficients c and d in the equation (4) can be obtained, for example, by the method of least squares.

If two pairs of scalar quantities u and v represented by equation (4) can be obtained by canonical correlation analysis, the estimation unit 118 converts the high-dimensional data y corresponding to each VCSEL element into the following simultaneous equations: By substituting, the estimated value of the line-of-sight direction can be calculated.

The subscript of each variable in equation (5) is a variable corresponding to the correlation coefficient obtained by the canonical correlation analysis. For example, the linear transformation vectors corresponding to the largest correlation variables can be denoted as a1 and b1, and the linear approximation parameters of the scalar quantities u and v can be denoted as c1 and d1.

Also, the linear transformation vectors corresponding to the second largest correlation variables can be denoted as a2 and b2, and the linear approximation parameters of the scalar quantities u and v can be denoted as c2 and d2.

Expression (5) is a mathematical expression representing an estimation model. When the digital voltage data Da corresponding to the high-dimensional data y is obtained, the estimating unit 118 substitutes the digital voltage data Da into Equation (5) to estimate the high-dimensional data x corresponding to the line-of-sight direction information Ed. can.

In order to construct an estimation model by canonical correlation analysis, sample data of the high-dimensional data x and y are prepared in advance before performing the gaze direction estimation operation. The sample data may be obtained by experiments or may be obtained by optical simulations.

As the number of sample data increases, the estimation accuracy of the estimation model improves. Further, in order to deal with individual differences in the eyeballs 30, a plurality of estimation models are created in advance, and the model with the highest accuracy is selected, or an estimation model is created for each eyeball 30, thereby improving the estimation accuracy. improves.

<Example of simulation result of incident position of laser light L4 on light receiving surface 41>
Next, referring to FIGS. 8 to 14, the incident position of the laser light L4 emitted from the VCSEL array 1, reflected by the eyeball 30, and incident on the light receiving surface 41 of the photodiode 4 is determined by optical simulation. will be explained.

FIG. 8 shows the incident positions of the laser beams L4 emitted from all of the VCSEL elements 11 to 16 included in the VCSEL array 1 and incident on the light receiving surface 41. FIG. 9 to 14 individually show the incident positions of the laser light L4 corresponding to the VCSEL elements 11 to 16, respectively.

9 shows VCSEL element 11, FIG. 10 shows VCSEL element 12, FIG. 11 shows VCSEL element 13, FIG. 12 shows VCSEL element 14, FIG. 13 shows VCSEL element 15, and FIG. shows the incident position of the laser beam L4 incident on .

8 to 14, the horizontal axis represents the Y-direction position [mm] along the Y-direction, and the vertical axis represents the Z-direction position [mm] along the Z-direction. In this optical simulation, the line-of-sight direction of the eyeball 30 is the stereoscopic direction, that is, θx=0 [degree] and θz=0 [degree].

The center of the light receiving surface 41 of the photodiode 4 indicated by the rectangular frame is located at 0 [mm] in the Y direction and 0 [mm] in the Z direction. Spots 411 to 416 represent beam spots of the laser light L4 emitted from the VCSEL elements 11 to 16 and reaching the vicinity of the light receiving surface 41 .

The spot 411 is the VCSEL element 11, the spot 412 is the VCSEL element 12, the spot 413 is the VCSEL element 13, the spot 414 is the VCSEL element 14, the spot 415 is the VCSEL element 15, and the spot 416 is the VCSEL element 16. , representing the beam spot based on

As shown in FIGS. 8-14, a portion of each of spots 411-416 is incident on light-receiving surface 41. FIG. From this, it was found that in the line-of-sight detection device 10 , the laser light L 4 emitted from the VCSEL elements 11 to 16 included in the VCSEL array 1 and reflected by the eyeball 30 can enter the light receiving surface 41 .

All the distances d between adjacent spots satisfy the following formula.
|h−w/2|≦d≦h+w/2

Here, an example in which a part of each of the spots 411 to 416 is incident on the light receiving surface 41 is shown. 10 effects are obtained. Moreover, if at least two or more of the spots 411 to 416 are incident on the light receiving surface 41, the effects of the line-of-sight detection device 10 can be obtained.

<Simulation of line-of-sight direction detection>
Next, a simulation method of sight line direction detection by the sight line detection device 10 and its results will be described.

(Simulation method)
First, the tilt range of the eyeball 30 is set to −4 [degrees]≦θx≦4 [degrees] and −4 [degrees]≦θy≦4 [degrees]. A sample of light intensity data was acquired for each VCSEL element in array 1 . Based on this sample, an estimation model was created using canonical correlation analysis.

Next, the estimated value of the line-of-sight data was obtained by substituting the light intensity data for each of the 60 pieces of line-of-sight data that were not used to create the estimation model into the created estimation model.

Next, the estimated value and the correct value of the line-of-sight data were compared.

(simulation result)
15 and 16 are diagrams illustrating simulation results of sight line direction detection by the sight line detection device 10. FIG. FIG. 15 shows the simulation result of the line-of-sight direction θx in the X direction, in other words, the horizontal direction, and FIG. 16 shows the simulation result of the line-of-sight direction θz in the Z direction, in other words, the vertical direction. In FIGS. 15 and 16 , the horizontal axis represents the correct value of the line-of-sight direction, and the vertical axis represents the estimated value by the estimation unit 118 .

As shown in FIGS. 15 and 16, it was found that there is a correlation between the correct values and the estimated values. The root-mean-square error (RMSE) Δθx of the result of visual line direction estimation in the horizontal direction is 0.8 [degree], and the root-mean-square error (RMSE) Δθz of the result of visual line direction estimation in the vertical direction is 0.6 [degree]. became.

From the simulation results of FIGS. 15 and 16, it was found that the line-of-sight detection device 10 can estimate the line-of-sight direction based on the electrical signal Se output from the photodiode 4 and obtain the line-of-sight direction information Ed.

<Comparative example, modified example>
(Comparative example)
FIG. 17 is a diagram showing the arrangement of VCSEL elements 11X to 16X included in a VCSEL array 1X according to a comparative example. This comparative example differs from the embodiment in the arrangement of the VCSEL elements 11X to 16X and the divergence angle of the laser light L4 incident on the photodiode 4, respectively.

Specifically, in the comparative example, as shown in FIG. 17, VCSEL element 11X and VCSEL element 12X, VCSEL element 12X and VCSEL element 13X, VCSEL element 14X and VCSEL element 15X, and VCSEL element 15X and VCSEL element 16X The distance along each Y direction is the distance D. FIG. On the other hand, the distances in the Z direction between VCSEL elements 11X and 14X, between VCSEL elements 12X and 15X, and between VCSEL elements 13X and 16X are twice the distance D.

Further, in the comparative example, in the arrangement of the VCSEL array 1X in FIG. 17, the spread angle of the laser light L4 incident on the photodiode 4 is smaller than the spread angle φ according to the embodiment.

As described above, in the comparative example, of the laser beams L emitted from the respective VCSEL elements, only one laser beam L4 emitted from one VCSEL element and reflected by the eyeball 30 enters the light receiving surface 41. ing. In other words, in the comparative example, the photodiode 4 cannot receive the multiple laser beams L4 emitted from the multiple VCSEL elements and reflected by the eyeball 30 .

FIG. 18 is a diagram showing results obtained by optical simulation of the incident positions of the laser beams L4 emitted from all the VCSEL elements 11X to 16X included in the VCSEL array 1X and incident on the light receiving surface 41. FIG.

As in FIG. 8 described above, the horizontal axis represents the Y-direction position [mm] along the Y-direction, and the vertical axis represents the Z-direction position [mm] along the Z-direction. The line-of-sight direction of the eyeball 30 is the stereoscopic direction, that is, θx=0 [degree] and θz=0 [degree].

In FIG. 18, the center of the light receiving surface 41 of the photodiode 4 indicated by the rectangular frame is positioned at 0 [mm] in the Y direction and 0 [mm] in the Z direction. Spots 411X to 416X represent beam spots of laser light L4 that is emitted from the VCSEL elements 11X to 16X, reflected by the eyeball 30, and reaches the vicinity of the light receiving surface 41. FIG.

Spot 411X is the VCSEL element 11X, spot 412X is the VCSEL element 12, spot 413X is the VCSEL element 13, spot 414X is the VCSEL element 14, spot 415X is the VCSEL element 15, and spot 416X is the VCSEL element 16. , representing the beam spot based on

As shown in FIG. 18 , in the optical simulation according to the comparative example, only spot 415X among spots 411X to 416X is incident on light receiving surface 41 .

19 and 20 are diagrams showing simulation results of line-of-sight direction detection according to the comparative example. FIG. 19 is a diagram showing simulation results of the line-of-sight direction θx in the X direction, and FIG. 20 is a diagram showing simulation results of the line-of-sight direction θz in the Z direction.

The view of FIGS. 19 and 20 is the same as that of FIGS. 15 and 16 described above, and the simulation method is also the same as the method of obtaining FIGS. 15 and 16 described above.

As shown in FIGS. 19 and 20, in the comparative example, a high correlation between the correct value and the estimated value was not obtained as compared with the embodiment. In particular, in the viewing direction θz in the Z direction in FIG. 20, the correlation between the correct value and the estimated value was low.

From the simulation results of FIGS. 19 and 20, it was found that the line-of-sight direction cannot be accurately estimated based on the electrical signal Se output from the photodiode 4 in the comparative example.

(Modification)
FIG. 21 is a diagram showing an example of arrangement of VCSEL elements 11a to 17a included in a VCSEL array 1a according to a modification of the embodiment. This modification differs from the VCSEL array 1 according to the first embodiment only in the arrangement of the VCSEL elements 11a to 17a.

Specifically, in the modified example, the VCSEL elements 11a to 17a are arranged in a hexagonal close-packed manner in a plan view viewed from the -X direction side. The interval between adjacent VCSEL elements among the VCSEL elements 11a to 17a is the interval D in each case.

Even in the arrangement of such a modified example, the photodiode 4 can receive a plurality of laser beams L4 emitted from a plurality of VCSEL elements among the VCSEL elements 11a to 17a and reflected by the eyeball 30. FIG.

FIG. 22 is a diagram showing the incident positions of the laser beams L4 emitted from all the VCSEL elements 11a to 17a included in the VCSEL array 1a and incident on the light receiving surface 41. FIG. 23 to 29 are diagrams showing incident positions of the laser beams L4 emitted from the VCSEL elements 11a to 17a and incident on the light receiving surface 41 for each VCSEL element.

23 shows VCSEL element 11a, FIG. 24 shows VCSEL element 12a, FIG. 25 shows VCSEL element 13a, FIG. 26 shows VCSEL element 14a, FIG. 27 shows VCSEL element 15a, FIG. 28 shows VCSEL element 16a, and FIG. The incident position of the laser beam L4 emitted and incident on the light receiving surface 41 is shown.

22 to 29, the horizontal axis indicates the Y-direction position [mm] along the Y-direction, and the vertical axis indicates the Z-direction position [mm] along the Z-direction. In this optical simulation, the line-of-sight direction of the eyeball 30 is the stereoscopic direction, that is, θx=0 [degree] and θz=0 [degree].

The center of the light receiving surface 41 of the photodiode 4 indicated by the rectangular frame is located at 0 [mm] in the Y direction and 0 [mm] in the Z direction. Spots 411 a to 417 a represent beam spots of laser light L 4 emitted from the VCSEL elements 11 to 17 and reaching the vicinity of the light receiving surface 41 .

Spot 411a is for VCSEL element 11a, spot 412a is for VCSEL element 12a, spot 413a is for VCSEL element 13a, spot 414a is for VCSEL element 14a, spot 415a is for VCSEL element 15a, spot 416a is for VCSEL element 16a, and spot 417a is for VCSEL element 17a. A beam spot based on laser light L emitted from .

As shown in FIG. 22 , in the optical simulation according to the modification, all of the spots 411 a to 416 a are partially incident on the light receiving surface 41 . 23 to 29, a portion of each of the spots 411a to 417a is incident on the light receiving surface 41. In FIGS.

30 and 31 are diagrams showing simulation results of line-of-sight direction detection according to the modification. FIG. 30 shows the simulation result of the line-of-sight direction θx in the X direction, and FIG. 31 shows the simulation result of the line-of-sight direction θz in the Z direction.

The view of FIGS. 30 and 31 is the same as that of FIGS. 15 and 16 described above, and the simulation method is the same as the method of obtaining FIGS. 15 and 16 described above.

As shown in FIGS. 30 and 31, it was found that there is a correlation between the correct value and the estimated value also in the modified example. The root-mean-square error (RMSE) Δθx of the result of visual line direction estimation in the horizontal direction is 0.6 [degrees], and the root-mean-square error (RMSE) Δθz of the result of visual line direction estimation in the vertical direction is 0.3 [degrees]. became.

From the simulation results of FIGS. 30 and 31, the line-of-sight detection device 10 having the VCSEL array 1a according to the modification estimates the line-of-sight direction based on the electrical signal Se output from the photodiode 4, and obtains the line-of-sight direction information Ed. I found it possible.

<Action and effect of line-of-sight detection device 10>
Next, the effects of the line-of-sight detection device 10 will be described.

Recently, technologies related to "augmented reality (AR)" and "virtual reality (VR)" are attracting attention. In particular, AR technology is expected to be applied not only for entertainment purposes but also in a wide range of fields such as work support at manufacturing sites and medical sites.

Glasses-type image display devices have been developed as devices for displaying AR. Among them, a retinal projection display device that projects an image directly onto the retina has the advantage that it is not necessary to focus the eye on the projected image. With this, you can always see a clear AR image while keeping your eyes focused on an object in the outside world. Furthermore, by combining eye-tracking technology that tracks the user's line-of-sight direction, it is possible to increase the angle of view of the display and to operate the displayed image by the line of sight.

The glasses-type image display device uses a line-of-sight detection device that detects the inclination of the eyeball and detects the line-of-sight direction of the person wearing the image display device. As such a line-of-sight detection device, in order to detect the inclination of the eyeball with high accuracy, a configuration is disclosed in which the eyeball is scanned with a laser beam irradiated by a MEMS mirror, and the reflected light of the scanned light from the eyeball is detected. there is

However, since the disclosed configuration uses the MEMS mirror, which is a movable part, the configuration of the line-of-sight detection device is complicated, and the configuration of the driving circuit for stably driving the MEMS mirror is also complicated. Become.

The line-of-sight detection device 10 (tilt detection device) according to the present embodiment includes a VCSEL array 1 (plurality of light emitting units), and a laser beam L4 emitted by the VCSEL array 1 and reflected by the eyeball 30 (reflected by the object). ) and outputs an electric signal Se based on the light intensity of the received laser light L4. The line-of-sight detection device 10 also has a processing unit 100 (output means) that outputs line-of-sight direction information Ed (detection result of the inclination of the object) of the eyeball 30 based on the electrical signal Se output from the photodiode 4 .

The photodiode 4 emits light from a plurality of VCSEL elements in a state in which the inclination of the eyeball 30 (object) is substantially the same, and emits a plurality of laser beams L4 reflected by the eyeball 30 (a plurality of light beams reflected by the object). ).

The line-of-sight detection device 10 detects the inclination of the eyeball 30 based on the plurality of laser beams L1 emitted from the VCSEL array 1, and therefore uses a movable part such as a MEMS mirror and a drive circuit for driving the movable part. The tilt of the eyeball 30 can be detected without any need. By not using a movable part such as a MEMS mirror and a drive circuit for driving the movable part, the configuration of the line-of-sight detection device 10 can be simplified. In addition, by simplifying the configuration of the line-of-sight detection device 10, the line-of-sight detection device 10 can be made smaller, or the manufacturing cost of the line-of-sight detection device 10 can be reduced.

In addition, since the line-of-sight detection device 10 receives a plurality of laser beams L4 emitted from a plurality of VCSEL elements and reflected by the eyeball 30, the movement of the eyeball 30 can be tracked based on the electrical signal Se from the photodiode 4. . The line-of-sight detection device 10 simplifies the configuration of the light receiving means and the configuration of the driving means for driving the light receiving means, compared to the case of tracking the movement of the eyeball 30 using other light receiving means such as PSD. can.

As described above, the present embodiment can provide a line-of-sight detection device with a simple configuration.

In the present embodiment, the VCSEL array 1 is provided side by side along the X direction and the Y direction (predetermined direction) along the light emitting surface 1A of the VCSEL array 1. The spacing D along both the direction and the Y direction are equal. Thereby, the photodiode 4 can receive a plurality of laser beams L4 emitted from the plurality of VCSEL elements and reflected by the eyeball 30. FIG. As a result, the line-of-sight detection device 10 can obtain the effects described above.

In addition, since the VCSEL elements 11 to 16 are arranged in the line-of-sight detection device 10 along the Y direction and the Z direction (two different directions), the line of sight detection device 10 detects the movement of the eyeball 30 around the Y axis and the Z axis. can be tracked. Thereby, the line-of-sight detection device 10 can expand the tilt detection range of the eyeball 30 and the detection range of the line-of-sight direction by the line-of-sight detection device 10 .

Further, in this embodiment, at least a portion of the plurality of laser beams L4 emitted from the plurality of VCSEL elements in the VCSEL array 1 and reflected by the eyeball 30 overlap on the light receiving surface 41 of the photodiode 4 . As a result, the plurality of laser beams L4 can be incident on the light receiving surface 41 at close intervals. is easier to receive.

In this embodiment, the length of the interval D along the Y and Z directions between adjacent VCSEL elements in the VCSEL array 1 is the length of the light receiving surface 41 of the photodiode 4 along the Y and Z directions. is shorter than the width w. As a result, the plurality of laser beams L4 can be incident on the light receiving surface 41 at close intervals. is easier to receive.

In addition, in this embodiment, the laser light L4 reflected by the eyeball 30 spreads as it approaches the photodiode 4 . As a result, the spot size of the laser beam L4 can be increased, so that the photodiode 4 can easily receive the multiple laser beams L4 emitted from the multiple VCSEL elements and reflected by the eyeball 30 . In addition, since the spot size of the laser beam L4 can be increased, at least part of the laser beam L4 can be easily incident on the light receiving surface 41, and the sight line detection device 10 can improve the stability of sight line direction detection.

In this embodiment, the eyeball 30 is irradiated with the laser beam L3 converged using the concave mirror 3, but the present invention is not limited to this. As shown in FIG. 32 , the line-of-sight detection device 10 may be configured such that the laser light L2 converged by the lens 2 is incident on the eyeball 30 without having a concave mirror. Alternatively, as shown in FIG. 33, the line-of-sight detection device 10 may be configured such that the laser light L1 emitted from the VCSEL array 1 is directly incident on the eyeball 30 without having a concave mirror and lens. . For example, FIG. 32 shows a case where laser light L2 focused by lens 2 is configured to enter eyeball 30 without a concave mirror. Since the laser light emitted from the VCSEL array 1 is configured to enter the eyeball 30 without passing through the concave mirror, the number of optical elements can be reduced, and the manufacturing cost can be reduced. On the other hand, having a concave mirror is desirable in that noise in the output signal of the photodiode 4 can be reduced because the photodiode 4 can be installed in the vicinity of the processing section.

[Second embodiment]
Next, the line-of-sight detection device 10a according to the second embodiment will be described. In addition, the same code|symbol is attached|subjected to the structure part same as 1st Embodiment, and the overlapping description is abbreviate|omitted suitably. This point also applies to the description of other embodiments shown below.

<Functional Configuration Example of Processing Unit 100a>
FIG. 34 is a block diagram showing an example of the functional configuration of the processing unit 100a included in the line-of-sight detection device 10a according to this embodiment. As shown in FIG. 34, the processing unit 100a includes an encoding unit 120, a light source modulation unit 121, an inner product calculation unit 122, and a second accumulation unit 123.

The line-of-sight detection device 10a controls light emission by the VCSEL array 1 so that the light source modulation unit 121 emits the laser light L time-modulated by codes having orthogonality. Further, the line-of-sight detection device 10a separates and detects a plurality of time-modulated laser beams L4 based on the electrical signal Se output from the photodiode 4 by the processing unit 100a. Thereby, in this embodiment, the processing time and processing load of the processing unit 100a are reduced. Further, in the present embodiment, even when the light intensity of the laser beam L4 incident on the eyeball 30 is low, the effect of background light is removed to ensure high line-of-sight detection accuracy.

The processing unit 100a realizes the functions of the encoding unit 120 and the inner product calculation unit 122 by executing a predetermined program stored in the ROM 102 by the CPU 101, and the functions of the light source modulation unit 121 by the light source driving circuit 105 and the like. Realize Also, the processing unit 100a realizes the function of the second storage unit 123 using the RAM 103 and the like.

The encoder 120 generates a plurality of encoded data and assigns the encoded data to each VCSEL element of the VCSEL array 1 via the light source modulator 121 . In addition, for the inner product operation by the inner product operation unit 122, the encoding unit 120 outputs the same data as the encoded data assigned to the VCSEL element to the inner product operation unit 122 as the reference data Rf.

The encoded data includes codes having orthogonality. This code is, for example, a Hadamard code. A Hadamard code is a code system used for error detection and correction of signals. Different encoded data are created according to the positions of the VCSEL elements in the VCSEL array 1 and stored in the SSD 104 or the like. The encoding unit 120 can allocate the encoded data acquired according to the position of the VCSEL element to the corresponding VCSEL element by referring to the SSD 104 or the like.

The light source modulation unit 121 is an example of a control unit that controls light emission by the VCSEL array 1 so as to emit laser light L1 time-modulated by codes having orthogonality. Specifically, the light source modulation unit 121 outputs a current-modulated drive signal Dr to each VCSEL element based on the encoded data input from the encoding unit 120, thereby temporally modulating the laser light L1. All VCSEL elements are made to emit light in parallel. The number of light source modulation units 121 is not limited to one, and the processing unit 100 a may have as many light source modulation units 121 as the number of VCSEL elements included in the VCSEL array 1 .

The inner product calculation unit 122 refers to the first storage unit 116 to acquire the digital voltage data Da for one cycle of the predetermined code cycle stored in the first storage unit 116, and obtains the digital voltage data Da and the code and the reference data Rf input from the transforming unit 120 are calculated. The inner product calculation unit 122 performs inner product calculation using all encoded data used for encoding, and causes the second accumulation unit 123 to accumulate the inner product calculation results. The inner product calculation unit 122 can separate and detect the digital voltage data Da corresponding to each VCSEL element by using the orthogonality of the encoded data by performing the inner product calculation using all the encoded data.

<Example of Processing by Processing Unit 100a>
FIG. 35 is a flowchart showing an example of processing by the processing unit 100a. The processing unit 100a starts the processing of FIG. 35, for example, in response to a user's start operation input signal.

First, in step S331, the processing unit 100a causes the encoding unit 120 to generate a plurality of encoded data, and assigns the encoded data to each VCSEL element of the VCSEL array 1 via the light source modulation unit 121. The encoding unit 120 also outputs the reference data Rf to the inner product calculation unit 122 .

Subsequently, in step S332, the processing unit 100a outputs the drive signal Dr time-modulated based on the encoded data to the VCSEL array 1 by the light source modulation unit 121. FIG. The VCSEL element to which the drive signal Dr is applied emits laser light L1 according to the drive signal Dr. After passing through the lens 2 and the concave mirror 3, the laser beam L1 is reflected by the eyeball 30 and enters the photodiode 4 as the laser beam L4. The photodiode 4 outputs an electrical signal Se corresponding to the light intensity of the received laser beam L4 to the processing unit 100a.

Subsequently, in step S333, the processing unit 100a causes the I/V conversion unit 114 to convert the electrical signal Se from the photodiode 4 into an analog voltage signal, and then the A/D conversion unit 115 to convert it into digital voltage data Da. do. The processing unit 100a causes the first storage unit 116 to store the digital voltage data Da for one code cycle in association with the VCSEL element that emits light.

Subsequently, in step S334, the processing unit 100a causes the inner product calculation unit 122 to refer to the first storage unit 116 and obtain the digital voltage data Da for one cycle of the predetermined code cycle stored in the first storage unit 116. is acquired, and the inner product operation of this digital voltage data Da and the reference data Rf input from the encoding unit 120 is executed. The inner product calculation unit 122 performs inner product calculation using all encoded data used for encoding, and causes the second accumulation unit 123 to accumulate the inner product calculation results.

Subsequently, in step S335, the processing unit 100a causes the estimating unit 118 to generate six digital voltage data corresponding to the VCSEL elements 11 to 16 stored in the second storage unit 123 using an estimation model created in advance. Based on Da, line-of-sight direction information Ed is calculated.

Subsequently, in step S<b>336 , the processing unit 100 a outputs the line-of-sight direction information Ed acquired by the estimation unit 118 to the external device via the output unit 119 .

Subsequently, in step S337, the processing unit 100a determines whether or not to end the processing. For example, the processing unit 100a can determine whether or not to end according to a user's end operation input signal or the like.

In step S337, when it is determined to end (step S337, Yes), the processing unit 100a ends the process. On the other hand, when it is determined not to end (step S337, No), the processing unit 100a performs the processes after step S331 again.

In this manner, the processing unit 100a can output line-of-sight direction information Ed based on the tilt of the eyeball 30. FIG.

<Action and effect of line-of-sight detection device 10a>
As described above, the line-of-sight detection device 10a has the light source modulation section 121 (control section) that controls the light emission of the VCSEL array 1 so as to emit the laser light L1 time-modulated by codes having orthogonality. Based on the electrical signal Se output from the photodiode 4, the processing unit 100a separates and detects a plurality of time-modulated laser beams L4. Codes having orthogonality are, for example, Hadamard codes.

The line-of-sight detection device 10a can reduce the processing time and processing load of the processing unit 100a by separating and detecting the laser beam L4. Further, even when the light intensity of the laser light L4 incident on the eyeball 30 is low, the line-of-sight detection device 10a can remove the influence of the background light by separating and detecting the laser light L4, thereby ensuring high line-of-sight detection accuracy. be able to.

Effects other than the above are the same as those of the first embodiment.

[Third Embodiment]
Next, a line-of-sight detection device 10b according to a third embodiment will be described.

<Functional Configuration Example of Processing Unit 100b>
FIG. 36 is a block diagram showing an example of the functional configuration of the processing unit 100b included in the line-of-sight detection device 10b according to this embodiment. As shown in FIG. 36, the processing unit 100a has a selection unit 111b and a switching unit 113b.

In this embodiment, a plurality of VCSEL elements included in the VCSEL array 1 are grouped into a plurality of groups, and the light source modulation unit 121 causes the VCSEL array 1 to emit light from the VCSEL elements of the plurality of groups so that each group emits light. Control light emission. As a result, the line-of-sight detection device 10 b reduces the number of VCSEL elements that emit light in parallel, and reduces the amount of laser light L that enters the eyeball 30 .

Here, a group is a collection of a plurality of VCSEL elements included in the VCSEL array 1 . For example, if VCSEL elements 11 through 16 are grouped into two groups, the first group includes VCSEL elements 11, 13 and 15 and the second group includes VCSEL elements 12, 14 and 16. including. Also, if the VCSEL elements 11 to 16 are grouped into three groups, each of the three groups includes two VCSEL elements. Note that the number of VCSEL elements forming this group may be one or more and five or less.

Encoded data may be prepared for the number of VCSEL elements included in each group. Since VCSEL elements belonging to different groups do not need to be detected separately, they may be assigned the same encoded data.

The number of groups in the line-of-sight detection device 10b and the groups to which the VCSEL elements 11 to 16 belong are predetermined.

The selection unit 111b selects a group to emit light from among the groups of the VCSEL elements 11 to 16, and outputs information indicating the selected group to the switching unit 113b.

The switching unit 113b switches connection between the light source modulation unit 121 and each VCSEL element so that the light source modulation unit 121 outputs the drive signal Dr to the selected group. After the digital voltage data Da is stored in the first storage unit 116, the switching unit 113b can switch to output the drive signal Dr to the VCSEL elements of another group that have not yet finished emitting light.

<Example of Processing by Processing Unit 100b>
FIG. 37 is a flowchart showing an example of processing by the processing unit 100b. The processing unit 100b starts the processing of FIG. 37, for example, in response to a start operation input signal by the user.

First, in step S351, the processing unit 100b causes the selection unit 111b to select which group of VCSEL elements to emit light from among the groups of VCSEL elements, and transmits information indicating the selected group to the switching unit 113b. Output.

Subsequently, in step S352, the processing unit 100b causes the encoding unit 120 to generate a plurality of encoded data, assigns the encoded data to each VCSEL element of the VCSEL array 1 via the light source modulation unit 121, The reference data Rf is output to the calculation unit 122 .

Subsequently, in step S<b>353 , the processing unit 100 b outputs the drive signal Dr time-modulated based on the encoded data to the VCSEL array 1 by the light source modulation unit 121 . The VCSEL element to which the drive signal Dr is applied emits laser light L1 according to the drive signal Dr. After passing through the lens 2 and the concave mirror 3, the laser beam L1 is reflected by the eyeball 30 and enters the photodiode 4 as the laser beam L4. The photodiode 4 outputs an electric signal Se corresponding to the light intensity of the received laser light L4 to the processing section 100b.

Subsequently, in step S354, the processing unit 100b causes the I/V conversion unit 114 to convert the electric signal Se from the photodiode 4 into an analog voltage signal, and then the A/D conversion unit 115 to convert it into digital voltage data Da. do. The processing unit 100a causes the first storage unit 116 to store the digital voltage data Da for one code cycle in association with the VCSEL element that emits light.

Subsequently, in step S355, the processing unit 100b causes the inner product calculation unit 122 to refer to the first accumulation unit 116 and obtain the digital voltage data Da for one period of the predetermined code period accumulated in the first accumulation unit 116. is acquired, and the inner product operation of this digital voltage data Da and the reference data Rf input from the encoding unit 120 is executed. The inner product calculation unit 122 performs inner product calculation using all encoded data used for encoding, and causes the second accumulation unit 123 to accumulate the inner product calculation results.

Subsequently, in step S356, the processing unit 100b causes the determination unit 117 to determine whether or not all the VCSEL elements 11 to 16 have been caused to emit light.

When it is determined in step S356 that all the VCSEL elements are not caused to emit light (step S356, No), the processing section 100b performs the processing from step S351 onward again. In this case, the processing unit 100b selects the group to which the VCSEL elements that have not emitted light yet belong by the selection unit 111b, and causes the VCSEL elements belonging to the group to emit light.

On the other hand, if it is determined in step S356 that all the VCSEL elements are caused to emit light (step S356, YES), in step S357 the processing unit 100b causes the estimation unit 118 to use an estimation model created in advance. Then, based on the six digital voltage data Da corresponding to the VCSEL elements 11 to 16 accumulated in the second accumulation unit 123, the line-of-sight direction information Ed is calculated.

Subsequently, in step S<b>358 , the processing unit 100 b outputs the line-of-sight direction information Ed acquired by the estimation unit 118 to the external device via the output unit 119 .

Subsequently, in step S359, the processing unit 100b determines whether or not to end the processing. For example, the processing unit 100b can determine whether or not to end according to a user's end operation input signal or the like.

In step S359, when it is determined to end (step S359, Yes), the processing unit 100b ends the process. On the other hand, when it is determined not to end (step S359, No), the processing unit 100b performs the processes after step S351 again.

In this manner, the processing unit 100b can output line-of-sight direction information Ed based on the tilt of the eyeball 30. FIG.

<Action and effect of line-of-sight detection device 10b>
As described above, in the present embodiment, a plurality of VCSEL elements included in the VCSEL array 1 are grouped into a plurality of groups, and the light source modulation unit 121 controls the VCSEL elements of the plurality of groups to emit light for each group. Secondly, light emission by the VCSEL array 1 is controlled.

For example, if the number of VCSEL elements to emit light is large, the amount of laser light L1 incident on the eyeball 30 from the VCSEL array 1 increases, which may be undesirable from the standpoint of safety. Since the line-of-sight detection device 10b causes the plurality of VCSEL elements included in the VCSEL array 1 to emit light for each group, the number of VCSEL elements that emit light in parallel can be reduced, and the amount of laser light L incident on the eyeball 30 can be reduced. Thereby, in this embodiment, the line-of-sight detection device 10b with excellent safety can be provided.

Functions and effects other than the above are the same as those of the first and second embodiments.

[Fourth Embodiment]
Next, a retinal projection display device 60 according to a fourth embodiment will be described. FIG. 38 is a diagram illustrating an example of the configuration of the retinal projection display device 60. As shown in FIG.

As shown in FIG. 38, the retinal projection display device 60 includes an RGB (Red, Green, Blue) laser light source 61, a scanning mirror 62, a plane mirror 63, a half mirror 64, an image generator 65, and a line-of-sight detector. a device 10;

The RGB laser light source 61 temporally modulates and outputs three colors of RGB laser light. The scanning mirror 62 two-dimensionally scans the light from the RGB laser light source 61 . The scanning mirror 62 is a MEMS mirror or the like. However, the present invention is not limited to this, as long as it has a reflecting portion for scanning light, such as a polygon mirror or a galvanomirror. MEMS mirrors are advantageous in terms of miniaturization and weight reduction. Incidentally, the driving method of the MEMS mirror may be any of an electrostatic method, a piezoelectric method, an electromagnetic method, and the like.

The plane mirror 63 reflects the scanning light from the scanning mirror 62 toward the half mirror 64 . Half mirror 64 transmits part of the incident light and reflects part of it toward eyeball 30 . The half mirror 64 has a concave curved shape, focuses the reflected light near the cornea 31 of the eyeball 30 , and forms an image at the position of the retina 33 . Thereby, an image formed by the scanning light is projected onto the retina 33 . Light 61 a indicated by a dashed line in the drawing represents light forming an image on the retina 33 . In the half mirror 64, the light amount of reflected light and transmitted light does not necessarily have to be one to one.

The line-of-sight detection device 10 transmits a feedback signal of the tilt of the eyeball 30 , that is, the line-of-sight direction to the image generator 65 .

The image generator 65 has a deflection angle control function of the scanning mirror 62 and a light emission control function of the RGB laser light source 61 . In addition, the image generator 65 receives a feedback signal of the line-of-sight direction from the line-of-sight detection device 10 . The line-of-sight direction feedback signal corresponds to the line-of-sight direction information Ed. The deflection angle of the scanning mirror 62 and the light emission of the RGB laser light source 61 are controlled according to the line-of-sight direction detected by the line-of-sight detection device 10 to rewrite the projection angle of the image or the content of the image. As a result, it is possible to form an image on the retina 33 that tracks changes in the line-of-sight direction (eye tracking).

In this embodiment, an example of the retinal projection display device 60 as a head-mounted display, which is a wearable terminal, is shown. It may be a device (a head-mounted display device) that is indirectly worn on a person's head via a member. Further, a binocular retinal projection display device having a pair of retinal projection display devices 60 for left and right eyes may be used.

Further, in this embodiment, the retinal projection display device 60 includes the line-of-sight detection device 10, but the retinal projection display device 60 may include the line-of-sight detection device 10a or 10b.

Although examples of embodiments of the present invention have been described above, the present invention is not limited to such specific embodiments, and various modifications can be made within the scope of the invention described in the scope of the claims. Transformation and change are possible.

For example, the line-of-sight detection device 10 can also be used in a virtual reality (VR) display device. The human visual field is classified into a high-resolution central visual field and a low-resolution peripheral visual field. A technology that uses this property to reduce the rendering load of a virtual reality image by increasing the resolution of the central portion of the virtual reality image while reducing the resolution of the peripheral portion is called fovea rendering. In particular, when combined with a line-of-sight detection device, it is possible to dynamically change the high-resolution area according to the user's line of sight. Real image can be recognized. Therefore, a virtual reality display device having the line-of-sight detection device 10 can implement fovea rendering technology with a simple configuration. As a result, a lightweight and low power consumption virtual reality display device can be realized, and the user can enjoy virtual reality images for a long time without feeling burdened.

The target object is not limited to the eyeball 30, and the present embodiment can be applied to any target object that is a three-dimensional object having curvature. For example, in the above-described embodiments, the device for detecting the tilt of the eyeball 30 is shown as an example of an optical device, but the present invention is not limited to this. For example, an optical device may be mounted on a robot hand or the like to detect the inclination of the robot hand as an example of the object.

It can also be applied to an optometry apparatus having a function of detecting the tilt of the eyeball and the position of the pupil (cornea). The optometry device refers to a device capable of performing various tests such as visual acuity test, eye refractive power test, intraocular pressure test, eye axial length test, and the like. The optometry apparatus is an apparatus capable of examining the eyeballs without contact, and includes a support for supporting the subject's face, an optometry window, and a display that keeps the direction of the subject's eyeballs (direction of the line of sight) constant during the optometry. It has a display unit for performing measurement, a control unit, and a measurement unit. In order to improve the measurement accuracy of the measurement unit, it is required to gaze at one point without moving the eyeball (line of sight). . At this time, the eyeball tilt position detection device of the present embodiment can be used when detecting the eyeball tilt position. The eyeball tilt position detection device is arranged on the side of the measuring section so as not to interfere with the measurement. Eyeball tilt position (line of sight) information obtained by the eyeball tilt position detection device can be fed back to the control unit, and measurement can be performed according to the eyeball tilt position information.

The line-of-sight detection device 10 can also be used in an educational environment. For example, from the user's line-of-sight information during a written test, it is possible to know which test questions the user spent time on. Based on that information, the teacher can provide appropriate feedback on the user's test results. By providing a line-of-sight detection device with a simple configuration according to the present invention, the load on the user can be reduced. In addition, since line-of-sight detection is possible in a wearable form, it is possible to obtain the above-described effects for test sheets and teaching materials on paper media.

The line-of-sight detection device 10 can also be employed as a user state estimation device that estimates the user's state based on information about the tilt of the eyeball, the position of the pupil (cornea), or the line-of-sight direction. As the user state estimation device, for example, there is a fatigue level estimation device for estimating the mental fatigue level of the user. The configuration of the present embodiment includes a line-of-sight detection device 10 and a fatigue degree estimation unit that estimates a degree of mental fatigue based on information on the line-of-sight direction detected by the line-of-sight detection device. As a method of estimating the user's mental fatigue level in the fatigue level estimation unit, for example, non-patent literature (Tseng, V.WS., Valliappan, N., Ramachandran, V. et al. Digital biomarker of mental fatigue. npj Digit Med. 4, 47 (2021)). According to this method, it is possible to estimate the degree of mental fatigue by performing a task of following the trajectory of an object displayed on the monitor with the eye for several minutes and measuring the movement of the line of sight at that time. A fatigue degree estimation device having the line-of-sight detection device 10 can estimate a user's mental fatigue degree with a simple configuration. Further, the fatigue level estimation device according to the present embodiment may have notification means for notifying the user of information for prompting him/her to take a break, for example, based on the estimated mental fatigue level.

Other user state estimation devices include, for example, attention level estimation devices that estimate a user's level of attention. The configuration of this embodiment includes a line-of-sight detection device 10 and an attention level estimation unit that estimates the user's level of attention based on information on the line-of-sight direction detected by the line-of-sight detection device. As a method of estimating the user's attention level in the attention level estimation unit, for example, there is a method of detecting minute eyeball vibrations called microsaccades and estimating the user's attention level based on the occurrence frequency. (Pastukhov A, Braun J. Rare but precious: microsaccades are highly informative about attentional allocation. Vision Res. 2010 Jun 11;50(12):1173-84.) Among the microtremors (small vibrations of the eyeball with an amplitude of about ±3.0 degrees that occur when a person is gazing at an object), it is a relatively large-amplitude and high-speed movement, and correlates with human attention. is known to exist. Since the line-of-sight detection device 10 can measure the inclination of the eyeball at high speed and with high accuracy, it can detect microsaccades with high precision compared to conventional line-of-sight detection devices.
Therefore, the attention level estimation device having the line-of-sight detection device 10 can estimate the user's attention level with a simple configuration.

Further, the injection degree estimation device according to the present embodiment can be employed in a driving support system. The configuration of the driving system of this embodiment includes the attention level estimation device described above and an operation control section that controls the movement of the moving object based on the attention level estimated by the attention level estimation device. For example, when the user's attention level estimated by the attention level estimation device is lower than a reference, the operation control unit controls the operation mode of the moving object, for example, the vehicle, so as to switch from the manual operation mode to the automatic operation mode. A driving assistance system having the line-of-sight detection device 10 can perform driving assistance with a simple configuration.

In addition, the injection degree estimation device according to the present embodiment can also be used to evaluate publications such as educational materials and instruction manuals. For example, it is possible to evaluate which part of the teaching material the user pays attention to and concentrates on using the gaze estimation device. The results can be used to improve the content and layout of teaching materials. By providing a line-of-sight detection device with a simple configuration according to the present invention, the load on the user can be reduced. In addition, since line-of-sight detection is possible in a wearable form, it is possible to obtain the above-described effects with respect to paper-medium teaching materials.

Further, in these embodiments, one piece of information regarding the tilt of the eyeball, the pupil position (cornea), or the line-of-sight direction detected by the line-of-sight detection device 10 is used by two or more image generation units and estimation units. There may be. By adopting such a configuration, it is possible to realize a simple configuration and downsizing.
For example, while using information about the eyeball tilt, pupil position (cornea), or line-of-sight direction detected by the line-of-sight detection device 10 as a feedback signal to the image generation unit of the retinal projection display device, the degree of fatigue of the fatigue level estimation device It may be used for fatigue level estimation in the estimation unit.
At this time, the image generation unit and the fatigue level estimation unit may be provided in the same information processing unit, or may be provided separately.

Also, the embodiment includes a tilt detection method for a three-dimensional object. For example, the tilt detection method of a three-dimensional object is a tilt detection method using a tilt detection device, wherein the tilt detection device emits light from a plurality of light emitting units, light is emitted from the plurality of light emitting units by light receiving means, and the object light reflected by an object is received, an electric signal based on the light intensity of the received light is output, and an output means detects a tilt of the object based on the electric signal output from the light receiving means. and the light-receiving means receives a plurality of lights emitted in parallel from the plurality of light-emitting portions and reflected by the object. Such a three-dimensional object tilt detection method can provide the same effect as the line-of-sight detection device described above.

Embodiments also include gaze detection methods. For example, the line-of-sight detection method is a tilt detection method using a line-of-sight detection device, wherein the line-of-sight detection device emits light from a plurality of light emitting units, emits light from the plurality of light emitting units by light receiving means, and is reflected by an object. outputting an electric signal based on the light intensity of the received light, outputting a detection result of the inclination of the object based on the electric signal output from the light receiving means, The light-receiving means receives a plurality of lights emitted from the plurality of light-emitting portions and reflected by the object. With such a line-of-sight detection method, the same effects as those of the line-of-sight detection device described above can be obtained.

In addition, all numbers such as ordinal numbers and numbers used above are examples for specifically describing the technology of the present invention, and the present invention is not limited to the numbers shown. Moreover, the connection relationship between the components is an example for specifically describing the technology of the present invention, and the connection relationship for realizing the function of the present invention is not limited to this.

Also, each function of the embodiments described above can be realized by one or more processing circuits. Here, the "processing circuit" in this specification means a processor programmed by software to perform each function, such as a processor implemented by an electronic circuit, or a processor designed to perform each function described above. devices such as ASICs (Application Specific Integrated Circuits), DSPs (digital signal processors), FPGAs (field programmable gate arrays) and conventional circuit modules.

Aspects of the present invention are, for example, as follows.
<1> a plurality of light emitting units, light receiving means for receiving the light emitted by the plurality of light emitting units and reflected by an object, and outputting an electrical signal based on the light intensity of the received light; and output means for outputting a detection result of the tilt of the object based on the output electrical signal, wherein the light receiving means detects the tilt of the object in a state where the tilt of the object is substantially the same. The tilt detection device receives a plurality of lights emitted from a light emitting part and reflected by the object.
<2> The tilt detection device according to <1>, wherein the plurality of light emitting units emit a plurality of lights in parallel.
<3> The plurality of light emitting portions are provided side by side along a predetermined direction along the light emitting surfaces of the plurality of light emitting portions, and the light emitting portions adjacent to each other in the plurality of light emitting portions are aligned along the predetermined direction. The tilt detection device according to <1> or <2>, wherein the intervals are equal.
<4> Any one of <1> to <3> above, wherein the plurality of light emitting units are arranged along two different directions along the light emitting surfaces of the plurality of light emitting units. 2. The tilt detection device according to claim 1.
<5> Let w be the width of the light-receiving surface of the light-receiving means, d be the distance between adjacent light beam centers on the light-receiving surface of the light-receiving means, and h be the beam radius of the light on the light-receiving surface of the light-receiving means. |h−w/2|≤d≤h+w/2 that satisfies the following equation when
The tilt detection device according to any one of <1> to <4>.
<6> The plurality of light emitting portions are provided side by side along a predetermined direction along the light emitting surfaces of the plurality of light emitting portions, and the light emitting portions adjacent to each other in the plurality of light emitting portions are aligned along the predetermined direction. The tilt detection device according to any one of <1> to <5>, wherein the length of the interval is shorter than the length of the light receiving surface of the light receiving means along the predetermined direction.
<7> The tilt detection device according to any one of <1> to <6>, wherein the light reflected by the object spreads as it approaches the light receiving means.
<8> Any one of <1> to <7>, wherein the beam diameter of the light reflected by the object is longer than the length of the light receiving surface along the predetermined direction on the light receiving surface of the light receiving means. 1. A tilt detection device according to one.
<9> A control unit for controlling light emission by the plurality of light emitting units so as to emit light time-modulated by codes having orthogonality, wherein the output unit responds to the electrical signal output from the light receiving unit. The tilt detection device according to any one of <1> to <8>, wherein the plurality of time-modulated lights are separated and detected based on the above.
<10> The tilt detection device according to <9>, wherein the code is a Hadamard code.
<11> The plurality of light emitting units are grouped into a plurality of groups, and the control unit controls light emission by the plurality of light emitting units such that the light emitting units in the plurality of groups emit light for each group. , the tilt detection device according to <9> or <10>.
<12> A line-of-sight detection device including the tilt detection device according to any one of <1> to <11>, and detecting a line of sight based on a detection result of an eyeball tilt detected by the tilt detection device. be.
<13> A retinal projection display device including the line-of-sight detection device according to <12>.
<14> A head-mounted display device including the line-of-sight detection device according to <12>.
<15> An eye examination apparatus including the line-of-sight detection apparatus according to <12>.
<16> A virtual reality display device including the line-of-sight detection device according to <12>.
<17> A user state estimation device, comprising: the line-of-sight detection device according to <12>; .
<18> The user state estimation device according to <17>, wherein the estimating unit estimates the state of the user based on the frequency of occurrence of minute vibrations of the user's eyeballs.
<19> The user state estimation device according to <17> or <18>, wherein the state of the user includes at least one of fatigue level and caution level of the user.
<20> The user state estimation device according to any one of <17> to <19>; and a driving support system.
<21> A tilt detection method using a tilt detection device, wherein the tilt detection device emits light from a plurality of light emitting units, and a light receiving unit receives light emitted from the plurality of light emitting units and reflected by an object. and outputting an electric signal based on the light intensity of the received light, outputting a detection result of the inclination of the object based on the electric signal output from the light receiving means by the output means, and the light receiving means and receiving a plurality of lights emitted from the plurality of light emitting units and reflected by the object in a state in which the object has substantially the same inclination.
<22> A line-of-sight detection method by a line-of-sight detection device, wherein the line-of-sight detection device emits light from a plurality of light emitting units, and a light receiving unit receives light emitted by the plurality of light emitting units and reflected by an object. and outputting an electric signal based on the light intensity of the received light, outputting a detection result of the inclination of the object based on the electric signal output from the light receiving means by the output means, and the light receiving means and receiving a plurality of lights emitted from the plurality of light-emitting units and reflected by the object in a state where the inclination of the object is substantially the same.

1, 1a VCSEL array (an example of multiple light emitting units)
2 lens 3 concave mirror 4 photodiode (an example of light receiving means)
41 Light-receiving surfaces 411 to 416, 411a to 417a Spot 10 Line-of-sight detection device (an example of a tilt detection device)
11 to 16, 11a to 16a VCSEL elements 1A, 11A to 16A light emitting surface 30 eyeball (an example of an object)
31 cornea 100 processing unit 101 CPU
102 ROMs
103 RAM
104 SSD
105 light source drive circuit 106 I/V conversion circuit 107 A/D conversion circuit 108 input/output I/F
111 selection unit 112 light source driving unit 113 switching unit 114 I/V conversion unit 115 A/D conversion unit 116 first accumulation unit 117 determination unit 118 estimation unit 119 output unit 120 encoding unit 121 light source modulation unit 122 inner product calculation unit 123 2 storage unit D interval e overlap region w width φ spread angle Da digital voltage data Dr drive signal Ed line-of-sight direction information Se electrical signals L, L1, L2, L3, L4, L41, L41a, L42 laser light Q converging position Rf Reference data

U.S. Patent No. 10213105

Claims (22)

a plurality of light emitting units;
a light-receiving means for receiving light emitted by the plurality of light-emitting portions and reflected by an object, and for outputting an electrical signal based on the light intensity of the received light;
output means for outputting a detection result of the inclination of the object based on the electrical signal output from the light receiving means;
The tilt detection device, wherein the light receiving unit receives a plurality of lights emitted from the plurality of light emitting units and reflected by the object when the object is tilted substantially at the same angle. 2. The tilt detection device according to claim 1, wherein said plurality of light emitting units emit a plurality of lights in parallel. The plurality of light emitting units are provided side by side along a predetermined direction along the light emitting surface of the plurality of light emitting units,
3. The tilt detection device according to claim 1, wherein the distances along the predetermined direction between adjacent light-emitting portions in the plurality of light-emitting portions are all equal. 3. The tilt detection device according to claim 1, wherein said plurality of light emitting portions are provided side by side along two different directions along the light emitting surfaces of said plurality of light emitting portions. Let w be the width of the light receiving surface of the light receiving means,
Let d be the distance between the beam centers of adjacent light beams on the light receiving surface of the light receiving means,
|h−w/2|≦d≦h+w/2 satisfying the following equation, where h is the beam radius of the light on the light receiving surface of the light receiving means.
The tilt detection device according to claim 1 or 2. The plurality of light emitting units are provided side by side along a predetermined direction along the light emitting surface of the plurality of light emitting units,
3. The method according to claim 1, wherein a length of an interval along said predetermined direction between adjacent light emitting portions in said plurality of light emitting portions is shorter than a length along said predetermined direction of a light receiving surface of said light receiving means. Tilt detection device as described. 3. The tilt detection device according to claim 1, wherein the light reflected by said object spreads as it approaches said light receiving means. 3. The tilt detection device according to claim 1, wherein the beam diameter of the light reflected by the object is longer on the light receiving surface of the light receiving means than the length along the predetermined direction of the light receiving surface. a control unit for controlling light emission by the plurality of light emitting units so as to emit light time-modulated by codes having orthogonality;
3. The tilt detection device according to claim 1, wherein said output means separates and detects a plurality of said time-modulated lights based on said electrical signal output from said light receiving means. 10. The tilt detection device according to claim 9, wherein the code is a Hadamard code. The plurality of light emitting units are grouped into a plurality of groups,
10. The tilt detection device according to claim 9, wherein said control unit controls light emission by said plurality of light emitting units so that said light emitting units in said plurality of groups emit light for each group. Having the tilt detection device according to claim 1 or claim 2,
A line-of-sight detection device that detects a line of sight based on a detection result of an inclination of an eyeball by the tilt detection device. A retinal projection display device comprising the line-of-sight detection device according to claim 12 . A head-mounted display device comprising the line-of-sight detection device according to claim 12 . An eye examination apparatus comprising the line-of-sight detection apparatus according to claim 12 . A virtual reality display device comprising the line-of-sight detection device according to claim 12 . A line-of-sight detection device according to claim 12;
A user state estimating device, comprising: a state estimating unit for estimating a state of a user based on information on the line-of-sight direction detected by the line-of-sight detection device. The user state estimation device according to claim 17, wherein the state estimation unit estimates the state of the user based on the frequency of occurrence of minute vibrations of the user's eyeballs. 18. The user state estimation device according to claim 17, wherein the state of the user includes at least one of fatigue level and caution level of the user. A user state estimation device according to claim 17;
A driving support system comprising: an operation control unit that controls an operation of a mobile object driven by the user based on the state estimated by the user state estimation device. A tilt detection method using a tilt detection device,
The tilt detection device is
It emits light by a plurality of light emitting parts,
the light receiving means receives the light emitted by the plurality of light emitting units and reflected by the object, and outputs an electrical signal based on the light intensity of the received light;
outputting a detection result of the tilt of the object based on the electrical signal output from the light receiving means,
The tilt detection method of a three-dimensional object, wherein the light receiving means receives a plurality of lights emitted from the plurality of light emitting units and reflected by the target when the tilt of the target is substantially the same. A line-of-sight detection method by a line-of-sight detection device,
The line-of-sight detection device is
It emits light by a plurality of light emitting parts,
the light receiving means receives the light emitted by the plurality of light emitting units and reflected by the object, and outputs an electrical signal based on the light intensity of the received light;
outputting a detection result of the tilt of the object based on the electrical signal output from the light receiving means,
The line-of-sight detection method, wherein the light receiving unit receives a plurality of lights emitted from the plurality of light emitting units and reflected by the object in a state where the inclination of the object is substantially the same.

Images