WO2025129583A1

WO2025129583A1 - Image projection device

Info

Publication number: WO2025129583A1
Application number: PCT/CN2023/140751
Authority: WO
Inventors: Yoji Okazaki
Original assignee: Guangdong Oppo Mobile Telecommunications Corp Ltd
Current assignee: Guangdong Oppo Mobile Telecommunications Corp Ltd
Priority date: 2023-12-21
Filing date: 2023-12-21
Publication date: 2025-06-26
Anticipated expiration: 2026-06-21
Also published as: CN121219621A

Abstract

According to the present disclosure, an image projection device includes: a light source; a scanner; a plurality of reflective mirrors including at least a first reflective mirror and a second reflective mirror; a line-of-sight direction detector; a control circuit; and a drive circuit, wherein the first reflective mirror is larger than a pupil diameter of the user and smaller than the second reflective mirror, and the drive circuit drives the light source with a pulse having a pulse width shorter than a pulse width determined by a modulation speed required in one pixel.

Description

IMAGE PROJECTION DEVICE

FIELD

The present disclosure relates to an image projection device, and more specifically, to an image projection device that is driven with low power consumption.

BACKGROUND

In recent years, Augmented Reality (AR) glasses, which are head-mounted display-type augmented reality wearable terminals, are being developed as one image projection device.

Image projection devices such as the AR glasses are required to be small, light, stylish and driven with low power consumption.

The image projection devices project an image on a retina by scanning a laser beam, which is emitted from a semiconductor laser diode (LD) element, with a scanning mirror, and irradiating the scanned laser beam from a projection optics to the user’s retina.

However, there is room for further improvement in conventional image projection devices about reducing the power consumption.

SUMMARY

The present disclosure aims to solve at least one of the technical problems mentioned above. Accordingly, the present disclosure needs to provide an image projection device.

According to the present disclosure, an image projection device includes:

a light source that emits a laser beam;

a scanner that scans the laser beam emitted from the light source;

a plurality of reflective mirrors, including at least a first reflective mirror that reflects the laser beam scanned by the scanner toward a retina of a user and is capable of changing a reflecting direction of the laser beam, and a second reflective mirror that reflects the laser beam reflected by the first reflective mirror toward the retina to project an image on the retina;

a line-of-sight direction detector that detects a line-of-sight direction (in other words, pupil position) of the user;

a control circuit that controls the reflecting direction of the laser beam reflected by the first reflective mirror, such that the image is projected from the line-of-sight direction (in other words, through pupil position) , which is detected by the line-of-sight direction detector, to the retina; and

a drive circuit that drives the light source with a pulse,

wherein the first reflective mirror is larger than a pupil diameter of the user and smaller than the second reflective mirror, and

the drive circuit drives the light source with the pulse having a pulse width shorter than a pulse width determined by a modulation speed required in one pixel.

BRIEF DESCRIPTION OF THE DRAWINGS

These and/or other aspects and advantages of embodiments of the present disclosure will become apparent and more readily appreciated from the following descriptions made with reference to the drawings, in which:

FIG. 1 is a diagram illustrating a configuration example of an image projection device according to a first embodiment of the present disclosure;

FIG. 2 is a diagram illustrating a configuration example of a light source according to the first embodiment of the present disclosure;

FIG. 3A is a diagram illustrating a current-optical output characteristics of a first laser element according to the first embodiment of the present disclosure;

FIG. 3B is a diagram illustrating a current-optical output characteristics of a second laser element according to the first embodiment of the present disclosure;

FIG. 3C is a diagram illustrating a current-optical output characteristics of a third laser element according to the first embodiment of the present disclosure;

FIG. 4 is a diagram illustrating a drive current of the laser elements according to the first embodiment of the present disclosure;

FIG. 5 is a diagram illustrating a configuration example of a scanner according to the first embodiment of the present disclosure;

FIG. 6 is a diagram illustrating a scanning method of the scanner according to the first embodiment of the present disclosure;

FIG. 7 is a diagram illustrating a large size tilt mirror module according to the first embodiment of the present disclosure;

FIG. 8 is a diagram illustrating a small size tilt mirror module according to the first embodiment of the present disclosure;

FIG. 9A is a diagram illustrating a power consumption and weight of the tilt mirror module according to the first embodiment of the present disclosure;

FIG. 9B is a diagram illustrating a relationship between a diameter of a MEMS mirror and a diameter of a tilt mirror according to the first embodiment of the present disclosure;

FIG. 10 is a diagram illustrating an operation of a line-of-sight direction detector according to the first embodiment of the present disclosure;

FIG. 11 is a diagram illustrating an intermittent driving of a global-shutter type image sensor according to the first embodiment of the present disclosure;

FIG. 12 is a diagram illustrating a foveated rendering according to the first embodiment of the present disclosure;

FIG. 13 is a diagram illustrating an operation of the tilt mirror module according to the first embodiment of the present disclosure;

FIG. 14A is a diagram illustrating a driving condition of the first laser element (R) according to an example of the first embodiment of the present disclosure;

FIG. 14B is a diagram illustrating a driving condition of the second laser element (G) according to the example of the first embodiment of the present disclosure;

FIG. 14C is a diagram illustrating a driving condition of the third laser element (B) according to the example of the first embodiment of the present disclosure;

FIG. 14D is a diagram illustrating a power consumption in response to the driving condition of the laser elements (RGB) according to the example of the first embodiment of the present disclosure;

FIG. 15A is a diagram illustrating a driving condition of the first laser element (R) according to a comparative example;

FIG. 15B is a diagram illustrating a driving condition of the second laser element (G) according to the comparative example;

FIG. 15C is a diagram illustrating a driving condition of the third laser element (B) according to the comparative example;

FIG. 15D is a diagram illustrating power consumption in accordance with the driving conditions of the laser elements (RGB) according to the comparative example;

FIG. 16 is a diagram illustrating the light source according to a first modification of the first embodiment of the present disclosure;

FIG. 17 is a diagram illustrating the scanner according to a second modification of the first embodiment of the present disclosure;

FIG. 18 is a diagram illustrating the light source according to a third modification of the first embodiment of the present disclosure;

FIG. 19 is a diagram illustrating the light source according to a fourth modification of the first embodiment of the present disclosure;

FIG. 20 is a diagram illustrating the light source according to a fifth modification of the first embodiment of the present disclosure;

FIG. 21 is a diagram illustrating a configuration example of the light source according to a second embodiment of the present disclosure;

FIG. 22A is a diagram illustrating a current-optical output characteristics and a current-voltage characteristics of the first laser element according to the second embodiment of the present disclosure;

FIG. 22B is a diagram illustrating a current-optical output characteristics and a current-voltage characteristics of the second laser element according to the second embodiment of the present disclosure;

FIG. 22C is a diagram illustrating a current-optical output characteristics and a current-voltage characteristics of the third laser element according to the second embodiment of the present disclosure;

FIG. 23A is a diagram illustrating a driving condition of the first laser element (R) according to a first example of the second embodiment of the present disclosure;

FIG. 23B is a diagram illustrating a driving condition of the second laser element (G) according to the first example of the second embodiment of the present disclosure;

FIG. 23C is a diagram illustrating a driving condition of the third laser element (B) according to the first example of the second embodiment of the present disclosure;

FIG. 23D is a diagram illustrating power consumption in accordance with the driving conditions of the laser elements (RGB) according to the first example of the second embodiment of the present disclosure;

FIG. 24A is a diagram illustrating a driving condition of the first laser element (R) according to a second example of the second embodiment of the present disclosure;

FIG. 24B is a diagram illustrating a driving condition of the second laser element (G) according to the second example of the second embodiment of the present disclosure;

FIG. 24C is a diagram illustrating a driving condition of the third laser element (B) according to the second example of the second embodiment of the present disclosure;

FIG. 24D is a diagram illustrating power consumption in accordance with the driving conditions of the laser elements (RGB) according to the second example of the second embodiment of the present disclosure;

FIG. 25 is a diagram illustrating the light source according to a first modification of the second embodiment of the present disclosure;

FIG. 26 is a diagram illustrating the light source according to a second modification of the second embodiment of the present disclosure;

FIG. 27 is a diagram illustrating another example of the light source according to the second modification of the second embodiment of the present disclosure;

FIG. 28 is a diagram illustrating an operation of the line-of-sight direction detector according to a third embodiment of the present disclosure;

FIG. 29 is a diagram illustrating an event rate (EPS: Event per Second) of an event sensor according to the third embodiment of the present disclosure;

FIG. 30A is a diagram illustrating a driving condition of the first laser element (R) according to a first example of a third embodiment of the present disclosure;

FIG. 30B is a diagram illustrating a driving condition of the second laser element (G) according to the first example of the third embodiment of the present disclosure;

FIG. 30C is a diagram illustrating a driving condition of the third laser element (B) according to the first example of the third embodiment of the present disclosure;

FIG. 30D is a diagram illustrating power consumption in accordance with the driving conditions of the laser elements (RGB) according to the first example of the third embodiment of the present disclosure;

FIG. 31A is a diagram illustrating a driving condition of the first laser element (R) according to a second example of the third embodiment of the present disclosure;

FIG. 31B is a diagram illustrating a driving condition of the second laser element (G) according to the second example of the third embodiment of the present disclosure;

FIG. 31C is a diagram illustrating a driving condition of the third laser element (B) according to the second example of the third embodiment of the present disclosure;

FIG. 31D is a diagram illustrating power consumption in accordance with the driving conditions of the laser elements (RGB) according to the second example of the third embodiment of the present disclosure;

FIG. 32A is a diagram illustrating a driving condition of the first laser element (R) according to an example of a fourth embodiment of the present disclosure;

FIG. 32B is a diagram illustrating a driving condition of the second laser element (G) according to the example of the fourth embodiment of the present disclosure;

FIG. 32C is a diagram illustrating a driving condition of the third laser element (B) according to the example of the fourth embodiment of the present disclosure;

FIG. 32D is a diagram illustrating power consumption in accordance with the driving condition of the laser elements (RGB) according to the example of the fourth embodiment of the present disclosure;

FIG. 33 is a diagram illustrating a weight of the image projection device according to an example of the present disclosure;

FIG. 34 is a diagram illustrating the weight of the image projection device according to another example of the present disclosure, and

FIG. 35 is a diagram illustrating the weight of the image projection device according to another example of the present disclosure.

DETAILED DESCRIPTION

Embodiments of the present disclosure will be described in detail and examples of the embodiments will be illustrated in the accompanying drawings. The same or similar elements and the elements having same or similar functions are denoted by like reference numerals throughout the descriptions. The embodiments described herein with reference to the drawings are explanatory, which aim to illustrate the present disclosure, but shall not be construed to limit the present disclosure.

[First Embodiment]

The image projection device according to the first embodiment of the present disclosure will be described below. The image projection device according to the first embodiment is a retina projection type head-mounted display that projects an image onto a user's retina. More specifically, the image projection device according to the first embodiment is an eye-tracking retina projection type head-mounted display that uses an edge-emitting laser as a light source and projects images onto the user's retina by tracking the user's line-of-sight direction. The line-of-sight direction may also be referred to as a pupil position. As shown in FIG. 1, the image projection device 1 according to the first embodiment includes a light source 11, a collimating optical system 12, a scanner 14, a projection optical system 15, a line-of-sight direction detector 16, an feedback circuit 17, which is an example of a control circuit, and a digital image controller (DIC) 18.

The light source 11 emits a laser beam to the scanner 14. As shown in FIG. 2, the light source 11 includes a first laser element 111, a second laser element 112, a third laser element 113, and a drive circuit 117.

In the first embodiment, each of the first laser element 111, the second laser element 112, and the third laser element 113 is an edge-emitting laser diode. That is, as shown in FIG. 2, each of the laser elements 111, 112, and 113 has a p-type semiconductor layer 101, an n-type semiconductor layer 102, and an active layer 103 between the p-type semiconductor layer 101 and the n-type semiconductor layer 102.

As shown in FIG. 2, the first laser element 111 emits a laser beam Lr with a wavelength from 610 nm to 680 nm (i.e., red) . The wavelength of the laser beam Lr may be 650 nm. The second laser element 112 emits a laser beam Lg with a wavelength from 500 nm to 530 nm (i.e., green) . The wavelength of the laser beam Lg may be 520 nm. The third laser element 113 emits a laser beam Lb with a wavelength from 430 nm to 470 nm (i.e., blue) . The wavelength of the laser beam Lb may be 450 nm.

The active layer 103 of the first laser element 111 (i.e., the first active layer) contains aluminum gallium indium phosphorus (AlGaInP) . Each of the active layer 103 (i.e., second active layer) of the second laser element 112 and the active layer 103 (i.e., third active layer) of the third laser element 113 contain indium gallium nitride (InGaN) .

Each laser element 111, 112, 113 is connected to the drive circuit 117 via wiring. The drive circuit 117 is connected to the digital image controller 18. The drive circuit drives the light source 11. Specifically, the drive circuit 117 applies a drive current between the p-type semiconductor layer 101 and the n-type semiconductor layer 102 under control of the digital image controller 18. When a drive current higher than an oscillation threshold current is applied, each of the laser elements 111, 112, and 113 emits a laser beam Lr, Lg, and Lb generated by recombination of holes and electrons in the active layer 103 from the edge surface (cleft surface) of the active layer 103.

As shown in FIG. 3A, in the first embodiment, the oscillation threshold current of the first laser element 111 is 30 mA or less at a case temperature of 0℃ to 60℃. More specifically, the oscillation threshold current of the first laser element 111 is 5 mA or more and 25 mA or less at the case temperature of 0℃ to 60℃ and is 15 mA or less at the case temperature of 25℃.

As shown in FIG. 3B, in the first embodiment, the oscillation threshold current of the second laser element 112 is 35 mA or less at the case temperature of 0℃ to 60℃. More specifically, the oscillation threshold current of the second laser element 112 is 15 mA or more and 32 mA or less at the case temperature of 0℃ to 60℃ and is 25 mA or less at the case temperature of 25℃.

As shown in FIG. 3C, in the first embodiment, the oscillation threshold current of the third laser element 113 is 15 mA or less at the case temperature of 0℃ to 60℃. More specifically, the oscillation threshold current of the third laser element 113 is 5 mA or more and 12 mA or less at the case temperature of 0℃ to 60℃ and is 10 mA or less at the case temperature of 25℃. These laser elements 111, 112, and 113 have an extremely high front surface reflectivity (almost 100%) and a low slope efficiency, which reduces the oscillation threshold current.

As shown in FIG. 4, the digital image controller 18 sends a signal (i.e., data) to the drive circuit 117 so that the drive circuit 117 applies a pulse current, which has a pulse width W and a pulse height H as a drive current, to each laser element 111, 112, 113. In other words, the drive circuit 117 drives each laser element 111, 112, 113 with a pulse. Specifically, the drive circuit 117 drives each laser element 111, 112, 113 with the pulse having a pulse width determined by a modulation speed required in one pixel. The pulse width determined by the modulation speed required in one pixel may be 11.2 nsec. More specifically, the drive circuit 117 drives each laser element 111, 112, 113 with the pulse having a pulse width W of 2.2nsec or less. More preferably, the drive circuit 117 drives each laser element 111, 112, 113 with the pulse having the pulse width W of 1.1 nsec or less.

In other words, the drive circuit 117 drives each laser element 111, 112, 113 having a sufficiently low oscillation threshold current with the pulse having a sufficiently short pulse width W. Accordingly, the laser light output is very small when the image is projected onto the retina, and each of the laser elements 111, 112, and 113 has a low oscillation threshold current, which makes the drive current even smaller. As a result, it is possible to reduce power consumption while reducing jitter. Furthermore, since a current value generated to drive the laser elements 111, 112, and 113 is small, capacitance loss or the like in the drive circuit 117 is decreased. As a result, the power consumption of the drive circuit 117 itself can be significantly reduced.

The collimating optical system 12 shown in FIG. 1 is disposed on an emitting side of the laser beams Lr, Lg, and Lb with respect to the light source 11. The collimating optical system 12 collimates and emits the laser beams Lr, Lg, and Lb emitted from the light source 11. In the example shown in FIG. 1, collimating optical system 12 includes a single lens. The configuration of the collimating optical system 12 is not limited to that shown in FIG. 1. For example, the collimating optical system 12 may include multiple lenses.

The scanner 14 is disposed on the emitting side of the laser beams Lr, Lg, and Lb with respect to the collimating optical system 12. The scanner 14 scans the laser beams Lr, Lg, and Lb emitted from the light source 11. Specifically, the scanner 14 scans the laser beams Lr, Lg, and Lb that have been collimated by the collimating optical system 12.

As shown in FIG. 5, the scanner 14 has, for example, a MEMS (Micro Electric Mechanical System) mirror 141 and an ASIC (Application Specific Integrated Circuit) (not shown) that drives the MEMS mirror 141. The MEMS mirror 141 has a first rotation axis A1 that rotates the MEMS mirror 141 to scan the laser beams Lr, Lg, and Lb in a first uniaxial direction. The MEMS mirror 141 also has a second rotation axis A2 that rotates the MEMS mirror 141 so as to scan the laser beams Lr, Lg, and Lb in a second uniaxial direction perpendicular to the first uniaxial direction. The ASIC applies a drive current to the piezoelectric element (not shown) provided around the MEMS mirror 141 to rotationally drive the MEMS mirror 141. The digital image controller 18 controls the rotational drive of the MEMS mirror 141 by the ASIC.

The MEMS mirror 141 scans the laser beams Lr, Lg, and Lb by rotating around the first rotation axis A1. The MEMS mirror 141 also scans the laser beams Lr, Lg, and Lb by rotating around the second rotation axis A2.

Since one MEMS mirror 141 scans laser beams Lr, Lg, and Lb in both the first and second uniaxial directions that are orthogonal to each other, there is no need to provide separate MEMS mirrors for scanning laser beams Lr, Lg, and Lb in the main scanning direction and in the sub-scanning direction. This allows the image projection system 1 to be downsized.

The MEMS mirror 141 rotates around the first rotation axis A1 using resonance. The MEMS mirror 141 also uses resonance to rotate around the second rotation axis A2. Since the MEMS mirror 141 uses resonance to rotate in both the first and second uniaxial directions that are orthogonal to each other, it is possible to reduce the power consumption required to drive the scanner 14.

The scanner 14 scans the laser beams Lr, Lg, and Lb emitted from the light source 11 using a Lissajous scan method. By scanning the laser beams Lr, Lg, and Lb in the Lissajous scan method, high-speed moving image projection with few afterimages can be performed on the user's retina. For example, even at a frame rate of about 60 Hz, a sufficiently fast moving image projection can be performed. In addition, by scanning the laser beams Lr, Lg, and Lb using the Lissajous scan method, power consumption of the MEMS mirror 141 can be reduced compared to scanning the laser beams Lr, Lg, and Lb using the raster scan method.

The relationship between the scanner 14 and the resolution N of the image is expressed by the following equation.

N = (θopt *D) / (1.1 *λ) (1)

In Equation (1) , θopt is an optical swing angle for scanning the laser beam emitted from the light source 11 as shown in FIG. 1 (hereafter the same applies) . D is a diameter of the MEMS mirror 141 (hereafter the same applies) . λ is the wavelength of the laser beam (hereafter the same applies) . Therefore, by increasing θopt *D, an image with higher resolution can be projected onto the retina. For example, by increasing the resonance frequency in the first and second uniaxial directions that are orthogonal to each other, and by increasing θopt *D in equation (1) , it is possible to increase the resolution. It is known that the highest image quality, such as the image quality in the case of dot pattern display, can be obtained when the GCD (greatest common divisor) of the resonance frequencies in the first and second uniaxial directions, that are orthogonal to each other, is made close to 1.

The projection optical system 15 is an optical system that projects an image by irradiating laser beams Lr, Lg, and Lb, which are scanned by the scanner 14, onto the retina of the user. As shown in FIG. 1, the projection optical system 15 has a first reflective mirror 151, a relay lens 152, and a second reflective mirror 153. In the example shown in FIG. 1, the projection optical system 15 includes two reflective mirrors, the first reflective mirror 151 and the second reflective mirror 153, as the plurality of reflective mirrors. However, the projection optical system 15 is not limited to such configuration, and may further include at least one reflective mirror (e.g., a fixed mirror) as long as the projection optical system 15 includes the first reflective mirror 151 and the second reflective mirror 153.

The first reflective mirror 151 is disposed on the emitting side of the laser beams Lr, Lg, and Lb with respect to the scanner 14. The first reflective mirror 151 reflects the laser beams Lr, Lg, and Lb scanned by the scanner 14 toward the retina of the user. The first reflective mirror 151 may change the reflecting direction of the laser beams Lr, Lg, and Lb. Specifically, the laser beams Lr, Lg, and Lb emitted from the scanner 14 are incident on the first reflective mirror 151. The first reflective mirror 151 reflects the laser beams Lr, Lg, and Lb toward the relay lens 152. The first reflective mirror 151 rotates under a control of the feedback circuit 17. The operation of the feedback circuit 17 may be controlled by a host device (i.e., CPU) .

As the first reflective mirror 151 rotates, reflecting directions of the laser beams Lr, Lg, and Lb are changed. The feedback circuit 17 controls the rotation of the first reflective mirror 151 based on the user's line-of-sight direction detected by the line-of-sight direction detector 16. Specifically, the feedback circuit 17 controls the rotation of the first reflective mirror 151 so that the laser beams Lr, Lg, and Lb reflected by the first reflective mirror 151 are projected from the line-of-sight direction (i.e., through the pupil position) detected by the line-of-sight direction detector 16 to the user's retina. In other words, the feedback circuit 17 controls the reflecting direction of the laser beams Lr, Lg, and Lb reflected by the first reflective mirror 151 such that images are projected from the line-of-sight direction, which is detected by the line-of-sight direction detector 16, to the user’s retina.

As shown in FIG. 7, the first reflective mirror 151 may be composed of a tilt mirror module. The tilt mirror module may include a driving system for the tilt mirror. Hereinafter, the first reflective mirror 151 in the configuration shown in FIG. 7 may be referred to as a large size first reflective mirror 151 or a large size tilt mirror module.

Specifically, the large size first reflective mirror 151 includes a movable body 1541, a gimbal mechanism 1542, a magnetic drive mechanism 1543, a fixed body 1544, a Hall sensor 1545, and a tilt mirror module drive mechanism 1546.

The movable body 1541 includes a tilt mirror 1541a. The tilt mirror 1541a has, for example, a circular shape in a planar view (i.e., viewed from a direction perpendicular to two rotation axes A3, A4 of the tilt mirror 1541a) . In the example shown in FIG. 7, the tilt mirror 1541a is formed on a plate-shaped mirror body 1540, which is separated from the movable body 1541, and is bonded to the movable body 1541 together with the mirror body 1540. A cross-section of the tilt mirror 1541a is curved in an arc toward the mirror body 1541. The movable body 1541 is supported by the fixed body 1544 via the gimbal mechanism 1542 so that the tilt mirror 1541a is rotatable (i.e., swingable) around two rotation axes A3, A4 perpendicular to each other.

The gimbal mechanism 1542 includes, for example, a metal plate spring. The gimbal mechanism 1542 supports the movable body 1541 with respect to the fixed body 1544 so that the tilt mirror 1541a swings about the two rotation axes A3, A4.

The magnetic drive mechanism 1543 generates a magnetic drive force between the movable body 1541 and the fixed body 1544. The magnetic drive force displaces the movable body 1541 with respect to the fixed body 1544. In the example shown in FIG. 7, the magnetic drive mechanism 1543 includes a coil 1543a and a magnet 1543b. The coil 1543a is provided on the movable body 1541. The magnet 1543b is provided on the fixed body 1544 opposite the coil 1543a.

The fixed body 1544 is provided so that an angle of the fixed body 1544 can be changed around the rotation axes A3 and A4. By changing the angle of the fixed body 1544 around the two rotation axes A3 and A4, the tilt mirror 1541a irradiates a laser beam onto the user's retina based on the user's line-of-sight direction and enables the image to be projected.

The Hall sensor 1545 outputs a detection signal in response to a tilt of the movable body 1541. The Hall sensor 1545 is provided on the movable body 1541 at a position near the magnet 1543b. In the example shown in FIG. 7, the Hall sensor 1545 is provided inside the coil 1543a. The detection signal output from the Hall sensor 1545 is input to the feedback circuit 17. The feedback circuit 17 detects the tilt of the movable body 1541 based on the input detection signal. The feedback circuit 17 controls the generation of the magnetic drive force by the magnetic drive mechanism 1543 based on the detected tilt of the movable body 1541. This allows the angle of the tilt mirror 1541a to be controlled quickly and precisely.

The tilt mirror module drive mechanism 1546 is a drive mechanism that changes the angle of the tilt mirror 1541a based on the line-of-sight direction detected by the line-of-sight direction detector 16. For example, the tilt mirror module drive mechanism 1546 includes a magnetic circuit, a piezoelectric element, and a motor.

By constructing the first reflective mirror 151 with the tilt mirror module including the gimbal mechanism 1542 as shown in FIG. 7, it is possible to rotate the first reflective mirror 151 at high speed, following changes in the line-of-sight direction.

The first reflective mirror 151 may be constructed with the tilt mirror module shown in FIG. 8, instead of the tilt mirror module shown in FIG. 7. Hereinafter, the first reflective mirror 151 with the structure shown in FIG. 8 may be referred to as a small size first reflective mirror 151 or a small size tilt mirror module.

Similarly, the small size first reflective mirror 151 shown in FIG. 8 includes the gimbal mechanism (not shown) , is provided with the tilt mirror 1541a and four pairs of magnets 1548, and includes the movable body 1541 having an approximately right triangular shape in a side view, forming a pyramid structure (i.e., a square pyramid shape with four slopes) in stereoscopic view. The four pairs of magnets 1548 are provided and distributed on the four slopes of the movable body 1541. In the example shown in FIG. 8, the tilt mirror 1541a is formed on a plate-shaped mirror body 1540, which is separated from the movable body 1541, and is bonded to the movable body 1541 together with the mirror body 1540.

Further, the small size first reflective mirror 151 includes four pairs of coils 1549 disposed opposite each of the four pairs of magnets 1548. The coils 1549 are provided on a movable body not shown. Further, the small size first reflective mirror 151 includes a tilt mirror module drive mechanism 1546 connected to the plurality of coils 1549. The tilt mirror module drive mechanism 1546 selectively generates an electromagnetic force between the plurality of coils 1549 and the plurality of magnets 1548 based on the line-of-sight direction detected by the line-of-sight direction detector 16. Generating the electromagnetic force by the gimbal mechanism allows the movable body 1541 to rotate (i.e., swing) around a rotation axis A5 which is perpendicular to the central line of the tilt mirror 1541a (dash-dotted line in FIG. 8) .

Further, the movable body 1541 may also rotate, by the gimbal mechanism, around a rotation axis A6 which is perpendicular to both the central line of the tilt mirror 1541a and the rotation axis A5. Rotating the movable body 1541 allows the tilt mirror 1541a to change the reflecting directions of the laser beams Lr, Lg, Lb in the two-axis direction of the rotation axes A5, A6 in accordance with the detected line-of-sight direction.

The overall module of the small size first reflective mirror 151 shown in FIG. 8 is formed smaller than the large size first reflective mirror 151 shown in FIG. 7. However, as shown in FIG. 9A, the small size first reflective mirror 151 has lower magnetic efficiency and large power consumption compared to the large size first reflective mirror 151.

Hereinafter, unless otherwise noted, the large size first reflective mirror 151 shown in FIG. 7 and the small size first reflective mirror 151 shown in FIG. 8 are referred to as the first reflective mirror 151 without distinction.

The first reflective mirror 151 is larger than a pupil diameter of the user and smaller than the second reflective mirror 153. Specifically, the tilt mirror 1541a of the first reflective mirror 151 is larger than the pupil diameter of the user and smaller than the second reflective mirror 153. By having the first reflective mirror 151 smaller than the second reflective mirror 153, the power consumption of the first reflective mirror 151 may be reduced.

In order to reduce the power consumption, it is preferred that the diameter of the first reflective mirror 151 is 8mm or less. Specifically, the diameter of the tilt mirror 1541a of the first reflective mirror 151 is preferably 8mm or less. In the example shown in FIG. 9A, the diameter is 8mm both for the large and small size first reflective mirrors 151. When the diameter of the first reflective mirror 151 is 8mm, a distance between the coil and magnet composing the tilt mirror module may be halved and a Lorentz force applied between the coil and magnet may be 4 times greater compared to when the diameter of the first reflective mirror 151 is 16mm. Accordingly, when the diameter of the first reflective mirror is 8mm, the power consumption of the first reflective mirror 151 may be reduced to 1/4 compared to when the diameter of the first reflective mirror 151 is 16mm.

In order to properly reduce the diameter of the first reflective mirror 151 to 8mm or less, it is effective to increase the diameter of the MEMS mirror 141 under a predetermined swing angle of the MEMS mirror 141. The diameter of the MEMS mirror 141 may be 2mm or more. For example, as shown in FIG. 9B, when the diameter of the MEMS mirror 141 is 2mm, the diameter of the first reflective mirror 151 may be decreased more easily than when the diameter of the MEMS mirror 141 is 1mm.

Specifically, since the resolution is proportional to θopt *D, when the diameter D of the MEMS mirror 141 is 2mm, the swing angle θopt of the MEMS mirror 141 may be reduced to 1/2 compared to when the diameter D of the MEMS mirror 141 is 1mm under the same resolution. By having the swing angle θopt of the MEMS mirror 141 be 1/2, as can be easily seen from FIG. 1, the diameter of the first reflective mirror 151 may be reduced by 1/2. More specifically, in the example shown in FIG. 9B, when the diameter of the MEMS mirror 141 is 1mm and the resolution is 1280 *720, the swing angle θopt of the MEMS mirror 141 is 36°, θmech is 18° (±9°) , and the diameter of the first reflective mirror 151 (i.e., tilt mirror 1541a) is 16mm.

In contrast, when the diameter of the MEMS mirror 141 is 2mm, the swing angle of the MEMS mirror 141 may be halved to 9.0° (±4.5°) at the same resolution (1280 *720) , and the diameter of the first reflective mirror 151 (i.e., tilt mirror 1541a) may be halved to 8mm.

In the example shown in FIG. 9A, the large size first reflective mirror 151 with diameter of 8mm has an average power consumption of 4mW to drive the tilt mirror 1541a, a power consumption of 4mW for the drive IC that drives the tilt mirror 1541a, a total power consumption of 8mW, and a weight of 2.75g. On the other hand, the small size first reflective mirror 151 with diameter of 8mm has an average power consumption of 11mW to drive the tilt mirror 1541a, a power consumption of 11mWfor the drive IC, a total power consumption of 22mW, and a weight of 1.63g. Accordingly, even if both diameters are 8mm, the large size first reflective mirror 151 has a lower power consumption compared to the small size first reflective mirror 151.

In general, the MEMS mirror 141 using the Lissajous scan method may have a smaller swing angle than the MEMS mirror 141 using the Raster scan method. This is because the MEMS mirror 141 using the Lissajous scan method may use biaxial resonance mode, is easy to be vacuum sealed, which will be described later, and is easy to increase the diameter. Accordingly, in the first embodiment, adopting the MEMS mirror 141 using the Lissajous scan method as described allows decreasing the diameter of the first reflective mirror 151 more easily. Decreasing the diameter of the first reflective mirror 151 is effective not only for reducing the power consumption, but also for realizing high-frequency characteristics of the first reflective mirror 151. In other words, decreasing the diameter of the first reflective mirror 151 allows the first reflective mirror 151 to stably maintain the predetermined tilt angle even when subjected to external vibration. In particular, when a vacuum sealed MEMS mirror 141 (see FIG. 17) is used, the MEMS mirror 141 can be made larger with lower power consumption because it is not affected by air resistance. Therefore, it is preferable to use the vacuum sealed MEMS mirror 141.

It is also possible to setof the first reflective mirror 151 to 8mm or less by disposing the MEMS mirror 141 and the tilt mirror as close as possible.

As shown in FIG. 1, the second reflective mirror 153 is disposed on the reflection side of the laser beams Lr, Lg, Lb with respect to the first reflective mirror 151. Laser beams Lr, Lg, Lb reflected by the first reflective mirror 151 are incident on the second reflective mirror 153 through the relay lens 152. The second reflective mirror 153 reflects the incident laser beams Lr, Lg, Lb toward the user’s retina. The second reflective mirror 153 is, for example, a free-form mirror. The second reflective mirror 153 may be a micro-structured diffractive optical element, a holographic optical element, or a meta material optical element.

The line-of-sight direction detector 16 includes an irradiator 161 and a line-of-sight position detector 162. The irradiator 161 irradiates an infrared beam to the user’s eye. For low power consumption, the irradiator 161 is configured with an infrared laser element such as a vertical cavity surface emitting laser (VCSEL) or an infrared LED. The line-of-sight position detector 162 receives reflected light of the infrared beam that was irradiated from the irradiator 161 to the user’s eye. The line-of-sight position detector 162 outputs a detection signal of the user’s line-of-sight direction in accordance with the amount of received reflected light. The line-of-sight position detector 162 outputs the detection signal to the feedback circuit 17 and the digital image controller 18. The line-of-sight direction detector 16 is composed of an eye-tracking camera.

More specifically, in the first embodiment, the line-of-sight direction detector 16 is configured with the eye-tracking camera including a global shutter type image sensor. The global shutter type image sensor is provided in the line-of-sight position detector 162. The global shutter type image sensor may be a CMOS sensor. As shown in FIG. 10, in the global shutter type image sensor, exposure by the image sensor and readout of electrons generated by photoelectric conversion caused by the exposure are performed for all lines of the image sensor at the same timing. Such global shutter type image sensor is driven, for example, by a drive circuit provided in the line-of-sight position detector 162. Operation of the drive circuit of the image sensor may be controlled by the host device (i.e., CPU) . As shown in FIG. 10, since the global shutter type image sensor captures the image of the user’s eye for all lines at the same timing, an image of the user’s eye that is close to a perfect circle may be detected. Detecting the image of the user’s eye that is close to the perfect circle allows appropriate detection of the user’s line-of-sight direction. Further, reading all lines at the same timing allows near-zero power consumption during standby.

In contrast, if the line-of-sight direction detector 16 is configured by an eye-tracking camera including a rolling shutter type image sensor, as shown in FIG. 10, the exposure and readout of electrons by the image sensor are performed in order from the top row of the image sensor. As shown in FIG. 10, since the rolling shutter type image sensor captures the image of the user’s eye with a time lag between the lines, an image of the user’s eye that is distorted from a perfect circle is detected. Since the image of the user’s eye that is distorted from the perfect circle is detected, the user’s line-of-sight direction cannot be detected appropriately. In addition, since readout for each line is also performed with a time lag, the standby power consumption is also relatively large, resulting in high power consumption.

Therefore, in the first embodiment, using the global shutter type image sensor for the line-of-sight direction detector 16 allows appropriate detection of the user’s line-of-sight direction and performing eye-tracking with high precision.

Further, as shown in FIG. 11, the global shutter type image sensor may be intermittently driven so that a measurement (i.e., detection) and standby of the line-of-sight direction are repeated. Intermittent driving of the global shutter type image sensor may be performed by a drive circuit provided in the line-of-sight position detector 162. Intermittent driving of the global shutter type image sensor allows suppressing of heat generation, noise, and power consumption of the line-of-sight direction detector 16 (i.e., eye-tracking camera) . Since the heat generation and noise of the line-of-sight direction detector 16 can be suppressed and high-precision detection can be performed even at a reduced frame rate, the power consumption of the line-of-sight direction detector 16 can be reduced. Specifically, by driving the global shutter type image sensor intermittently at 30fps (i.e., detecting the line-of-sight direction per every 32.2 msec) , a power consumption on the image sensor side of the eye-tracking camera can be reduced to 11.8mW.

Further, a saccade, which is an eye movement in which humans shift the focus of the eye to a new point of gaze, typically occurs every 20 to 100msec. When the global shutter type image sensor is driven intermittently at 30fps, the detection of the line-of-sight direction occurs every 32.2 msec. In this case, it becomes possible to suppress the power consumption of the line-of-sight direction detector 16 and perform eye-tracking with high precision, since detection speed of the line-of-sight direction can be kept within the range of eye movement speed based on the saccade.

The light source 11 may emit the laser beams Lr, Lg, Lb following a foveated rendering shown in FIG. 12, in accordance with the detection result of the line-of-sight direction by the line-of-sight direction detector 16, under the control of the digital image controller 18. In other words, the light source 11 may emit the laser beams Lr, Lg, Lb such that the resolution of the image projected to a fovea of the user’s retina from the line-of-sight direction detected by the line-of-sight direction detector 16 increases and the resolution of the image projected to the retina around the fovea from directions other than the detected line-of-sight direction decreases.

In the example shown in FIG. 12, the resolution of a foveal area is the highest, the resolution of a blend area adjacent outside the foveal area is lower than the resolution of the foveal area, and the resolution of a peripheral area adjacent outside the blend area is the lowest. By emitting the laser beam Lr, Lg, Lb according to the foveated rendering, the resolution of the area which the user gazes at can be increased to improve visibility, and the resolution of the area which the user does not gaze at can be decreased to further reduce power consumption.

The feedback circuit 17 shown in FIG. 1 controls the projection optical system 15 based on the detected line-of-sight direction detected by the line-of-sight direction detector 16. Specifically, as shown in FIG. 13, the feedback circuit 17 rotates the first reflective mirror 151 according to a change in the line-of-sight direction detected by the line-of-sight direction detector 16. This allows the laser beam to be focused on a pupil and continue to project the image onto the retina regardless of the change in the line-of-sight direction. This allows the user to continue viewing the image even after moving the eyes.

By projecting the laser beam onto the retina by tracking the user's line-of-sight direction, an eye box can be widened. Further, since flux of the laser beam is narrower than a diameter of the observer’s pupil, focus adjustment can be eliminated. In addition, images with high resolution can be projected. Further, by using the RGB laser, color gamut of the image can be expanded. Moreover, since the image is projected directly to the use’s eye, color irregularities in the image can be eliminated, the color gamut can be expanded, and highly realistic images can be projected.

The feedback circuit 17 may control the rotation of the first reflective mirror 151 based on the detection results of the angular velocity sensor (not shown) and the line-of-sight direction detector 16. Specifically, although the feedback circuit 17 basically controls the first reflective mirror 151 based on the line-of-sight direction, when the angular velocity (i.e., vibration) of the image projection device 1 is detected by the angular velocity sensor, the feedback circuit 17 may control the rotation of the first reflective mirror 151 to offset the detected angular velocity. This allows the image projection device 1 to continue to project images onto the retina in accordance with the user's line-of-sight direction even when the image projection device 1 vibrates.

The digital image controller 18 controls the emission of laser beams Lr, Lg, and Lb by the light source 11. Specifically, the digital image controller 18 obtains image data of an image to be projected onto the user's retina from a memory or a host device (i.e., CPU) . The digital image controller 18 drives the first laser element 111, the second laser element 112, and the third laser element 113 through the drive circuit 117 according to pixel values indicated in the image data. As described above, the digital image controller 18 may control the emission of the laser beams Lr, Lg, Lb from each laser element 111, 112, 113 following the foveated rendering in accordance with the detection result of the line-of-sight direction by the line-of-sight direction detector 16. As described above, the digital image controller 18 performs pulse driving of each laser element 111, 112, 113 with a pulse width W of 2.2 nsec or less.

As described above, in the first embodiment, the first reflective mirror 151 is larger than the pupil diameter of the user and smaller than the second reflective mirror 153. Since the first reflective mirror 151 is smaller than the second reflective mirror 153, the power consumption of the first reflective mirror 151 can be reduced. Further, by using the MEMS mirror in which the mirror diameter is large and the swing angle is small, the diameter of the first reflective mirror 151 can be reduced to 8mm or less. Since the diameter of the first reflective mirror 151 can be reduced to 8mm or less, the power consumption of the first reflective mirror 151 can be further reduced, the first reflective mirror 151 (i.e., projection optical system 15) becomes small and light, and the response speed (i.e., high-frequency characteristics) of the first reflective mirror 151 can be improved. Further, reducing the diameter of the first reflective mirror 151 to be 8mm or less, as will be described in the examples below, the power consumption of the entire image projection device 1 can be reduced to less than 200mW and the weight of the entire image projection device 1 can be reduced to about 40g.

As a result, it is possible to realize a lightweight AR glasses for daily use that can output high quality and realistic images and achieve high luminance and low power consumption by projecting directly onto the retina.

Further, in the first embodiment, by using a magnetic circuit type tilt mirror module as shown in FIGS. 7 and 8 as the first reflective mirror 151, and by reducing the mirror diameter of the first reflective mirror 151 to 8 mm or less, the high-speed response can be further improved and can follow a shock with a gravitational acceleration of 5G.

Further, in the first embodiment, the feedback circuit 17 (i.e., control circuit) controls the reflecting direction of the laser beam reflected by the first reflective mirror 151, so that the image is projected from the line-of-sight direction of the user detected by the line-of-sight direction detector 16 to the user’s retina. This allows the user to continue viewing the image even after moving the eyes.

Further, in the first embodiment, by configuring the line-of-sight direction detector 16 with the eye-tracking camera equipped with the global shutter type image sensor, the line-of-sight direction can be properly detected to improve the visibility of the image.

Further, in the first embodiment, since the light source 11 emits the laser beams Lr, Lg, Lb following the foveated rendering, the visibility of the image can be improved, and the power consumption can be reduced.

Further, in the first embodiment, the drive circuit 117 drives each of the first laser element 111 that emits the laser beam Lr of a wavelength from 610 nm to 680 nm and has an oscillation threshold current of 15 mA or less, the second laser element 112 that emits the laser beam Lg of the wavelength from 500 nm to 530 nm and has the oscillation threshold current of 25 mA or less, and the third laser element 113 that emits the laser beam Lb of the wavelength from 430 nm to 470 nm and has the oscillation threshold current of 10 mA or less, with a pulse having the pulse width of 2.2 nsec or less.

Therefore, in the first embodiment, the pulse current value (i.e., peak current value) for the driving of the laser elements 111, 112, and 113 with the pulse can be sufficiently small and power consumption can be reduced by driving the laser elements 111, 112, and 113, which have good response with sufficiently small oscillation threshold current, with the pulse having a short pulse width. Since the pulse current value can be made sufficiently small, jitter and capacitance loss and the like of the drive circuit 117 can be reduced. This makes it possible to reduce power consumption while maintaining good image quality.

Next, an example showing the power consumption of the image projection device 1 according to the first embodiment will be described, comparing it with a comparative example.

[Example]

In the example of the first embodiment, each of the laser elements 111, 112, and 113 with the current-optical output characteristics shown in FIG. 3A to FIG. 3C was driven under four different driving conditions.

The four drive conditions are one type of continuous driving (CW: continuous wave) and three types of driving with the pulse. FIG. 14A shows the driving conditions for the first laser element 111 (R) . FIG. 14B shows the driving conditions for the second laser element 112 (G) . FIG. 14C shows the driving conditions for the third laser element 113 (B) . As shown in FIG. 14A to FIG. 14C, the specific parameters for the continuous driving and the driving with the pulse are different for each of the laser elements 111, 112, and 113.

In FIG. 14A to FIG. 14C, "LD Power (mW) " is the optical output of the laser elements 111, 112, and 113. "Iop" is the drive current of the laser elements 111, 112, and 113. “P.C. (CW: mW) ” is the power consumption corresponding to the continuous driving with the same pulse width as the pulse width of one pixel, not to the driving with a shorter pulse width in the pulse width of one pixel. The pulse width of one pixel is determined by a laser modulation speed (i.e., modulation frequency) required for projecting of one pixel. In FIG. 14A to FIG. 14C, the pulse width of one pixel is 11.2 nsec.

“Duty” is the duty ratio for obtaining the same amount of light when driving with the shorter pulse within the pulse width of one pixel. P.C. (Pulse: mW) " is the power consumption corresponding to performing the driving with the pulse. “Power Ratio (%) ” is the percentage of a ratio of “P.C. (Pulse: mW) ” to “P.C. (CW: mW) ” . "Pulse width (nsec) " is the pulse width set during the driving with the pulse. "Pulse number" is the value obtained by dividing the pulse width required for one pixel by the above "Pulse width (nsec) " , which is the maximum number of pulses per pixel. In order to reduce the power consumption of the laser elements 111, 112, and 113 to the maximum, it is desirable to use a set pulse width (nsec) and a pulse number of 1 to output the peak LD power that has an average LD power same as the LD power in CW.

Further, in the example of the first embodiment, the diameters of the small size tilt mirror module shown in FIG. 8 and the large size tilt mirror module shown in FIG. 7 were both 8mm. Also, in the example of the first embodiment, the global shutter type eye-tracking camera was intermittently driven at 30 fps.

In the first example of the first embodiment, the power consumption corresponding to the driving conditions of the laser elements 111, 112, and 113 was calculated by simulation. FIG. 14D shows the power consumption corresponding to driving conditions of the laser elements (RGB) . In FIG. 14D, "TPC (100%) " is the total power consumption of the laser elements 111, 112, and 113 when Pixel on is 100%when the pixels are lit in white. "TPC (30%) " is the total power consumption of the laser elements 111, 112, and 113 when Pixel on is 30%of normal video output. “DIC” is the power consumption required for image processing of the digital image controller 18. “MEMS+ASIC” is the power consumption of the scanner 14, that is, the MEMS mirror 141 and the ASIC.

"Laser module" is the power consumption of the laser elements 111, 112, 113 for emitting the laser beams Lr, Lg, and Lb. "Laser module" matches "TPC (30%) " . “LDD” is the power consumption of the drive circuit 117 used to control the pulse width, frequency, intensity, etc. of the drive signal of the laser elements 111, 112, 113. “Pulse generation loss” is power consumption that is additionally lost for generating drive pulses. More specifically, "Pulse generation loss" means the power consumption loss due to the wiring when the pulse current of the laser elements 111, 112, and 113 is applied and the power consumption loss due to the transistor when the current is turned on/off. In other words, the "pulse generation loss" is the power consumption that is lost in the drive circuit 117 and wiring, which is proportional to the power consumption (= TPC (30%) ) applied to the laser elements 111, 112, and 113, the "pulse generation loss" being apart from the power applied to the laser elements 111, 112, and 113. "sum" is the total power consumption of "DIC" , "MEMS+ASIC" , "Laser module" , "LDD" , and "Pulse generation line-of-sights" .

“Small size tilt mirror module” is the power consumption of the small size tilt mirror module shown in FIG. 8. “Large size tilt mirror module” is the power consumption of the large size tilt mirror module shown in FIG. 7. “Eye-tracking” is the power consumption of the eye-tracking camera. The “Eye-tracking” is 15.3 mW, which is the sum of the power consumption 11.8 mW of the image sensor (i.e., line-of-sight direction detector 162) and the power consumption 3.5 mW of the infrared sensor (i.e., irradiator 161) of the eye-tracking camera. “PC1” is the power consumption of the entire image projection device 1 using the small size tilt mirror module. “PC2” is the power consumption of the entire image projection device 1 using the large size tilt mirror module.

The meanings of the terms in the FIGs of the example of the first embodiment are the same in the subsequent examples and comparative examples.

According to the example of the first embodiment, the power consumption "PC2" can be reduced to 200 mW or less by driving the laser elements 111, 112, and 113 having the low oscillation threshold current with the pulse having the pulse width of 2.2 nsec or less. Further, in the example of the first embodiment, by driving the laser elements 111, 112, and 113 having the low oscillation threshold current with the pulse having the pulse width of 1.1 nsec, the both the power consumption "PC1" and "PC2" can be reduced to 200 mW or less.

In addition, in the example of the first embodiment, by reducing the diameter of the tilt mirror module to 8 mm, a ratio of the “Small size tilt mirror module” and the “Large size tilt mirror module” to the power consumption ( “PC1” or “PC2” ) of the entire image projection device 1 can be sufficiently small. Furthermore, in the example of the first embodiment, by driving the global shutter type eye-tracking camera intermittently, a ratio of the “Eye-tracking” to the power consumption of the entire image projection device 1 can be sufficiently small, while appropriately detecting the line-of-sight direction. In other words, in the example of the first embodiment, the “DIC” and the “MEMS+ASIC” becomes dominant over the power consumption of the entire image projection device 1.

[Comparative Example]

In a comparative example, unlike the example of the first embodiment, the diameter of the small size tilt mirror and the large size tilt mirror were 16 mm. Further, in the comparative example, unlike the example of the first embodiment, the rolling shutter type eye-tracking camera was driven at 120 fps. The first laser element 111 (R) was driven according to the driving conditions shown in FIG. 15A. The second laser element 112 (G) was driven according to the driving conditions shown in FIG. 15B. The third laser element 113 (B) was driven according to the driving conditions shown in FIG. 15C.

As shown in FIG. 15D, in the comparative example, since the diameter of the tilt mirror is 16mm, the “Small size tilt mirror module” and the “Large size tilt mirror module” are larger than in the example of the first embodiment. Further, in the comparative example, since the rolling shutter type eye-tracking camera is driven at 120 fps, it is not possible to perform high-precision eye-tracking and the “Eye-tracking” is larger than that of the example of the first embodiment. As a result, in the comparative example, the power consumptions “PC1” and “PC2” cannot be reduced to 200 mW or less even if the laser elements 111, 112, 113 are driven with the pulse having the pulse width of 1.1 nsec or less.

According to the example of the first embodiment described above, it was confirmed that the power consumption can be effectively reduced by using the tilt mirror of diameter 8 mm and the global shutter type eye-tracking camera.

Various modifications shown below may be applied to the image projection device 1 according to the first embodiment.

[First Modification]

In the embodiment described above, examples in which the laser elements 111, 112, 113 are single-beam elements are described.

In contrast, as shown in FIG. 16, in the first modification of the first embodiment, each of the laser elements 111, 112, and 113 is disposed in plurality adjacent to each other.

As a result, the plurality of the first laser elements 111 emit multibeams with a wavelength from 610nm to 680nm. The plurality of the second laser elements 112 emit the multibeams with a wavelength from 500nm to 530nm. The plurality of the third laser elements 113 emit the multibeams with a wavelength from 430nm to 470nm.

According to the first modification, by using the multibeams, the multibeams are displayed at the same time, and the resonance frequency of the MEMS can be reduced. By decreasing the resonant frequencies in the first and second uniaxial directions that are orthogonal to each other, θopt *D in equation (1) can be increased so as not to reach the physical limit of destruction of the MEMS mirror 141. Since θopt *D can be increased, the resolution of the image can be increased. For example, by increasing the number of beams in the multibeams, the resonance frequency can be further reduced according to the number of beams, and θopt *D can be increased by that amount, and images can be projected onto the user's retina with resolutions higher than FHD (Full High Definition) , that is, FHD, 2K, 4K, and 8K resolutions.

In addition, since θopt *D can be increased, images with good visibility can be projected onto the user's retina with a high FOV (Field of View) , such as, e.g., 60°.

[Second Modification]

As shown in FIG. 17, in the second modification of the first embodiment, the MEMS mirror 141 and the ASIC are vacuum sealed by a sealing member 143 provided above the MEMS mirror 141 and the ASIC.

According to the second modification of the first embodiment, by vacuum sealing the MEMS mirror 141, power consumption can be further reduced. For example, the power consumption of the MEMS mirror 141 and the ASIC can be reduced to less than 10 mW.

[Third Modification]

As shown in FIG. 18, in a third modification of the first embodiment, the light source 11 includes a collimating lens 118 disposed on the emitting side of the laser elements 111, 112, and 113 and a multiplexing optical system 119 disposed on the emitting side of the collimating lens 118. The collimating lens 118 collimates and emits the laser beams Lr, Lg, and Lb emitted from the laser elements 111, 112, and 113. The multiplexing optical system 119 multiplexes the laser beams Lr, Lg, and Lb emitted from the collimating lens 118 while reflecting them to the scanner 14 side. The collimating lens 118 may be a meta-lens bonded to the laser elements 111, 112, and 113. The multiplexing optical system 119 may be a dichroic mirror.

According to the third modification of the first embodiment, by including the multiplexing optical system 119, misalignment of the optical axes of the laser beams Lr, Lg, and Lb can be reduced, and thus high-quality images with less color shift and image flow can be projected.

[Fourth Modification]

As shown in FIG. 19, in a fourth modification of the first embodiment, the light source 11 multiplexes the laser beams Lr, Lg, Lb with a waveguide type optical multiplexer 134.

As shown in FIG. 19, the waveguide type optical multiplexer 134 includes waveguides 1341-1343 and an emission outlet 1344. The waveguides 1341-1343 propagate laser beams Lr, Lg, and Lb emitted from the first, second, and third laser elements 111, 112, and 113, respectively, and multiplex the laser beams Lr, Lg, and Lb in the multiplexing area 1345. The emission outlet 1344 emits the laser beams Lr, Lg, and Lb, which are multiplexed in the multiplexing area 1345, to the scanner 14 side.

According to the fourth modification of the first embodiment, since the waveguide type optical multiplexer 134 can reduce misalignment of the optical axes of the laser beams Lr, Lg, and Lb, it is possible to project high quality images with less color shift and image flow.

[Fifth Modification]

As shown in FIG. 20, in a fifth modification of the first embodiment, the light source 11 includes two waveguide type optical multiplexers 134a, 134b.

In other words, in the example shown in FIG. 20, the light source 11 includes the first laser element 111a, the second laser element 112a, the third laser element 113a, and the waveguide type optical multiplexer 134a of a first group. Also, the light source 11 includes the first laser element 111b, the second laser element 112b, the third laser element 113b, and the waveguide type optical multiplexer 134b of a second group.

The waveguide type optical multiplexer 134a of the first group multiplexes the laser beams Lr, Lg, and Lb emitted from each of the first, second, and third laser elements 111a, 112a, and 113a of the first group and multiplexes them in the multiplexing area 1345a of the waveguides 1341a to 1343a to emit them from the emission outlet 1344a.

The waveguide type optical multiplexer 134b of the second group multiplexes the laser beams Lr, Lg, and Lb emitted from each of the first, second, and third laser elements 111b, 112b, and 113b of the second group and multiplexes them in the multiplexing area 1345b of the waveguides 1341b to 1343b to emit them from the emission outlet 1344b.

The emission outlet 1344a of the waveguide type optical multiplexer 134a of the first group and the emission outlet 1344b of the waveguide type optical multiplexers 134b of the second group are arranged at a predetermined angle with respect to the first and second uniaxial directions of the MEMS mirror 141.

That is, in the fifth modification of the first embodiment, the light source 11 emits the multibeams from two emission outlets 1344a and 1344b. By emitting multibeams, the resonance frequency of the MEMS mirror 141 can be reduced, and θopt *D in the above equation (1) can be increased, so the resolution can be increased.

A distance between the emission outlets 1344a, 1344b is preferably less than 20 μm, more preferably less than 10 μm, and even more preferably less than 5 μm. Since the waveguide type coupler has no thermal interference, it is possible to reduce a spacing between the emission outlets 1344a, 1344b. By reducing the spacing between the emission outlets 1344a, 1344b, the light source 11 can emit a high-resolution multibeam with a small beam pitch.

To reduce the distance between the emission outlets 1344a, 1344b, the waveguides 1341a-1343a, 1341b-1343b of the waveguide type optical multiplexers 134a, 134b may have a steeper curve than the configuration in FIG. 18. Similar to the first embodiment, the output of the laser beams Lr, Lg, and Lb can be small because the image projection system 1 according to the second embodiment uses the retina projection method. Therefore, even if there is a line-of-sights of laser beams Lr, Lg, and Lb due to the steep curves of the waveguides 1341a-1343a, 1341b-1343b, an image with good visibility can be projected onto the user's retina. Even if the laser elements 111, 112, and 113 have a kink on the low-current side, by driving the laser elements 111, 112, and 113 at a high current and by causing a line-of-sights of laser beams Lr, Lg, and Lb in the waveguides 1341a to 1343a and 1341b to 1343b, the kink can be avoided and the laser beams Lr, Lg, and Lb with an appropriate output power can be projected onto the retina.

[Second Embodiment]

Next, the image projection device 1 according to a second embodiment will be described, focusing on the differences from the embodiments described above. In the embodiment described above, the examples in which the laser elements 111, 112, and 113 are edge-emitting laser diodes are described. In contrast, in the second embodiment, the laser elements 111, 112, and 113 are vertical cavity surface emitting lasers (VCSELs) that emit laser beams Lr, Lg, and Lb in a direction perpendicular to the semiconductor substrate. The emission direction of the laser beams Lr, Lg, and Lb shown in FIG. 21 is perpendicular to the semiconductor substrate (not shown) included in each of the laser elements 111, 112, and 113.

The vertical cavity surface emitting lasers are described, for example, in two references: "Kenichi Terao et al (2021) . PROCEEDINGS OF THE INTERNATIONAL DISPLAY WORKSHOPS, VOL. 28" and "Tatsushi Hamaguchi et al (2018) . Lateral optical confinement of GaN-based VCSEL using an atomically smooth monolithic curved mirror. Scientific Reports" . That is, the semiconductor substrate in each of the laser elements 111, 112, and 113 may be disposed between a first highly reflective distributed Bragg reflector (DBR) and a second high reflective distributed Bragg reflector with an active layer in between. Alternatively, the semiconductor substrate may be disposed outside of either the first highly reflective distributed Bragg reflector or the second highly reflective distributed Bragg reflector.

The vertical cavity surface emitting lasers with a highly reflective distributed Bragg reflector can effectively expand color gamut because the desired wavelength can be selected and there is no wavelength change. In addition, the desired oscillation threshold current can be set by adjusting material composition, reflectance, and shape of the highly reflective distributed Bragg reflector.

In the second embodiment, the first laser element 111 has the current-optical output (OP) and a current-voltage (V) characteristics shown in FIG. 22A. The second laser element 112 has the current-optical output and the current-voltage characteristics shown in FIG. 22B. The third laser element 113 has the current-optical output and the current-voltage characteristics shown in FIG. 22C.

The oscillation threshold current of the first laser element 111 is 1 mA or less. The oscillation threshold current of the second laser element 112 is 3 mA or less. The oscillation threshold current of the second laser element 112 may be 2mA or less, or further 1mA or less, by increasing a crystallinity of InGaN in the active layer or by using quantum dots. The oscillation threshold current of the third laser element 113 is 1 mA or less. Therefore, the oscillation threshold current is significantly reduced compared to that of the edge-emitting laser.

As shown in FIG. 21, each of the laser elements 111, 112, and 113 is disposed on semiconductor chips 114 corresponding to each of the laser elements 111, 112, and 113. The semiconductor chips 114 are disposed on the drive circuit 117. By disposing each semiconductor chips 114 corresponding to each of the laser elements 111, 112, and 113 on the drive circuit 117, wiring of the drive circuit 117 can be shortened. As a result, high frequency characteristics and high gradation characteristics can be achieved.

According to the second embodiment, the power consumption can be significantly reduced by using the vertical cavity surface emitting laser, in which the oscillation threshold current is much lower than that of the edge-emitting laser diode.

In addition, the power consumption can be further reduced because the capacitance loss, which depends on the drive current, can be reduced.

Since the laser elements 111, 112, and 113 can be driven with the low oscillation threshold current in the retinal projection system, where the required optical output is low, the peak current of the drive pulse can be reduced. Since the peak current can be reduced, it is possible to perform the pulse driving with the short pulse width of 1 nsec or less while reducing the jitter. This further reduces the power consumption.

Next, an example showing the power consumption of the image projection device 1 according to the second embodiment will be described.

[First Example]

In a first example of the second embodiment, each of the laser elements 111, 112, and 113 with the current-optical output characteristics shown in FIG. 22A to FIG. 22C was driven under four different driving conditions similar to the example of the first embodiment.

Specifically, the first laser element 111 was driven according to the driving conditions shown in FIG. 23A. The second laser element 112 was driven according to the driving conditions shown in FIG. 23B. The third laser element 113 was driven according to the driving conditions shown in FIG. 23C. This resulted in the power consumption shown in FIG. 23D.

As can be seen in FIG. 23D, in the first example of the second embodiment, the “Small size tilt mirror module” , the “Large size tilt mirror module” , and the “Eye-tracking” are the same as in the example of the first embodiment. On the other hand, in the first example of the second embodiment, by using the laser elements 111, 112, 113 having low oscillation threshold currents, the power consumption “sum” can be reduced compared to that of the example in the first embodiment.

As a result, in the first example of the second embodiment, the power consumptions “PC1” and “PC2” can be reduced compared to that of the example in the first embodiment. For example, in the first example of the second embodiment, by driving the laser elements 111, 112, 113 with the pulse having the pulse width of 1.1nsec, the power consumption “PC1” can be reduced to 190mW and the power consumption “PC2” can be reduced to 176mW. Further, in the first example of the second embodiment, the power consumptions “PC1” and “PC2” can be reduced to less than 200mW even during continuous driving (CW) .

[Second Example]

In a second example of the second embodiment, the first laser element 111 with the current-optical output characteristics shown in FIG. 22A was driven under driving conditions shown in FIG. 24A. The second laser element 112 with the current-optical output characteristics shown in FIG. 22B was driven under driving conditions shown in FIG. 24B. The third laser element 113 with the current-optical output characteristics shown in FIG. 22C was driven under driving conditions shown in FIG. 24C. This resulted in the power consumption shown in FIG. 24D.

As shown in FIG. 24D, in the second example of the second embodiment, by vacuum sealing the MEMS mirror 141 (see FIG. 17) and increasing an efficiency of a thin piezo film provided on the MEMS mirror 141, the power consumption “MEMS+ASIC” is reduced to 10mW. By reducing the power consumption “MEMS+ASIC” to 10mW, the power consumptions “PC1” and “PC2” in case of the driving with the pulse can be reduced to 150mW or less.

In the second example of the second embodiment, “DIC” and “LDD” becomes dominant over the power consumption of the entire image projection device 1.

Various modifications shown below may be applied to the image projection device 1 according to the second embodiment.

[First Modification]

As shown in FIG. 25, in the first modification of the second embodiment, the light source 11 includes two of each of the laser elements 111, 112, 113 having the current-optical output characteristics shown in FIG. 22A to FIG. 22C. In other words, the light source 11 emits the RGB multibeam in the first modification of the second embodiment.

According to the first modification of the second embodiment, high resolutions such as FHD, 2K, 4K, and 8K resolutions can be realized by using the multibeam of the vertical cavity surface emitting laser.

[Second Modification]

As shown in FIG. 26, in the second modification of the second embodiment, the light source 11 includes four first laser elements 111a-111d. The four first laser elements 111a-111d are disposed in a staggered manner adjacent to each other. The arrangement allows the distance P between the centers of the emitting portions E of the first laser elements 111a-111d to be as small as possible, even if the first laser elements 111a-111d have a spread around the emitting portion E. The staggered arrangement of the first laser elements 111a-111d also makes it possible to create an arbitrary pitch P. Therefore, by disposing a large number of first laser elements 111a-111d at a high density, the light source 11 can be further miniaturized.

Also, the resonance frequency of the MEMS mirror 141 can be reduced, and θopt *D in equation (1) can be further increased so as not to reach the physical limit of breakage of the MEMS mirror 141. Since θopt *D can be made larger, the image resolution and FOV can be further increased. For example, by increasing the number of beams in the multibeams, the resonance frequency can be further reduced according to the number of beams, and θopt *D can be increased by that amount, and images can be projected onto the user's retina at resolutions higher than FHD, that is, FHD, 2K, 4K, and 8K resolutions. Also, since θopt *D can be increased, an image with good visibility can be projected on the user's retina with a high FOV of 60°, for example.

The number of the laser elements is not limited to four, i.e., the number of the laser elements is arbitrary and may be composed of two or three, or five or more. Specifically, the plurality of first laser elements 111, the plurality of second laser elements 112, and the plurality of third laser elements 113 may be six laser elements disposed in a staggered manner, as shown in FIG. 27. According to the example shown in FIG. 27, the distance between the centers of the emitting portions E can be further reduced than in FIG. 26, so that the light source 11 can be made even smaller and the resolution can be increased.

As described above, in the second embodiment, by reducing the diameter of the first reflective mirror 151 to 8mm or less, the power consumption of the first reflective mirror 151 can be further reduced, the first reflective mirror 151 (i.e., projection optics 15) can be small and light, and the high-speed response (i.e., high-frequency characteristics) of the first reflective mirror 151 can be improved. Further, by using the VCSEL and the global shutter CMOS sensor, the power consumption of the entire image projection device 1 can be reduced to less than 200mW and weight of the entire image projection device 1 can be reduced to about 40g. In addition, the use of multi-beam VCSELs facilitates higher resolution.

[Third Embodiment]

Next, the image projection device 1 according to a third embodiment will be described, focusing on the differences from the embodiments described above. The example of the second embodiment was described for the image projection device 1 configured with the eye-tracking camera where the line-of-sight direction detector 16 includes the global shutter type image sensor.

On the other hand, as described in FIG. 28, the third embodiment differs from the second embodiment in that the line-of-sight direction detector 16 is configured with an eye-tracking camera including an event sensor (i.e., an event camera) .

As shown in FIG. 28, a conventional image sensor (i.e., a conventional camera) cannot detect the user’s line-of-sight direction (i.e., image of the pupil) during time between frames, even if the image sensor is operated at, for example, 10 fps. Therefore, if the user’s line-of-sight detection changes suddenly, the camera cannot follow the change in the line-of-sight detection, and thus it impossible to track the line-of-sight with high precision. In order to track the line-of-sight with high precision using conventional image sensors, the image sensor is required to be operated at, for example, 30 fps. However, when the image sensor is operated at 30 fps, the detection framerate cannot be decreased, and thus it is relatively difficult to reduce the power consumption.

On the other hand, according to the third embodiment, the event camera captures changes in a brightness of the subject as an event, and when the brightness of the subject has changed, the event camera detects only an area where the brightness has changed. Therefore, as shown in FIG. 28, the event camera can detect only the pupil that has moved only when a change in the line-of-sight, i.e., movement of the pupil occurs. This allows tracking the line-of-sight with high precision and reducing the power consumption.

For example, as shown in FIG. 29, when the event camera is operated at 100 EPS (Event per Second) , the event camera can detect 100 events (i.e., pupil movement) per second. Detecting 100 events per second means that the event camera can detect the line-of-sight with higher precision compared to when the above-mentioned global shutter type image sensor is operated at 30 fps. As shown in FIG. 29, the power consumption of the event camera operated at 100 EPS is 10mW. This value is smaller than 15.3mW, which is the power consumption of the global shutter type eye-tracking camera.

Accordingly, the third embodiment allows tracking the line-of-sight with higher precision and further reducing the power consumption.

Next, an example that shows the power consumption of the image projection device 1 according to the third embodiment will be described.

[First Example]

In a first example of the third embodiment, the first laser element 111 with the current-optical output characteristics shown in FIG. 22A was driven under driving conditions shown in FIG. 30A. The second laser element 112 with the current-optical output characteristics shown in FIG. 22B was driven under driving conditions shown in FIG. 30B. The third laser element 113 with the current-optical output characteristics shown in FIG. 22C was driven under driving conditions shown in FIG. 30C. This resulted in the power consumption shown in FIG. 30D.

As shown in FIG. 30D, in the first example of the third embodiment, the “Eye-tracking” can be reduced to 10mW by using the event camera. Further, in the first example of the third embodiment, the “DIC” can be reduced to 20mW by performing the foveated rendering. As a result, according to the first example of the third embodiment, the power consumptions “PC1” and “PC2” can be reduced compared to the embodiments described above. Specifically, in the first example of the third embodiment, the power consumptions “PC1” and “PC2” can be reduced to 104mW or less for both of the continuous driving and the driving with the pulse.

[Second Example]

In a second example of the third embodiment, the first laser element 111 with the current-optical output characteristics shown in FIG. 22A was driven under driving conditions shown in FIG. 31A. The second laser element 112 with the current-optical output characteristics shown in FIG. 22B was driven under driving conditions shown in FIG. 31B. The third laser element 113 with the current-optical output characteristics shown in FIG. 22C was driven under driving conditions shown in FIG. 31C. This resulted in the power consumption shown in FIG. 31D.

The second example of the third embodiment differs from the first example of the third embodiment in that it uses a drive circuit 117 optimally designed using 20nm semiconductor design. By using the drive circuit 117 optimally designed with a low-current drive transistor, the “LDD” can be reduced to 10mW, as shown in FIG. 31D. In particular, in the case of the VCSEL with a low current drive of several mA, close to 1 mA, the transistor itself is not used and the VCSEL can be driven only with an operational amplifier, and thus the power consumption for driving the transistor can be eliminated to enable 10 mW of power consumption. As a result, according to the second example of the third embodiment, the power consumption "PC1" and "PC2" can be significantly reduced to less than 100mW compared to the first example of the third embodiment.

In FIG. 31D, the dominant factors for the power consumption are the “DIC” and the “Small size tilt mirror” which is about 20mW. The “DIC” can be reduced to half using the cutting-edge 2-4nm semiconductor process, and the power consumption of the tilt mirror can be reduced to less than 10mW using the large size tilt mirror. Accordingly, the total power consumption can be reduced to less than 60mW.

As a result, the weight of the battery can be decreased, making AR glasses suitable for everyday use.

[Fourth Embodiment]

Next, the image projection device 1 according to the fourth embodiment will be described, focusing on the differences from the embodiments described above. The third embodiment described an example of the image projection device 1 where the line-of-sight direction detector 16 is configured with the event camera and where the laser elements 111, 112, 113 are VCSEL.

On the other hand, the fourth embodiment differs from the third embodiment in that the laser elements 111, 112, 113 are edge-emitting laser diodes.

Examples showing the power consumption of the image projection device 1 according to the fourth embodiment will be described.

[Example]

In an example of the fourth embodiment, the first laser element 111 with the current-optical output characteristics shown in FIG. 3A was driven under driving conditions shown in FIG. 32A. The second laser element 112 with the current-optical output characteristics shown in FIG. 3B was driven under driving conditions shown in FIG. 32B. The third laser element 113 with the current-optical output characteristics shown in FIG. 3C was driven under driving conditions shown in FIG. 32C. This resulted in the power consumption shown in FIG. 32D.

In the example of the fourth embodiment, by vacuum sealing the MEMS mirror 141 (see FIG. 17) and increasing the efficiency of a thin piezo film provided on the MEMS mirror 141, the power consumption “MEMS+ASIC” can be reduced to 10mW.

Further, according to the example of the fourth embodiment, the “DIC” can be reduced to 20mW by performing the foveated rendering. As a result, according to the example of the fourth embodiment, the power consumptions “PC1” and “PC2” can be reduced to about 100mW or less than 100mW by driving the laser elements 111, 112, 113 with the pulse having the pulse width of 1.1nsec or less.

[Example of the weight of the image projection device]

Next, referring to FIGS. 33-35, an example of the weight of the image projection device 1 will be described. In FIGS. 33-35, “PC of display” corresponds to the power consumption “PC1” or “PC2” described above.

As shown in FIG. 33, according to the example of the present disclosure, the weight of the image projection device excluding the glass portion can be reduced than before by reducing the diameter and the power consumption of the tilt mirror.

Taking the small size tilt mirror module as an example, in the conventional image projection device 1, the battery cannot be lightened since the total power consumption including 300mW power consumption of the CPU is 900mW, the tilt mirror modules on the right and left are heavy as 13g since the diameter of the tilt mirror is 16mm, and the weight of the image projection device 1 excluding the glass portion is 59.4g. The weight including the glass portion is 65.4g, as shown in FIG. 34.

On the other hand, according to one example of the present disclosure, the battery can be lightened by reducing the total power consumption to 700mW, the weight of the tilt mirror modules on the right and left can be reduced to 3.3g by reducing the diameter of the tilt mirror to 8mm; and the weight of the image projection device 1 excluding the glass portion can be reduced to 42.9g. In this case, the weight including the glass portion can be reduced to 48.9g.

Further, according to another example of the present disclosure, the battery can be further lightened by reducing the total power consumption to 490mW; the weight of the tilt mirror modules on the right and left can be reduced to 3.3g by reducing the diameter of the tilt mirror to 8mm, and the weight of the image projection device excluding the glass portion can be reduced to 37.9g. In this case, the weight including the glass portion is reduced to 43.9g.

Further, according to another example of the present disclosure, the battery can be further lightened by reducing the power consumption of one of the display devices to 61mW as shown in FIG. 35 and by reducing the total power consumption to 318mW, the weight of the tilt mirror modules on the right and left can be reduced to 3.3g by reducing the diameter of the tilt mirror to 8mm, and the weight of the image projection device 1 excluding the glass portion can be reduced to 39.9g.

Therefore, according to the examples shown in FIGS. 33-35, it becomes possible to reduce the weight of the image projection device 1 including the glass portion to less than 50g to enhance the usability of the image projection device 1 (i.e., AR glass) .

With the waveguide method, since color unevenness tends to occur in the image, high brightness is difficult to achieve, and prescription glasses are required, it is difficult to achieve highly realistic AR images. With the retina projection method that can project directly onto the retina, highly bright images can be realized in principle with free focus, and furthermore, high-quality, realistic AR images can be realized with a total weight of 40g or less by reducing the power consumption of the eye position sensor for eye tracking, the tilt mirror module, and the display device as in this example. Such AR glasses for daily use can be realized only with the retina projection method and is technically difficult with other methods.

In the description of embodiments of the present disclosure, it is to be understood that terms such as "central" , "longitudinal" , "transverse" , "length" , "width" , "thickness" , "upper" , "lower" , "front" , "rear" , "left" , "right" , "vertical" , "horizontal" , "top" , "bottom" , "inner" , "outer" , "clockwise" and "counterclockwise" should be construed to refer to the orientation or the position as described or as shown in the drawings under discussion. These relative terms are only used to simplify description of the present disclosure, and do not indicate or imply that the device or element referred to must have a particular orientation, or constructed or operated in a particular orientation. Thus, these terms cannot be constructed to limit the present disclosure.

In addition, terms such as "first" and "second" are used herein for purposes of description and are not intended to indicate or imply relative importance or significance or to imply the number of indicated technical features. Thus, the feature defined with "first" and "second" may comprise one or more of this feature. In the description of the present disclosure, "a plurality of" means two or more than two, unless specified otherwise.

In the description of embodiments of the present disclosure, unless specified or limited otherwise, the terms "mounted" , "connected" , "coupled" and the like are used broadly, and may be, for example, fixed connections, detachable connections, or integral connections; may also be mechanical or electrical connections; may also be direct connections or indirect connections via intervening structures; may also be inner communications of two elements, which can be understood by those skilled in the art according to specific situations.

In the embodiments of the present disclosure, unless specified or limited otherwise, a structure in which a first feature is "on" or "below" a second feature may include an embodiment in which the first feature is in direct contact with the second feature, and may also include an embodiment in which the first feature and the second feature are not in direct contact with each other, but are contacted via an additional feature formed therebetween. Furthermore, a first feature "on" , "above" or "on top of" a second feature may include an embodiment in which the first feature is right or obliquely "on" , "above" or "on top of" the second feature, or just means that the first feature is at a height higher than that of the second feature; while a first feature "below" , "under" or "on bottom of" a second feature may include an embodiment in which the first feature is right or obliquely "below" , "under" or "on bottom of" the second feature, or just means that the first feature is at a height lower than that of the second feature.

Various embodiments and examples are provided in the above description to implement different structures of the present disclosure. In order to simplify the present disclosure, certain elements and settings are described in the above. However, these elements and settings are only by way of example and are not intended to limit the present disclosure. In addition, reference numbers and/or reference letters may be repeated in different examples in the present disclosure. This repetition is for the purpose of simplification and clarity and does not refer to relations between different embodiments and/or settings. Furthermore, examples of different processes and materials are provided in the present disclosure. However, it would be appreciated by those skilled in the art that other processes and/or materials may be also applied.

Reference throughout this specification to "an embodiment" , "some embodiments" , "an exemplary embodiment" , "an example" , "a specific example" or "some examples" means that a particular feature, structure, material, or characteristic described in connection with the embodiment or example is included in at least one embodiment or example of the present disclosure. Thus, the appearances of the above phrases throughout this specification are not necessarily referring to the same embodiment or example of the present disclosure. Furthermore, the particular features, structures, materials, or characteristics may be combined in any suitable manner in one or more embodiments or examples.

Any process or method described in a flow chart or described herein in other ways may be understood to include one or more modules, segments or portions of codes of executable instructions for achieving specific logical functions or steps in the process, and the scope of a preferred embodiment of the present disclosure includes other implementations, in which it should be understood by those skilled in the art that functions may be implemented in a sequence other than the sequences shown or discussed, including in a substantially identical sequence or in an opposite sequence.

The logic and/or step described in other manners herein or shown in the flow chart, for example, a particular sequence table of executable instructions for realizing the logical function, may be specifically achieved in any computer readable medium to be used by the instruction execution system, device or equipment (such as the system based on computers, the system comprising processors or other systems capable of obtaining the instruction from the instruction execution system, device and equipment and executing the instruction) , or to be used in combination with the instruction execution system, device and equipment. As to the specification, "the computer readable medium" may be any device adaptive for including, storing, communicating, propagating or transferring programs to be used by or in combination with the instruction execution system, device or equipment. More specific examples of the computer readable medium comprise but are not limited to: an electronic connection (an electronic device) with one or more wires, a portable computer enclosure (a magnetic device) , a random access memory (RAM) , a read only memory (ROM) , an erasable programmable read-only memory (EPROM or a flash memory) , an optical fiber device and a portable compact disk read-only memory (CDROM) . In addition, the computer readable medium may even be a paper or other appropriate medium capable of printing programs thereon, this is because, for example, the paper or other appropriate medium may be optically scanned and then edited, decrypted or processed with other appropriate methods when necessary to obtain the programs in an electric manner, and then the programs may be stored in the computer memories.

It should be understood that each part of the present disclosure may be realized by the hardware, software, firmware or their combination. In the above embodiments, a plurality of steps or methods may be realized by the software or firmware stored in the memory and executed by the appropriate instruction execution system. For example, if it is realized by the hardware, likewise in another embodiment, the steps or methods may be realized by one or a combination of the following techniques known in the art: a discrete logic circuit having a logic gate circuit for realizing a logic function of a data signal, an application-specific integrated circuit having an appropriate combination logic gate circuit, a programmable gate array (PGA) , a field programmable gate array (FPGA) , etc.

Those skilled in the art shall understand that all or parts of the steps in the above exemplifying method of the present disclosure may be achieved by commanding the related hardware with programs. The programs may be stored in a computer readable storage medium, and the programs comprise one or a combination of the steps in the method embodiments of the present disclosure when run on a computer.

In addition, each function cell of the embodiments of the present disclosure may be integrated in a processing module, or these cells may be separate physical existence, or two or more cells are integrated in a processing module. The integrated module may be realized in a form of hardware or in a form of software function modules. When the integrated module is realized in a form of software function module and is sold or used as a standalone product, the integrated module may be stored in a computer readable storage medium.

The storage medium mentioned above may be read-only memories, magnetic disks, CD, etc.

Although embodiments of the present disclosure have been shown and described, it would be appreciated by those skilled in the art that the embodiments are explanatory and cannot be construed to limit the present disclosure, and changes, modifications, alternatives and variations can be made in the embodiments without departing from the scope of the present disclosure.

Claims

An image projection device, comprising:

a light source that emits a laser beam;

a scanner that scans the laser beam emitted from the light source;

a plurality of reflective mirrors, comprising at least a first reflective mirror that reflects the laser beam scanned by the scanner toward a retina of a user and is capable of changing a reflecting direction of the laser beam, and a second reflective mirror that reflects the laser beam reflected by the first reflective mirror toward the retina to project an image on the retina;

a line-of-sight direction detector that detects a line-of-sight direction of the user;

a control circuit that controls the reflecting direction of the laser beam reflected by the first reflective mirror, such that the image is projected from the line-of-sight direction, which is detected by the line-of-sight direction detector, to the retina; and

a drive circuit that drives the light source with a pulse,

wherein the first reflective mirror is larger than a pupil diameter of the user and smaller than the second reflective mirror, and

the drive circuit drives the light source with the pulse having a pulse width shorter than a pulse width determined by a modulation speed required in one pixel.
The image projection device according to claim 1, wherein a diameter of the first reflective mirror is 8mm or less.
The image projection device according to claim 1, wherein the first reflective mirror comprises a tilt mirror module having a driving system.
The image projection device according to claim 1, wherein the line-of-sight direction detector is an eye-tracking camera.
The image projection device according to claim 4, wherein the line-of-sight direction detector comprises a global shutter type image sensor.
The image projection device according to claim 5, wherein the line-of-sight direction detector intermittently drives the global shutter type image sensor.
The image projection device according to claim 4, wherein the line-of-sight direction detector comprises an event sensor.
The image projection device according to claim 1, wherein the scanner scans the laser beam emitted from the light source with a Lissajous scan method.
The image projection device according to claim 1, wherein the light source emits the laser beam such that a resolution of the image projected from the line-of-sight direction detected by the line-of-sight direction detector increases and a resolution of the image projected from directions other than the detected line-of-sight direction decreases.
The image projection device according to claim 1, wherein the light source comprises:

at least one first laser element that emits the laser beam of a wavelength from 610 nm to 680 nm and has an oscillation threshold current of 15 mA or less;

at least one second laser element that emits the laser beam of the wavelength from 500 nm to 530 nm and has the oscillation threshold current of 25 mA or less; and

at least one third laser element that emits the laser beam of the wavelength from 430 nm to 470 nm and has the oscillation threshold current of 10 mA or less.
The image projection device according to claim 10, wherein

a diameter of the first reflective mirror is 8mm or less, and

the line-of-sight direction detector is an eye tracking camera comprising a global shutter type image sensor, the line-of-sight direction detector intermittently driving the global shutter type image sensor.
The image projection device according to claim 10, wherein

a diameter of the first reflective mirror is 8mm or less, and the line-of-sight direction detector is an eye tracking camera comprising an event sensor.
The image projection device according to claim 10, wherein each of the first, second, and third laser element is an edge-emitting laser diode having a low oscillation threshold current.
The image projection device according to claim 13, wherein

a diameter of the first reflective mirror is 8mm or less, and

the line-of-sight direction detector is an eye tracking camera comprising a global shutter type image sensor, the line-of-sight direction detector intermittently driving the global shutter type image sensor.
The image projection device according to claim 13, wherein

a diameter of the first reflective mirror is 8mm or less, and

the line-of-sight direction detector is an eye tracking camera comprising an event sensor.
The image projection device according to claim 10, wherein each of the first, second, and third laser element is a vertical cavity surface emitting laser (VCSEL) having a low oscillation threshold current.
The image projection device according to claim 10, wherein

the oscillation threshold current of the first laser element is 1 mA or less,

the oscillation threshold current of the second laser element is 3 mA or less, and

the oscillation threshold current of the third laser element is 1 mA or less.
The image projection device according to claim 17, wherein the oscillation threshold current of the second laser element is 1mA or less.
The image projection device according to claim 10, wherein the light source comprises a waveguide type optical multiplexer that multiplexes the laser beams emitted from each of the first, second, and third laser elements by making the laser beams incident on a waveguide.
The image projection device according to claim 10, wherein the drive circuit drives the first, second, and third laser elements with the pulse having a pulse width of 2.2 nsec or less.
The image projection device according to claim 3, where the first reflective mirror further comprises:

a movable body where a tilt mirror is provided;

a gimbal mechanism that supports the movable body in a swinging manner; and

a magnetic drive mechanism that swings the movable body.
The image projection device according to claim 1, wherein the second reflective mirror is a free-form mirror.
The image projection device according to claim 1, wherein the second reflective mirror is a holographic optical element.
The image projection device according to claim 10, wherein

the first laser element comprises a first active layer;

the second laser element comprises a second active layer;

the third laser element comprises a third active layer;

the first active layer comprises Aluminum Gallium Indium Phosphide (InGaAlP) , and

the second and third active layers comprise Indium Gallium Nitride (InGaN) .
The image projection device according to claim 10, wherein the light source comprises a plurality of each of the first, second, and third laser elements.
The image projection device according to claim 1, wherein the scanner comprises a MEMS (Micro Electric Mechanical System) mirror.
The image projection device according to claim 26, wherein

a diameter of the MEMS mirror is 2mm or more, and

a swinging angle of the MEMS mirror is 9.0° or less when scanning the laser beam.
The image projection device according to claim 26, wherein the scanner further comprises an ASIC (Application Specific Integrated Circuit) that drives the MEMS mirror.
The image projection device according to claim 28, wherein the MEMS mirror and the ASIC are vacuum sealed.
The image projection device according to claim 29, wherein

a diameter of the first reflective mirror is 8mm or less, and

the light source comprises:

at least one first laser element that emits the laser beam of a wavelength from 610 nm to 680 nm and has an oscillation threshold current of 15 mA or less;

at least one second laser element that emits the laser beam of the wavelength from 500 nm to 530 nm and has the oscillation threshold current of 25 mA or less; and

at least one third laser element that emits the laser beam of the wavelength from 430 nm to 470 nm and has the oscillation threshold current of 10 mA or less.
The image projection device according to claim 29, wherein

a diameter of the first reflective mirror is 8mm or less, and

each of the first laser element, the second laser element, and the third laser element is an edge-emitting laser diode having a low oscillation threshold current.