WO2023074138A1

WO2023074138A1 - Image display apparatus, electronic appliance, and integrated type device

Info

Publication number: WO2023074138A1
Application number: PCT/JP2022/033700
Authority: WO
Inventors: 利文安井
Original assignee: ソニーグループ株式会社
Priority date: 2021-10-29
Filing date: 2022-09-08
Publication date: 2023-05-04

Abstract

The purpose of the present invention is to provide a compact eye-sensing optical system to a small-sized image display apparatus that can be mounted to a heat mount display and an EVF, and to reduce an influence on a form factor.　An image display apparatus according to the present invention is provided with an optical system and an integration type device (110) having: an image light emission part (140) for emitting image light that passes through the optical system and forms an image on the retina; eye-sensing light emission parts (150) for emitting eye-sensing light that passes through the optical system and is reflected by the pupil; and an eye-sensing light reception part (130) having a light-reception unit pixel that includes a plurality of light-reception unit subpixels separated from each other and having a light reception microlens for forming images on the plurality of light-reception unit subpixels from the eye-sensing light which is reflected by the pupil, passes through the optical system, and enters the microlens.

Description

Image display devices, electronic equipment and integrated devices

The present disclosure includes an image display device that can be mounted on a head-mounted display or an EVF (Electronic View Finder), an electronic device such as a head-mounted display having this image display device, an integrated device included in this image display device, Regarding.

Head-mounted displays (VR (Virtual Reality), AR (Augmented Reality)) may use eye-sensing technology to detect the line of sight of the viewer.

According to Patent Document 1, the image monitor device incorporates an image generation device that displays an image of a subject, and an optical system and an eyepiece are arranged in front of it. A pair of infrared diodes that emit infrared rays toward the eyes of the observer are arranged near the bottom of the eyepiece. A dichroic mirror is positioned between the eyepieces at an inclination angle greater than 45 degrees with respect to the optical axis of the eyepieces, the dichroic mirror reflecting each of the corneal reflections of the observer. Each corneal reflection image reflected by the dichroic mirror is condensed by the imaging lens and formed on the image sensor (abstract).

JP-A-8-9205

The optical system for eye sensing restricts the form factor, so the performance may deteriorate, especially if the overall size is reduced. For example, the technique of Patent Document 1 can be implemented in a large image monitor device that has space for a dichroic mirror. I can't.

In view of the circumstances described above, the purpose of the present disclosure is to provide a compact image display device that can be mounted on a head-mounted display or EVF with a compact eye sensing optical system to reduce the impact on the form factor.

An image display device according to one aspect of the present disclosure includes:
an optical system;
an image light projection unit that emits image light that is imaged on the retina through the optical system;
an eye-sensing light projector that emits eye-sensing light that is reflected by the pupil;
an eye-sensing light receiving unit that receives the eye-sensing light reflected by the pupil via the optical system;
a light-receiving microlens for forming an image of the eye-sensing light reflected by the pupil and incident through the optical system on the eye-sensing light-receiving unit;
an integrated device having
and
the light-receiving microlens is not included in the optical path of the image light and is not included in the optical path along which the eye sensing light reaches the pupil;
The entrance pupils of the image light projecting section and the light receiving microlens are arranged at the same position in the optical axis direction of the optical system.

By using an integrated device, the degree of freedom in the form factor is increased, the size is reduced, and the eye-sensing accuracy can be improved because the eye-sensing light receiving part can be viewed from the front.

The eye sensing light receiving unit has a cut filter that cuts the image light,
Alternatively, it may have the light-receiving microlens for cutting image light.

As a result, the accuracy of eye sensing can be increased.

Light projected from the eye-sensing light projecting unit passes through the optical system and is reflected by a pupil, and the eye-sensing light projecting unit and the entrance pupils of the light-receiving microlenses are at the same position in the optical axis direction of the optical system. may be placed in

The degree of freedom in the form factor has been increased, the size has been reduced, and the eye-sensing accuracy can be improved because the eye-sensing light receiving part can be viewed from the front.

The eye-sensing light projector may have a light-projecting microlens for increasing the amount of the eye-sensing light.

By increasing the amount of eye-sensing light with the projection microlens, it can also be used in a triple-pass system.

The projection microlens of the eye-sensing light projection unit may be decentered so that the direction of the eye-sensing light coincides with the exit angle direction of the image light.

A radial direction of the effective image circle from the center of the image light projecting unit is defined as an α direction,
The direction orthogonal to the α direction is the β direction,
Let D be the distance from the center of the image light projection unit to the center of the eye sensing light projection unit,
B is the pupil diameter of the image light projection unit,
Let the emergence angles from D with respect to B be θ _{α+_D} , θ _{α−_D} , θ _{β+_D} , θ _{β−_D} , θ _{α_Center_D} , where θ _{α_Center_D} is the chief ray angle,
Let the projection range of the eye sensing light projection unit be α direction C _αPRJ and β direction C _βPRJ ,
Assuming that the light receiving range angles of the eye sensing light with respect to the light receiving range C are ξα _{+_PRJ} , ξα _{−_PRJ} , ξβ _{+_PRJ} , and ξβ _{−_PRJ} ,
The light-receiving microlens of the eye-sensing light projector may be decentered so as to satisfy the formulas (1-1′), (1-2′), (1-3′) and (1-4′). .
ξ _{α+_PRJ} − θ _{α_Center_D} ≧C _αPRJ /B・(θ _{α+_D} −θ _{α_Center_D} ) (1−1′)
ξ _{α−_PRJ} −θ _{α_Center_D} ≦C _αPRJ /B・(θ _{α−_D} −θ _{α_Center_D} ) (1-2′)
ξ _{β＋_PRJ} ≧C _{β_D} /B・θ _{β_D} (1−3′)
ξ _{β-_PRJ} ≤ C _{β_D} /B・θ _{β_D} (1-4')

The light-receiving microlens of the eye-sensing light receiving section may be decentered so that the eye-sensing light reflected by the pupil is imaged at the center of the eye-sensing light-receiving section.

Accurate eye sensing is possible.

A radial direction of the effective image circle from the center of the image light projecting unit is defined as an α direction,
The direction orthogonal to the α direction is the β direction,
Let A be the distance from the center of the image light projecting unit to the center of the eye sensing light receiving unit,
B is the pupil diameter of the image light projection unit,
Let the emergence angles from A with respect to B be θ _{α+_A} , θ _{α−_A} , θ _{β+_A} , θ _{β−_A} , θ _{α_Center_A} , where θ _{α_Center_A} is the chief ray angle,
Let the light receiving range of the eye sensing light receiving portion be α direction C _αDET and β direction C _βDET ,
Assuming that the light receiving range angles of the eye sensing light with respect to the light receiving range C are ξα _{+_DET} , ξα _{−_DET} , ξβ _{+_DET} , and ξβ _{−_DET} ,
The light-receiving microlenses of the eye-sensing light-receiving section may be decentered so as to satisfy formulas (1-1), (1-2), (1-3) and (1-4).
ξ _{α+_DET} −θ _{α_Center_A} ≧C _αDET /B・(θ _{α+_A} −θ _{α_Center_A} ) (1-1)
ξ _{α−_DET} −θ _{α_Center_A} ≦C _αDET /B・(θ _{α−_A} −θ _{α_Center_A} ) (1-2)
ξ _{β＋_DET} ≧C _{β_A} /B・θ _{β_A} (1-3)
ξ _{β-_DET} ≦ C _{β_A} /B・θ _{β_A} (1-4)

The image light projecting section may be provided between the eye sensing light projecting section and the eye sensing light receiving section, and the optical axis in a plane perpendicular to the optical axis of the optical system.

As a result, an appropriate optical path can be obtained with the addition of a minimum number of members, so a compact image display device can be realized.

the image light projection unit has a rectangular shape on a plane perpendicular to the optical axis of the optical system,
The eye sensing light receiving section may be positioned within the effective image circle of the optical system on a straight line connecting the center of the long side of the image light projecting section and the optical axis of the optical system.

　The imaging resolution due to the projection of the image light from the image light projection unit does not deteriorate, and high resolution can be achieved.

The eye-sensing light receiving part may include a plurality of light-receiving unit sub-pixels separated from each other, the light-receiving microlenses may be plural, and each micro-lens may form an image of the eye-sensing light on each light-receiving unit sub-pixel.

In this way, in the eye-sensing light receiving section, each light receiving unit pixel has a plurality of light receiving unit sub-pixels separated from each other, so that each light receiving unit sub-pixel receives an eye from a specific on-pupil object point. Only sensing light can be imaged. Furthermore, although the Z-direction positions of the image light projection unit that emits the image light to be imaged on the retina and the eye sensing light projection unit that emits the eye sensing light to be imaged on the retina are substantially the same, The lens allows the eye-sensing light to form an image on each light-receiving unit pixel of the eye-sensing light receiving section. As a result, it is possible to achieve both miniaturization and improvement in form factor flexibility by using an integrated device, as well as improvement in eye sensing accuracy.

By providing a partition extending in the optical axis direction of the optical system between adjacent light-receiving unit sub-pixels, or by shielding the periphery of each light-receiving unit sub-pixel,
The plurality of light-receiving unit sub-pixels may be separated from each other.

By shielding each light-receiving unit pixel, the resolution is improved, and by obtaining a sufficient amount of eye-sensing light, calculation is facilitated, and it can also be used in a triple-pass system.

The plurality of light-receiving unit sub-pixels included in some of the light-receiving unit pixels and the plurality of light-receiving unit sub-pixels included in the other portion of the light-receiving unit pixels may be offset.

By offsetting each light-receiving unit pixel, it is possible to increase the pseudo-resolution while minimizing the loss of the amount of eye-sensing light. By sufficiently obtaining , it is possible to use it even in a triple-pass system while facilitating the calculation.

The image light projection unit has an inner area and an outer area,
the pixel size of the outer region is larger than the pixel size of the inner region;
The eye-sensing light receiver may be provided as a pixel within the outer region of the image light projector.

By providing the eye-sensing light receiving part as a pixel in the outer area of the image light projecting part, the eye-sensing light receiving part can be arranged further inside, so that the image quality is improved by being less affected by the aberration of the VR optical system.

The eye-sensing light projector may be provided as a pixel within the outer region of the image light projector.

By providing the eye sensing light projection part as a pixel in the outer region of the image light projection part, it is possible to perform eye sensing with high accuracy.

An electronic device according to one aspect of the present disclosure includes:
An image display device according to any one of the above.

An integrated device according to one aspect of the present disclosure includes:
an image light projection unit that emits image light that is imaged on the retina through an optical system;
an eye-sensing light projector that emits eye-sensing light that is reflected by the pupil;
an eye-sensing light receiving unit that receives the eye-sensing light reflected by the pupil via the optical system;
a light-receiving microlens for forming an image of the eye-sensing light reflected by the pupil and incident through the optical system on the eye-sensing light-receiving unit;
and
the light-receiving microlens is not included in the optical path of the image light and is not included in the optical path along which the eye sensing light reaches the pupil;
The entrance pupils of the image light projecting section and the light receiving microlens are arranged at the same position in the optical axis direction of the optical system.

1 shows a typical image display device. Light projection and light reception are shown schematically. Schematically shows a cross-sectional view of an eyeball. 1 is a side view schematically showing an image display device according to a first embodiment; FIG. It is a side view which shows an optical system. It is a front view which shows an integrated device typically. It is a side view which shows an image light projection part typically. FIG. 4 is a side view schematically showing an eye sensing light receiving section; FIG. 4 is a side view schematically showing an eye sensing light receiving section; 4 is a graph showing the relationship between defocusing position and contrast; It is a graph which shows the emission angle of an image light projection part. 4 shows the distribution of eye-sensing light emitted by the eye-sensing light projector. 3 shows an imaging range of an eye sensing light receiving section and a projection range of an eye sensing light projecting section; FIG. 11 is a front view schematically showing an integrated device according to a second embodiment; 4 is a graph showing the relationship between spatial frequency and contrast; An example of image correction is shown. FIG. 4 is a diagram for explaining crosstalk; FIG. 1A and 1B are a side view and a partially enlarged view schematically showing an optical system; FIG. FIG. 11 is a front view schematically showing an integrated device according to a third embodiment; 4 is a graph showing the relationship between spatial frequency and contrast; It is a front view which shows typically the integrated device which concerns on 4th Embodiment. FIG. 4 is a front view schematically showing an eye sensing light receiving section; FIG. 4 is a side view schematically showing an eye sensing light receiving section; FIG. 11 is a front view schematically showing light-receiving unit pixels included in an eye-sensing light-receiving section according to a fifth embodiment; It is a front view which shows typically the integrated device which concerns on 6th Embodiment. It is a front view which shows typically the inner area|region and outer area|region of an image light projection part. FIG. 11 is a front view schematically showing an integrated device according to a seventh embodiment; It is a front view which shows typically the inner area|region and outer area|region of an image light projection part. It is a figure for demonstrating the case where the viewer wears spectacles. FIG. 20 is a front view schematically showing an integrated device according to a ninth embodiment; FIG. 21 is a front view schematically showing an integrated device according to a tenth embodiment; FIG. 21 is a front view schematically showing an integrated device according to an eleventh embodiment; 3 shows an image display device according to a third embodiment.

Hereinafter, embodiments of the present disclosure will be described with reference to the drawings.

　I. 1st embodiment

1. Concept of this embodiment

There is a need for high resolution such as 8K and 16K for image display devices installed in VR head-mounted displays. On the other hand, in order to display high-resolution data such as 8K and 16K over the entire display area, the transmission speed band to the image display panel may be insufficient. Foveated Rendering is effective in securing the transmission band. Foveal rendering uses the fact that the human eye generally has high resolution only at the fixation point and sharply drops in resolution around it. This is a method of securing a transmission band by displaying an image. Eye sensing techniques such as the following are typically used to detect the position of the gaze point.

Fig. 1 shows a typical image display device.

A typical image display device 10 mounted on a VR head-mounted display has an image light projector 11, an optical system 12, an eye sensing light projector 15, and an eye sensing light receiver 14. . The image light projection unit 11 emits image light (RGB light) that forms an image on the retina. The optical system 12 causes parallel image light (broken lines in the drawing) to enter the pupil and form an image of the image light on the retina of the eye 16 . On the other hand, the eye-sensing light projector 15 is an infrared projector that emits eye-sensing light (infrared rays) (solid line in the drawing) that is reflected by the pupil (cornea or lens). The eye-sensing light receiving unit 14 is an infrared camera, receives eye-sensing light (infrared rays) reflected by the pupil, and forms a Purkinje image (object information on the pupil), which is an object point on the pupil.

The image light projection unit 11 and the optical system 12 require angle information on the pupil because the image light is incident on the pupil in parallel. On the other hand, the eye-sensing optical path of the eye-sensing light projecting unit 15 forms an image of an object point on the pupil on the eye-sensing light receiving unit 14, so object information on the pupil is required. In this way, the image optical path and the eye sensing optical path require different information on the pupil. Therefore, the eye sensing light projecting unit 15 and the eye sensing light receiving unit 14 are separated from the image light projecting unit 11 in the optical axis direction (hereinafter also referred to as the Z direction) of the optical system. It is typically installed with For this reason, there are problems such as form factor restrictions, the increase in the size of the VR lens especially when the angle of view is large, and the deterioration of eye sensing accuracy in oblique viewing.

In view of the circumstances as described above, in the image display device of the present embodiment, the eye sensing light projecting section and the eye sensing light receiving section are arranged in the optical axis direction (Z direction) of the optical system with respect to the image light projecting section. set approximately equal to More specifically, the image light projecting section, the eye sensing light projecting section, and the eye sensing light receiving section are configured as an integrated device configured on a plane perpendicular to the Z direction. As a result, the image display device of the present embodiment increases the degree of freedom of the form factor by using an integrated device, is miniaturized, and can be viewed from the front by the eye-sensing light receiving part, so that the accuracy of eye-sensing can be improved. Plan.

FIG. 2 schematically shows light projection and light reception.

As shown in (A), the light projected from the integrated device uses the position information (Z position) at the light projecting part of the integrated device, the angle information (Z position) at the pupil, and the position information (Z position) at the retina. can be regarded as converting to On the other hand, as shown in (B), the light reception by the integrated device is based on the angle information (Z position) on the retina and the position information (Z position) on the pupil, and the angle information (Z position) on the light receiving unit of the integrated device. can be regarded as converting to Although it is not possible to directly acquire the angle information in the light receiving part of the integrated device, it is possible to acquire the angle information by acquiring the light field. A light field is a vector function that represents the amount of light flowing in multiple directions through space. As shown in (C), angle information on the XY plane orthogonal to the Z direction can be reconstructed from the light field.

FIG. 3 schematically shows a cross-sectional view of an eyeball.

As described above, in this embodiment, both image light and eye sensing light are emitted from the integrated device. The image light forms an image on the retina, and the Purkinje image of the eye sensing light forms a virtual image on the pupil (cornea, lens). Thus, the imaging position of the image light and the imaging position of the eye sensing light are different in the Z direction. For this reason, when light is received by an integrated device, it is necessary to deal with the problem that this difference in the Z direction causes defocus and blurring. Therefore, the image display device according to this embodiment has the following configuration.

2. Configuration of image display device

FIG. 4 is a side view schematically showing the image display device according to the first embodiment.

An XY plane view of the image display device viewed from the optical axis direction (Z direction) of the optical system is hereinafter referred to as a front view. A view of the image display device seen from a direction orthogonal to the Z direction is called a side view.

The image display device 100 is mounted on, for example, a head-mounted display (not shown). Specifically, two image display devices 100 are mounted on one head-mounted display so as to correspond to both eyes of the viewer.

The image display device 100 has an integrated device 110 and an optical system 120 .

FIG. 5 is a side view showing the optical system.

The optical system 120 causes parallel image light to enter the pupil and forms an image of the image light on the retina of the eye 16 . The optical system 120 includes, for example, a polarizing plate 121, a quarter-wave plate 122, a lens 123, a half mirror 124, a quarter-wave plate 125, a polarizing plate 126, and a lens 127 in order from the light emitting surface of the integrated device 110. including. The illustrated configuration of the optical system 120 is an example of the VR optical system, and is not limited to the illustrated configuration. However, if the configuration of the VR optical system is different, the angle of the peripheral ray will change, so it is necessary to design the light field capture angle accordingly.

FIG. 6 is a front view schematically showing the integrated device.

The integrated device 110 has an image light projecting section 140 , an eye sensing light projecting section 150 and an eye sensing light receiving section 130 . It has an image light projecting section 140 , an eye sensing light projecting section 150 and an eye sensing light receiving section 130 . Although it is desirable that the eye sensing light receiving part 130 is formed on the same wafer, it is not necessarily formed on the same wafer. Image light projecting section 140 , eye sensing light projecting section 150 , and eye sensing light receiving section 130 are arranged at the same position in the optical axis direction (Z direction) of optical system 120 . In other words, the image light projecting section 140, the eye sensing light projecting section 150, and the eye sensing light receiving section 130 are arranged on a common XY plane orthogonal to the Z direction.

FIG. 7 is a side view schematically showing the image light projecting section.

The image light projection unit 140 emits image light that passes through the optical system 120 and forms an image on the retina. Specifically, the image light projection unit 140 is a μOLED (Micro Organic Light Emitting Diode) panel and emits RGB light. The image light projector 140 has a plurality of light emitting cells 141 and a lens 142 on each light emitting cell 141 . Light-emitting cell 141 includes color filter 144 and light-emitting layer 145 . The lens 142 is a lens for more efficiently forming an image of the image light emitted by the light emitting cell 141 on the retina. Lens 142 is covered with cover glass 143 . In other words, the image light emitted by the image light projecting section 140 forms an image on the retina of the viewer by the optical system 120 and the lens 142 on each light emitting cell 141 provided in the image light projecting section 140 .

The image light projection unit 140 includes μOLED (micro organic light emitting diode), LCD (liquid crystal display), LCOS (liquid crystal on silicon), LED (light emitting diode), LD (laser diode), VCSEL (vertical Light-emitting devices such as cavity surface emitting lasers, vertical cavity surface emitting lasers, QD (Quantum Dot) OLEDs, etc. may also be used.

The image light projection unit 140 has, for example, a rectangular shape on the XY plane perpendicular to the optical axis (Z direction) of the optical system 120 . The size of the RGB light emitting area of the image light projecting section 140 is, for example, 1.28 inches diagonally (=32.5 mm) and 26 mm wide×19.5 mm long. The diameter of the effective image circle (that is, the effective diameter of the image light projection section 140) is φ32.5 mm, which is equal to the diagonal distance (32.5 mm). A center 21 of the image light projection unit 140 coincides with the optical axis of the optical system 120 . Hereinafter, the radiation direction from the center 21 of the image light projection unit 140 (the radial direction of the effective image circle 22) is defined as the α direction, and the direction orthogonal to the α direction is defined as the β direction.

A plurality of eye-sensing light projection units 150 (four in this example) are provided around the image light projection unit 140 . A distance D from the center 21 of the image light projecting section 140 to each eye sensing light projecting section 150 is, for example, 13 mm. The size of each eye sensing light projection unit 150 is, for example, a rectangle of 1.3 mm×0.98 mm. Each eye-sensing light projector 150 emits eye-sensing light that passes through the optical system 120 and is reflected by the pupil. The eye sensing light is, for example, infrared. Each eye-sensing light projector 150 has an LED 151 and a light-projecting microlens 152 for increasing the amount of eye-sensing light. A projection microlens 152 is provided on the LED 151 . A triple-pass system can also be used by increasing the light intensity of the eye-sensing light with the projection microlens 152 . The projection microlens 152 may be decentered so that the direction of the eye-sensing light coincides with the exit angle direction of the image light. As a result, the projection microlens 152 increases the light amount of the eye sensing light, so that it can be used even in a triple-pass system.

FIG. 9 is a side view schematically showing the eye sensing light receiving part.

The eye sensing light receiving section 130 has a light receiving unit pixel 133, a light receiving microlens 132, and a cut filter 134 (bandpass filter) for cutting image light. The eye sensing light receiving section 130 is, for example, an infrared light receiving area of 1.3 mm wide×0.98 mm long. The distance A from the center 21 of the image light projecting section 140 to the center of the eye sensing light receiving section 130 is preferably as short (close) as possible, for example, 10.6 mm. Power consumption can be reduced by using an event-based vision sensor for the eye sensing light receiving unit 130 .

The eye sensing light receiving section 130 is positioned outside the image light projecting section 140 with respect to the center 21 of the image light projecting section 140 corresponding to the optical axis of the optical system 120 . Conversely, in the XY plane orthogonal to the optical axis (Z direction) of the optical system 120, the image light projecting section 140 is positioned between the eye sensing light projecting section 150 and the eye sensing light receiving section 130 and the optical axis. provided in between. For this reason, the imaging resolution due to the projection of the image light from the image light projection unit 140 is not deteriorated, and the resolution can be increased and the productivity can be improved.

The eye sensing light receiving unit 130 is on a straight line connecting the center of the long side of the rectangular image light projecting unit 140 and the optical axis of the optical system 120 (the center 21 of the image light projecting unit 140). It lies within the effective image circle 22 of system 120 . For this reason, the imaging resolution due to the projection of the image light from the image light projection section 140 does not deteriorate, and the resolution can be increased.

The eye-sensing light receiving unit 130 consists of light receiving unit pixels 131 . For example, the size is 3.25 μm, the resolution is 156 LP/mm, the number of pixels is 400×300 pixels, and F=1.29.

The light-receiving microlenses 132 form images on the light-receiving unit pixels 131 of the eye-sensing light reflected by the pupil and incident through the optical system 120 .

The light-receiving microlens 132 is not included in the optical path of the image light, and is not included in the optical path along which the eye-sensing light emitted by the eye-sensing light projector 150 passes through the optical system 120 and reaches the pupil. In other words, of the three optical paths of the image light, the projection of the eye sensing light, and the reception of the eye sensing light, the optical path of the image light and the projection of the eye sensing light are suitable only for the optical system 120. An optical path is obtained. On the other hand, only the optical path for receiving the eye-sensing light can be obtained by adding the light-receiving microlens 132 in addition to the optical system 120 . As a result, an appropriate optical path can be obtained by adding a minimum number of members, so that a compact image display device 100 can be realized.

In this way, although the image light projection unit 140 that emits the image light to be imaged on the retina and the eye sensing light projection unit 150 that emits the eye sensing light to be imaged on the retina are substantially at the same position in the Z direction. Instead, the eye-sensing light can be imaged on each light-receiving unit pixel 131 of the eye-sensing light receiving unit 130 by the light-receiving microlens 132 arranged at an appropriate position. As a result, it is possible to achieve both miniaturization and improvement in form factor flexibility by using the integrated device 110 and improvement in eye sensing accuracy.

The image sensor surface of the eye-sensing light receiving unit 130 is positioned such that the center of the eye-sensing light receiving unit 130 is shifted in the Y-direction with respect to the optical system 132 because the eye-sensing light of 20 degrees from the top corresponds to the center of the pupil. may be provided, but is not required. For example, the eye sensing light receiver 130 may be shifted -0.49 mm in the Y direction. At the position indicated by the arrow in the figure, the entrance pupil Z position of the light receiving microlens 132 coincides with the OLED emission Z position. n in the figure means the refractive index. The focal length of the optical system 132 is 1.29 mm, which is substantially equal to the back focus of the optical system 132 of 1.37 mm. In other words, it is a configuration for forming an image with parallel light, thereby converting angle information into position information to obtain an eye-sensing image.

FIG. 10 is a graph showing the relationship between the defocusing position of the light receiving microlens 132 and the contrast.

When the center of the eye-sensing light receiving unit 130 is shifted with respect to the light receiving microlens 132, two pixels are generally resolved.

FIG. 11 is a graph showing the captureable angles of the optical system 120. FIG.

When the image height of the optical system 120 is 10.6 mm (that is, the distance A from the center 21 of the image light projector 140 to the center of the eye sensing light receiver 130 = 10.6 mm), the exit angle is It is 15deg to 25deg (center 20deg). When the image height is 13 mm (that is, the distance D from the center 21 of the image light projection unit 140 to the eye sensing light projection unit 150 is 13 mm), the emergence angle is 22 degrees to 33 degrees (27.5 degrees at the center).

FIG. 12 shows the distribution of eye-sensing light emitted by the eye-sensing light projector.

As shown in (A), the light distribution of a typical eye-sensing light projection unit (including LEDs and projection microlenses) is, for example, 20 degrees.

On the other hand, in the present embodiment, as shown in (B), the distribution area of the eye-sensing light projecting section 150 (including the LED 151 and the light projecting microlens 152) is widened and the orientation directions are aligned. For example, it is 5.5 deg to 49.5 deg in the α direction and 42 deg in the β direction. The calculation method for the conditions for the transmission and reception of infrared rays is as follows.

As described above, let D be the distance from the center 21 of the image light projection unit 140 to the center of the eye sensing light projection unit 150 (see FIG. 6). The radiation direction from the center 21 of the image light projection unit 140 (the radial direction of the effective image circle 22) is defined as the α direction, and the direction orthogonal to the α direction is defined as the β direction.

First, with respect to the pupil diameter φB (B is, for example, 4 mm, see FIG. 5) of the image light projection unit 140, the angles that can be captured by the optical system 120 at the image height D (see FIG. 11) are θ _{α+_D} , θ _{α− _D} , θ _{β+_D} , θ _{β−_D} , θ _{α_Center_D} . _{θα_Center_D} is the chief ray angle at the image height D.

Next, the required eye sensing light projection range angle for the range C (C is, for example, C _{α_PRJ} =16 mm, C _{β_PRJ} =16 mm) in the vicinity of the pupil to be illuminated by the eye sensing light projection unit 150 is calculated as ξ _{α+_PRJ} , ξ _{α−_PRJ} , ξ _{β+_PRJ} , ξ _{β−_PRJ} .

ξ _{α+_PRJ} − θ _{α_Center_D} ≧C _{α_PRJ} /B・(θ _{α+_D} −θ _{α_Center_D} ) (1-1)
ξ _{α−_PRJ} −θ _{α_Center_D} ≦C _{α_PRJ} /B・(θ _{α−_D} −θ _{α_Center_D} ) (1-2)
ξ _{β＋_PRJ} ≧C _{β_PRJ} /B・θ _{β_D} (1-3)
ξ _{β-_PRJ} ≦ C _{β_PRJ} /B・θ _{β_D} (1-4)

The projection microlenses 152 of the eye-sensing light projection unit 150 are arranged so that the direction of the eye-sensing light is It may be decentered so as to match the exit angle direction of the image light. As a result, the projection microlens 152 increases the light amount of the eye sensing light, so that it can be used even in a triple-pass system.

The light-receiving microlenses 132 of the eye-sensing light-receiving unit 130 each receive the eye-sensing It may be decentered so as to image the light to the center of the plurality of light receiving unit sub-pixels 131 . This enables accurate eye sensing.

The following formula holds for the angle of emergence from image height D (see FIGS. 6 and 11).

θα _{+_D} = 33deg
θ _{α-_D} = 22deg
θ _{α_Center_D} =27.5 degrees
θβ _{+_D} = 5.2deg
θ _{β-_D} =-5.2deg
Note that θβ _{+_D} and θβ _{−_D} are approximately equal to the upper ray and lower ray at an image height of 0 mm.

At this time, regarding the eye-sensing light projection range angles of the eye-sensing light projection unit 150, the eye-sensing light projection range angles are ξ _{α+_PRJ} , ξ _{α−_PRJ} , ξ _{β+_PRJ} , and ξ _{β−_PRJ} . The following formula holds.

ξα _{+_PRJ} ≧49.5deg
ξ _{α-_PRJ} ≤ 5.5deg
ξβ _{+_PRJ} ≧20.8deg
ξ _{β-_PRJ} ≤ -20.8deg

Similarly, regarding the angle of the eye-sensing light receiving range of the eye-sensing light receiving unit 130, the following formula holds for the angle of emergence from the image height A (see FIGS. 6 and 11).　

θα _{+_A} =25deg
θ _{α-_A} = 15deg
θ _{α_Center_A} = 20deg
θβ _{+_A} = 5.2deg
θ _{β-_A} =-5.2deg
Note that θβ _{+_A} and θβ _{−_A} are substantially equal to the upper ray and lower ray at an image height of 0 mm.

FIG. 13 shows the imaging range of the eye sensing light receiving section and the projection range of the eye sensing light projecting section.

For the eye sensing light receiving unit 130 having a size of 1.3 mm×0.98 mm, a range C near the pupil to be acquired (C is, for example, C _{α_DET} =12 mm, C _{β_DET} =16 mm) (light receiving coordinate system in FIG. 13) , the eye sensing light receiving range angles are ξα _{+_DET} , ξα _{−_DET} , ξβ _{+_DET} , and ξβ _{−_DET} . The following formula holds.

ξ _{α+_DET} −θ _{α_Center_A} ≧C _{α_DET} /B・(θ _{α+_A} −θ _{α_Center_A} ) (1−1′)
ξ _{α−_DET} −θ _{α_Center_A} ≦C _{α_DET} /B・(θ _{α−_A} −θ _{α_Center_A} ) (1-2′)
ξ _{β＋_DET} ≧C _{β_DET} /B・θ _{β_A} (1−3′)
ξ _{β-_DET} ≦ C _{β_DET} /B・θ _{β_A} (1-4')

ξα _{+_DET} = 35deg
ξ _{α-_DET} = 5deg
ξβ _{+_DET} = 20.8deg
ξ _{β-_DET} = -20.8deg

These satisfy the above formula. In this case, the imaging range of the eye sensing light receiving section 130 is rectangular. On the other hand, the eye-sensing light projection unit 150 has α=16 mm, and illuminates a slightly larger range. For example, the projection range of the eye-sensing light projecting section 150 is a circle (the projection coordinate system in FIG. 13) surrounding the rectangular imaging range of the eye-sensing light receiving section 130 .

As described above, the image display apparatus 100 of the present embodiment is compact because the image light projecting section 140, the eye sensing light projecting section 150, and the eye sensing light receiving section 130 are provided in the compact integrated device 110. Form factor constraints can be reduced.

FIG. 29 is a diagram for explaining a case where the viewer wears glasses.

As shown in (A), in a typical image display device 10, when a viewer wears spectacles 31, frontal reflection from an eye-sensing light projector 15 (infrared projector) is received. There is a flare optical path 32 that is captured by the unit 14 (infrared camera). Since the plurality of eye-sensing light projectors 15 are evenly arranged, the flare from the eyeglasses 31 is strong and enters the eye-sensing light receiver 14 in some flare light path 32 . In order to prevent this, it is preferable to arrange a plurality of eye-sensing light receiving units 14, but in this case, the form factor is further restricted. In addition, due to the restriction of the position of the eye sensing light receiving section 14, oblique viewing becomes severe, and the camera size of the eye sensing light receiving section 14 becomes extremely small, which tends to deteriorate the eye sensing accuracy.

On the other hand, as shown in (B), in the image display device 100 of the present embodiment, the flare light path 32 naturally enters the eye-sensing light receiving section 130, but a plurality of eye-sensing light receiving sections 130 are arranged. Since there are minimal restrictions on the form factor, the required number of eye-sensing light receiving units 130 can be arranged. Also, since the oblique viewing angle can be made relatively loose, it is easy to maintain eye sensing accuracy.

The cut filter 134 that cuts image light may be replaced by a light-receiving microlens 132 made of a material that cuts image light. Also, the microlens 132 may be composed of a plurality of compound lenses instead of a single lens.

II. Second embodiment

In the following, explanations and illustrations of configurations that are the same as those already explained will be omitted, and different points will mainly be explained and illustrated.

FIG. 14 is a front view schematically showing an integrated device according to the second embodiment.

The integrated device 210 according to the second embodiment and the integrated device 110 according to the first embodiment are different in the number and positions of the eye-sensing light projecting units 250 and the eye-sensing light receiving units 230, and The eye sensing light receiving part has a plurality of light receiving unit sub-pixels separated from each other. The specs of the image light projecting section 240 according to the second embodiment are the same as the specs of the image light projecting section 140 according to the first embodiment. The eye sensing light receiving part 230 is composed of light receiving unit pixels 131 . It has a plurality of light-receiving unit sub-pixels 131 separated from each other. For example, by providing a partition 135 extending in the optical axis direction (Z direction) of the optical system 120 between adjacent light-receiving unit sub-pixels 131, the plurality of light-receiving unit sub-pixels 131 are separated from each other. The light-receiving unit sub-pixel 131 is configured by dividing each light-receiving unit pixel 133 into 2×2 pixels (4 pixels) or more. The light-receiving microlens 132 forms an image on the light-receiving unit pixel 131 of the eye-sensing light reflected by the pupil and incident through the optical system 120 .

Six eye-sensing light projection units 250 are provided around the image light projection unit 240 . The six eye-sensing light projectors 250 are arranged in a substantially circular shape to obtain a circularly arranged Purkinje image. The specifications of the eye-sensing light projection unit 250 are the same as those of the eye-sensing light projection unit 150 according to the first embodiment. Since six Purkinje images can be acquired by providing six eye sensing light projecting units 250, eye sensing can be performed more accurately.

Eight eye sensing light receiving units 230 are provided around the image light projecting unit 240 . The arrangement of the eight eye-sensing light receiving units 230 is not limited to the illustrated example. By arranging the eight eye-sensing light receiving units 230 with more variation, it is possible to adjust wafer efficiency and acquisition signal variation. The size of the eye sensing light receiving section 230 is, for example, 1.6 mm×1 mm. Each eye sensing light receiver 230 includes a plurality of (eg, 8×5) light receiving unit pixels 133 . The size of each light receiving unit pixel 133 (see FIG. 8) is, for example, 200 μm. Each light-receiving unit pixel 133 has a plurality of (for example, 80-division) light-receiving unit sub-pixels 131 (see FIG. 8) separated from each other. The size of the light-receiving unit sub-pixel 131 is, for example, 2.5 μm.

The light-receiving microlens 132 (see FIG. 8) covering the light-receiving unit pixel 133 has, for example, a radius of 0.2157 mm, an aspherical surface, a total lens length of 0.64 mm, and f=0.64 mm. The light-receiving microlens 132 images a maximum angle of 13.5 degrees to 100 μm. The light receiving microlens 132 has a diffraction limit of 2.5 μm and MTF (Modulation Transfer Function)=200 LP/mm. Since the focal length of the light-receiving microlens 132 is short, this configuration can be made thin, which is advantageous for wafer integration of the light-projecting section and the light-receiving section.

As a premise, the viewer's eyebox (the range in which the pupil moves) is φ8 mm. At this time, if the pupil size is φ4 mm, the size of the pupil position is φ12 mm. When f of the eyepiece of the optical system 220 is 18.6, the maximum axial angle θ of the image light projection unit 240 corresponding to a 1.28-inch panel is θ=18.5 deg. The peripheral maximum on-axis angle θ is about 73% and θ=13.5 deg. Assuming that the size of the pupil position is φ12 mm, the light receiving resolution of the eye sensing light receiving section 230 is sufficient with about 80×80 pixels.

FIG. 15 is a graph showing the relationship between the spatial frequency of the light receiving microlens 132 and the contrast.

According to the integrated device 210 having the above configuration, for example, compared to the case of four eye-sensing light projection units of 1 mm and one eye-sensing light-receiving unit of 3 mm, the light amount is +0.1 dB brighter, and the eye sensing becomes more efficient. can be done accurately. By increasing the amount of eye-sensing light, it can also be used in a triple-pass system.

FIG. 17 is a diagram for explaining crosstalk.

Crosstalk is a phenomenon in which an electrical signal leaks to an adjacent element due to leakage of an optical signal incident on a certain element. In order to prevent light leakage in the lateral direction between adjacent eye-sensing light receiving portions 230 (that is, to prevent the occurrence of crosstalk), a wall 34 is provided in the Z direction, or the eye-sensing light receiving portions 230 are sufficiently spaced from each other. It is better to let go.

FIG. 18 is a side view and a partially enlarged view schematically showing the optical system.

The positional relationship of the multi-lens array in the optical system 220 will be explained. When the Z position of the object plane of the image light projecting unit 240, that is, the display surface (approximately the outermost surface) coincides with the Z position of the entrance pupil of the light receiving microlens 132 of the eye sensing light receiving unit 230, a Purkinje image without blurring can be obtained. image can be obtained. On the other hand, if these Z positions are shifted, a blurred Purkinje image will result.

III. Third embodiment

FIG. 19 is a front view schematically showing an integrated device according to the third embodiment. FIG. 33 shows an image display device according to the third embodiment.

The difference between the integrated device 310 according to the third embodiment and the integrated device 110 according to the first embodiment is the arrangement of the eye-sensing light projecting unit 15 (FIG. 33) and the number of the eye-sensing light receiving units 330. and different positions, and the eye-sensing light-receiving part has a plurality of light-receiving unit sub-pixels separated from each other. The specs of the image light projecting section 340 according to the third embodiment are the same as the specs of the image light projecting section 140 according to the first embodiment.

Six eye-sensing light projection units 15 (FIG. 33) are provided around the optical system 12 . The six eye-sensing light projectors 15 (FIG. 33) are arranged in a substantially circular shape in order to obtain a circularly arranged Purkinje image. The eye-sensing light projection unit 15 (FIG. 33) irradiates each Lambertian light. Since six Purkinje images can be acquired by providing six eye sensing light projection units (not shown), eye sensing can be performed more accurately.

Two eye sensing light receiving units 330 are provided around the image light projecting unit 340 . The arrangement of the two eye sensing light receiving units 330 is not limited to the illustrated example. The size of the eye sensing light receiving section 330 is, for example, 18 mm×3 mm. Each eye sensing light receiver 330 includes a plurality of (eg, 90×15) light receiving unit pixels 133 (see FIG. 8). The size of each light receiving unit pixel 133 is, for example, 200 μm. Each light-receiving unit pixel 133 has a plurality of (for example, 80-division) light-receiving unit sub-pixels 131 (see FIG. 8) separated from each other. The size of the light-receiving unit sub-pixel 131 is, for example, 2.5 μm.

The light-receiving microlens 132 (see FIG. 8) covering the light-receiving unit pixel 133 has, for example, a radius of 0.2157 mm, an aspherical surface, a total lens length of 0.64 mm, and f=0.64 mm. The light-receiving microlens 132 images a maximum angle of 13.5 degrees to 100 μm. The light receiving microlens 132 has a diffraction limit of 2.5 μm and MTF (Modulation Transfer Function)=200 LP/mm.

As a premise, the viewer's eyebox (the range in which the pupil moves) is φ8 mm. At this time, if the pupil size is φ4 mm, the size of the pupil position is φ12 mm. When the eyepiece lens f of the optical system 320 is f=18.6, the maximum axial angle θ of the image light projection unit 340 corresponding to a 1.28-inch panel is θ=18.5 deg. The peripheral maximum on-axis angle θ is about 73% and θ=13.5 deg. Assuming that the size of the pupil position is φ12 mm, the light receiving resolution of the eye sensing light receiving section 330 is sufficient with about 80×80 pixels.

FIG. 16 shows an example of image correction.

When the eye-sensing light receiving units 330 acquire an image at a position close to the maximum angle, the obtainable image shifts little by little depending on the image height position of each eye-sensing light receiving unit 330 . To avoid this, the lens shift amount should be changed according to the distance from each central image height. Alternatively, the two types of images shown in the figure may be synthesized by adding a shift in the signal processing stage.

FIG. 20 is a graph showing the relationship between the spatial frequency of the light receiving microlens 132 and the contrast.

According to the integrated device 310 having the above configuration, for example, compared to the case of four eye-sensing light projection units of 1 mm and one eye-sensing light-receiving unit of 3 mm, the light amount is +9.6 dB brighter, and the eye sensing is more effective. can be done accurately. The light-receiving area can be widened, and by increasing the amount of eye-sensing light, a triple-pass system can also be used.

IV. Fourth embodiment

FIG. 21 is a front view schematically showing an integrated device according to the fourth embodiment.

The integrated device 410 according to the fourth embodiment differs from the integrated device 110 according to the first embodiment in the number and positions of the eye-sensing light projecting units 450 and the eye-sensing light receiving units 430, and The eye sensing light receiving part has a plurality of light receiving unit sub-pixels separated from each other. The specs of the image light projecting section 440 according to the fourth embodiment are the same as the specs of the image light projecting section 140 according to the first embodiment.

Six eye-sensing light projection units 450 are provided around the image light projection unit 440 . The six eye-sensing light projectors 450 are arranged in a substantially circular shape to obtain a circularly arranged Purkinje image. The specifications of the eye-sensing light projection unit 450 are the same as those of the eye-sensing light projection unit 150 according to the first embodiment. Since six Purkinje images can be acquired by providing six eye sensing light projecting units 450, eye sensing can be performed more accurately.

Two eye sensing light receiving units 430 are provided around the image light projecting unit 440 . The arrangement of the two eye sensing light receiving units 430 is not limited to the illustrated example. The size of the eye sensing light receiving section 430 is, for example, 12 mm×3 mm. Each eye sensing light receiver 430 includes a plurality of (eg, 60×15) light receiving unit pixels 433 . The size of each light receiving unit pixel 433 is, for example, 200 μm. Each light-receiving unit pixel 433 has a plurality of light-receiving unit sub-pixels 431 (for example, 80 divisions) separated from each other. The size of the light-receiving unit sub-pixel 431 is, for example, 2.5 μm.

The light-receiving microlens 432 covering the light-receiving unit pixel 433 has, for example, a radius of 0.2157 mm, an aspherical surface, a total lens length of 0.64 mm, and f=0.64 mm. The light-receiving microlens 432 images a maximum angle of 13.5 degrees to 100 μm.

FIG. 22 is a front view schematically showing the eye sensing light receiving part. FIG. 23 is a side view schematically showing the eye sensing light receiving section.

Each light-receiving unit pixel 433 has a plurality of light-receiving unit sub-pixels 431 separated from each other. For example, each light-receiving unit pixel 433 is divided into 2×2 pixels (4 pixels) or more, and 3 out of 4 sub-pixels are shielded by the shielding plate 435 . In other words, each light-receiving unit pixel 433 is randomly shielded to a quarter size. By shielding the periphery of the light-receiving unit sub-pixel 431 in this manner, the plurality of light-receiving unit sub-pixels 431 are separated from each other. The pseudo-resolution in this configuration is 160×160 pixels.

According to the integrated device 410 having the above configuration, for example, compared to the case of four eye-sensing light projection units of 1 mm and one eye-sensing light-receiving unit of 3 mm, the light amount is +1.8 dB brighter, and the eye sensing is more effective. can be done accurately. By randomly shielding each light-receiving unit pixel 433, the resolution can be improved and a sufficient amount of eye sensing light can be obtained, thereby facilitating calculation and enabling use in a triple-pass system.

　V. Fifth embodiment

FIG. 24 is a front view schematically showing light-receiving unit pixels included in an eye-sensing light-receiving portion according to the fifth embodiment.

The difference between the integrated device according to the fifth embodiment and the integrated device 410 according to the fourth embodiment is the configuration of the light-receiving unit pixel 533 . The front view of the integrated device according to the fifth embodiment is the same as the front view (FIG. 21) of the integrated device 410 according to the fourth embodiment, so illustration is omitted.

Each light-receiving unit pixel 533 has a plurality of light-receiving unit sub-pixels 531 separated from each other. For example, a plurality of light receiving unit sub-pixels 531 are separated from each other by providing a partition 535 extending in the Z direction between adjacent light receiving unit sub-pixels 531 . The light-receiving unit sub-pixel 531 is configured by dividing each light-receiving unit pixel 533 into 2×2 pixels (4 pixels) or more.

The plurality of light-receiving unit sub-pixels 531A included in some of the light-receiving unit pixels 533A and the plurality of light-receiving unit sub-pixels 531B included in the other portion of the light-receiving unit pixels 533B are offset. For example, a plurality of light-receiving unit sub-pixels 531A included in a portion of light-receiving unit pixels 533A and a plurality of light-receiving unit sub-pixels 531B included in another portion of light-receiving unit pixels 533B are randomly divided into half pixels. offset. The pseudo-resolution in this configuration is 160×160 pixels.

According to the integrated device 410 having the above configuration, for example, compared to the case of four eye-sensing light projection units of 1 mm and one eye-sensing light-receiving unit of 3 mm, the light amount is +7.9 dB brighter, and the eye sensing becomes more effective. can be done accurately. By randomly offsetting each light-receiving unit pixel 533, it is possible to increase the pseudo-resolution while minimizing the loss of the light amount of the eye sensing light. By sufficiently obtaining , it is possible to use it even in a triple-pass system while facilitating the calculation.

　VI. Sixth embodiment

FIG. 25 is a front view schematically showing an integrated device according to the sixth embodiment.

The integrated device 610 according to the sixth embodiment differs from the integrated device 110 according to the first embodiment in the number and positions of the eye-sensing light projecting units 650, and the eye-sensing light receiving units are different from each other. It has a plurality of separated light-receiving unit sub-pixels, a configuration of the image light projecting section 640 , and an eye sensing light receiving section 630 arranged inside the image light projecting section 640 .

Six eye-sensing light projection units 650 are provided around the image light projection unit 640 . The six eye-sensing light projectors 650 are arranged in a substantially circular shape in order to obtain a circularly arranged Purkinje image. The specifications of the eye-sensing light projection unit 650 are the same as those of the eye-sensing light projection unit 150 according to the first embodiment. Since six Purkinje images can be acquired by providing six eye sensing light projecting units 650, eye sensing can be performed more accurately.

FIG. 26 is a front view schematically showing the inner area and the outer area of the image light projecting section.

The size of the image light projection section 640 is the same as the size of the image light projection section 140 of the first embodiment. The image light projector 640 has an inner region 640A and an outer region 640B.

As shown in (A), the inner region 640A of the image light projection unit 640 is configured in an RGB Bayer array. For 4K implementation, for example, the pixel size is 6.7 μm.

As shown in (B), the pixel size of the outer region 640B of the image light projection unit 640 is larger than the pixel size of the inner region 640A. In other words, the outer region 640B has a lower resolution than the inner region 640A. For example, the outer region 640B may be 1/3 of a side with a half angle of 20 degrees, 1/6 of a side with a half angle of 30 degrees, and 1/12 of a side with a half angle of 40 degrees. At the extreme periphery of outer region 640B, for example, the pixel size may be 80 μm.

The eye-sensing light receiving section 630 is provided as a pixel within the outer region 640B of the image light projecting section 640 . Specifically, a plurality of light-receiving unit pixels 633 are scattered in the outer region 640</b>B, and the plurality of light-receiving unit pixels 633 constitute the eye-sensing light receiving section 630 . Each light-receiving unit pixel 633 is divided into a plurality of light-receiving unit sub-pixels 131 (2.5 μm pitch). For example, each light-receiving unit pixel 633 includes 32×32 light-receiving unit sub-pixels 131 (see FIG. 8).

As in FIG. 24, the plurality of light-receiving unit sub-pixels 131 included in some of the light-receiving unit pixels 633 and the plurality of light-receiving unit sub-pixels 131 included in the other portion of the light-receiving unit pixels 633 are offset. good too. For example, the plurality of light-receiving unit sub-pixels 131 included in some of the light-receiving unit pixels 633 and the plurality of light-receiving unit sub-pixels 131 included in the other portion of the light-receiving unit pixels 633 are randomly divided into quarter pixels. can be offset. The pseudo-resolution in this configuration is 128×128 pixels. As in FIG. 24, the offset value may be 1/2.

According to this embodiment, by providing the eye-sensing light receiving unit 630 as a pixel in the outer region 640B of the image light projecting unit 640, the eye-sensing light receiving unit 630 can be arranged further inside, so that the VR optical system can be The image quality is improved because it is less affected by aberrations. In addition, by randomly offsetting each light-receiving unit pixel 633, it is possible to increase the pseudo-resolution while minimizing the loss of the light amount of the eye sensing light. By sufficiently obtaining , it is possible to use it even in a triple-pass system while facilitating the calculation.

VII. Seventh embodiment

FIG. 27 is a front view schematically showing an integrated device according to the seventh embodiment.

The difference between the integrated device 710 according to the seventh embodiment and the integrated device 610 according to the sixth embodiment is that, in addition to the eye sensing light receiving section 730, an eye sensing light receiving section 730 is provided inside the image light projecting section 740. The point is that the light projecting section 750 is arranged.

FIG. 28 is a front view schematically showing the inner area and the outer area of the image light projecting section.

The size of the image light projection section 740 is the same as the size of the image light projection section 140 of the first embodiment. The image light projector 740 has an inner region 740A and an outer region 740B.

As shown in (A), the configuration of the inner region 740A of the image light projecting section 740 is the same as that of the sixth embodiment. The pixel sizes of the inner area 740A and the outer area 740B are the same as in the sixth embodiment.

As shown in (B), the configuration of the eye sensing light receiving section 730 is the same as that of the sixth embodiment. The eye-sensing light projectors 750 are provided as pixels within the outer region 740B of the image light projectors 740 . Specifically, a plurality of eye-sensing light projecting sections 750 are scattered in the outer region 740B. For the eye-sensing light projection unit 750, IR-OLEDs may be mounted instead of IR-LEDs by different coatings for IR of OLEDs.

According to the present embodiment, by providing the eye sensing light projection section 750 as a pixel in the outer region 740B of the image light projection section 740, more accurate eye sensing is possible by switching on/off. Further, by providing the eye-sensing light receiving section 730 as a pixel in the outer region 640B of the image light projecting section 740, the eye-sensing light receiving section 730 can be arranged further inside, so that it is less likely to be affected by the aberration of the VR optical system. Improves image quality. In addition, by randomly offsetting each light-receiving unit pixel 733, it is possible to increase the pseudo-resolution while minimizing the loss of the light amount of the eye sensing light. By sufficiently obtaining , it is possible to use it even in a triple-pass system while facilitating the calculation.

　VIII. Eighth embodiment

The difference between the integrated device according to the eighth embodiment and the integrated device 410 according to the fourth embodiment is the configuration of the eye sensing light receiving section. The front view of the integrated device according to the eighth embodiment is the same as the front view (FIG. 21) of the integrated device 410 according to the fourth embodiment, so illustration is omitted. The same reference numerals as in the fourth embodiment are used for the following description.

The size of the eye sensing light receiving section 430 is, for example, 12 mm x 3 mm. Each eye sensing light receiver 430 includes a plurality of (eg, 2400×600) light receiving unit pixels 433 . The size of each light receiving unit pixel 433 is, for example, 5 μm. Each light-receiving unit pixel 433 has a plurality of (eg, 5-divided) light-receiving unit sub-pixels 431 separated from each other. The size of the light-receiving unit sub-pixel 431 is, for example, 1 μm.

The light-receiving microlens 132 covering the light-receiving unit pixel 133 has, for example, f=16 um and t=16 um. The light-receiving microlens 132 images the maximum angle of 13.5 degrees to 2.5 μm.

According to the integrated device 410 having the above configuration, for example, compared to the case of four eye-sensing light projection units of 1 mm and one eye-sensing light-receiving unit of 3 mm, the light amount is +7.9 dB brighter, and the eye sensing becomes more effective. can be done accurately. By randomly offsetting each light-receiving unit pixel 533, it is possible to increase the pseudo-resolution while minimizing the loss of the light amount of the eye sensing light. By obtaining a sufficient amount of light, it is possible to use it even in a triple-pass system while facilitating calculations. In terms of manufacturing, the microlens array of the light receiving microlenses 132 and the image sensor of the eye sensing light receiving section 430 can be manufactured by a wafer process.

IX. Ninth embodiment

FIG. 30 is a front view schematically showing an integrated device according to the ninth embodiment.

The difference between the integrated device 910 according to the ninth embodiment and the integrated device 410 according to the fourth embodiment is that a plurality of eye sensing light receiving sections 930 of different sizes are provided. Generally, the lower the image height, the better the MTF (Modulation Transfer Function), so it is desirable to arrange the eye sensing light receiving section as close to the center of the image light projecting section as possible. In addition, by arranging a large number of eye-sensing light receiving units, it is possible to avoid direct reflected light and improve accuracy. An image light projection unit 940 and an eye sensing light projection unit 950 are the same as those in the fourth embodiment.

　X. Tenth embodiment

FIG. 31 is a front view schematically showing an integrated device according to the tenth embodiment.

The difference between the integrated device 1010 according to the tenth embodiment and the integrated device 410 according to the fourth embodiment is that the number and positions of the eye sensing light projecting units 1050 are different. In order to avoid direct reflected light, when a large number of eye-sensing light receiving units 1030 are arranged, the interval between eye-sensing light projecting units 1050 is changed to improve accuracy. Further, the eye sensing performance is improved by making the shape of the eye sensing light projecting portion 1050 (LED or OLED) not rectangular but asymmetrical. The image light projecting section 1040 is the same as in the fourth embodiment.

XI. 12th embodiment

FIG. 32 is a front view schematically showing an integrated device according to the eleventh embodiment.

The difference between the integrated device 1110 according to the eleventh embodiment and the integrated device 1010 according to the tenth embodiment is the eye sensing light projecting section 1150 . As the eye-sensing light projection unit 1150, an IR-OLED (LED) is arranged so as to be able to drive the pixels, and an arbitrary pattern is displayed according to the situation, thereby improving eye-sensing accuracy. Furthermore, by using an event-based vision sensor for the eye sensing light receiving unit 1130, the signal of the event-based vision sensor is obtained by controlling the light emission timing of the IR-OLED, thereby performing eye sensing with lower power consumption and higher accuracy. This system has a particularly high affinity with this optical system. When the OLED is placed outside the VR optical system, it is necessary to control the "direction" in order to produce a pattern in the Purkinje image, but such control is difficult. When the OLED is placed in the VR optical system, the positional information is rewritten into the directional information. The image light projector 1140 is the same as in the fourth embodiment.

A polarization sensor may also be used as the eye sensing light receiving section. By using a polarization sensor, unnecessary noise can be reduced and accuracy can be improved.

The present disclosure may have the following configurations.

(1)
an optical system;
an image light projection unit that emits image light that is imaged on the retina through the optical system;
an eye-sensing light projector that emits eye-sensing light that is reflected by the pupil;
an eye-sensing light receiving unit that receives the eye-sensing light reflected by the pupil via the optical system;
a light-receiving microlens for forming an image of the eye-sensing light reflected by the pupil and incident through the optical system on the eye-sensing light-receiving unit;
an integrated device having
and
the light-receiving microlens is not included in the optical path of the image light and is not included in the optical path along which the eye sensing light reaches the pupil;
An image display device, wherein the image light projecting section and the entrance pupil of the light receiving microlens are arranged at the same position in the optical axis direction of the optical system.
(2)
The image display device according to (1) above,
The eye sensing light receiving unit has a cut filter that cuts the image light,
Alternatively, the image display device has the light-receiving microlens for cutting image light.
(3)
The image display device according to (1) or (2) above,
Light projected from the eye-sensing light projecting unit passes through the optical system and is reflected by a pupil, and the eye-sensing light projecting unit and the entrance pupils of the light-receiving microlenses are at the same position in the optical axis direction of the optical system. image display device.
(4)
The image display device according to any one of (1) to (3) above,
The image display device, wherein the eye-sensing light projector has a light-projecting microlens for increasing the amount of the eye-sensing light.
(5)
The image display device according to (4) above,
The image display device, wherein the projection microlens of the eye-sensing light projection unit is decentered so that the direction of the eye-sensing light coincides with the exit angle direction of the image light.
(6)
The image display device according to any one of (1) to (5) above,
A radial direction of the effective image circle from the center of the image light projecting unit is defined as an α direction,
The direction orthogonal to the α direction is the β direction,
Let D be the distance from the center of the image light projection unit to the center of the eye sensing light projection unit,
B is the pupil diameter of the image light projection unit,
Let the emergence angles from D with respect to B be θ _{α+_D} , θ _{α−_D} , θ _{β+_D} , θ _{β−_D} , θ _{α_Center_D} , where θ _{α_Center_D} is the chief ray angle,
Let the projection range of the eye sensing light projection unit be α direction C _αPRJ and β direction C _βPRJ ,
Assuming that the light receiving range angles of the eye sensing light with respect to the light receiving range C are ξα _{+_PRJ} , ξα _{−_PRJ} , ξβ _{+_PRJ} , and ξβ _{−_PRJ} ,
The light-receiving microlens of the eye-sensing light projector is decentered ξα _{+_PRJ} so as to satisfy the equations (1-1′), (1-2′), (1-3′) and (1-4′). −θ _{α_Center_D} ≧C _αPRJ /B・(θ _{α+_D} −θ _{α_Center_D} ) (1−1′)
ξ _{α−_PRJ} −θ _{α_Center_D} ≦C _αPRJ /B・(θ _{α−_D} −θ _{α_Center_D} ) (1-2′)
ξ _{β＋_PRJ} ≧C _{β_D} /B・θ _{β_D} (1−3′)
ξ _{β-_PRJ} ≤ C _{β_D} /B・θ _{β_D} (1-4')
Image display device.
(7)
The image display device according to any one of (1) to (6) above,
The image display device, wherein the light-receiving microlens of the eye-sensing light receiving section is decentered so as to form an image of the eye-sensing light reflected by the pupil at the center of the eye-sensing light-receiving section.
(8)
The image display device according to any one of (1) to (7) above,
A radial direction of the effective image circle from the center of the image light projecting unit is defined as an α direction,
The direction orthogonal to the α direction is the β direction,
Let A be the distance from the center of the image light projecting unit to the center of the eye sensing light receiving unit,
B is the pupil diameter of the image light projection unit,
Let the emergence angles from A with respect to B be θ _{α+_A} , θ _{α−_A} , θ _{β+_A} , θ _{β−_A} , θ _{α_Center_A} , where θ _{α_Center_A} is the chief ray angle,
Let the light receiving range of the eye sensing light receiving portion be α direction C _αDET and β direction C _βDET ,
Assuming that the light receiving range angles of the eye sensing light with respect to the light receiving range C are ξα _{+_DET} , ξα _{−_DET} , ξβ _{+_DET} , and ξβ _{−_DET} ,
The light-receiving microlenses of the eye-sensing light-receiving unit are decentered ξ _{α+_DET} −θ _{α_Center_A} ≧ so as to satisfy the formulas (1-1), (1-2), (1-3) and (1-4). C _αDET /B・(θ _{α+_A} −θ _{α_Center_A} ) (1-1)
ξ _{α−_DET} −θ _{α_Center_A} ≦C _αDET /B・(θ _{α−_A} −θ _{α_Center_A} ) (1-2)
ξ _{β＋_DET} ≧C _{β_A} /B・θ _{β_A} (1-3)
ξ _{β-_DET} ≦ C _{β_A} /B・θ _{β_A} (1-4)
Image display device.
(9)
The image display device according to any one of (1) to (8) above,
The image display device, wherein the image light projecting section is provided between the eye sensing light projecting section and the eye sensing light receiving section and the optical axis in a plane perpendicular to the optical axis of the optical system.
(10)
The image display device according to any one of (1) to (9) above,
the image light projection unit has a rectangular shape on a plane perpendicular to the optical axis of the optical system,
The image display device, wherein the eye sensing light receiving section is positioned within an effective image circle of the optical system on a straight line connecting the center of the long side of the image light projecting section and the optical axis of the optical system.
(11)
The image display device according to any one of (1) to (10) above,
The image display device, wherein the eye-sensing light receiving part includes a plurality of light-receiving unit sub-pixels separated from each other, the light-receiving microlenses are plural, and each microlens forms an image of the eye-sensing light on each light-receiving unit sub-pixel.
(12)
The image display device according to (11) above,
By providing a partition extending in the optical axis direction of the optical system between adjacent light-receiving unit sub-pixels, or by shielding the periphery of each light-receiving unit sub-pixel,
An image display device, wherein the plurality of light-receiving unit sub-pixels are separated from each other.
(13)
The image display device according to (11) or (12) above,
The plurality of light-receiving unit sub-pixels included in some of the light-receiving unit pixels and the plurality of light-receiving unit sub-pixels included in the other portion of the light-receiving unit pixels are offset from each other.
(14)
The image display device according to any one of (1) to (13) above,
The image light projection unit has an inner area and an outer area,
the pixel size of the outer region is larger than the pixel size of the inner region;
The image display device, wherein the eye sensing light receiving section is provided as a pixel in the outer region of the image light projecting section.
(15)
The image display device according to (14) above,
The image display device, wherein the eye-sensing light projecting section is provided as a pixel in the outer region of the image light projecting section.
(16)
An electronic device comprising the image display device according to any one of (1) to (15) above.
(17)
an image light projection unit that emits image light that is imaged on the retina through an optical system;
an eye-sensing light projector that emits eye-sensing light that is reflected by the pupil;
an eye-sensing light receiving unit that receives the eye-sensing light reflected by the pupil via the optical system;
a light-receiving microlens for forming an image of the eye-sensing light reflected by the pupil and incident through the optical system on the eye-sensing light-receiving unit;
and
the light-receiving microlens is not included in the optical path of the image light and is not included in the optical path along which the eye sensing light reaches the pupil;
An integrated device in which the image light projecting section and the entrance pupil of the light receiving microlens are arranged at the same position in the optical axis direction of the optical system.

Although the embodiments and modifications of the present technology have been described above, the present technology is not limited to the above-described embodiments, and various modifications can be made without departing from the gist of the present technology. Of course.

REFERENCE SIGNS LIST 100 image display device 110 integrated device 120 optical system 130 eye-sensing light receiver 131 light-receiving unit sub-pixel 132 light-receiving microlens 133 light-receiving unit pixel 134 cut filter 135 partition 140 image light projector 141 light-emitting cell 142 lens 143 cover glass 144 Color filter 145 Light-emitting layer 150 Eye-sensing light projector 151 LED
152 projection microlens

Claims

an optical system;
an image light projection unit that emits image light that is imaged on the retina through the optical system;
an eye-sensing light projector that emits eye-sensing light that is reflected by the pupil;
an eye-sensing light receiving unit that receives the eye-sensing light reflected by the pupil via the optical system;
a light-receiving microlens for forming an image of the eye-sensing light reflected by the pupil and incident through the optical system on the eye-sensing light-receiving unit;
an integrated device having
and
the light-receiving microlens is not included in the optical path of the image light and is not included in the optical path along which the eye sensing light reaches the pupil;
An image display device, wherein the image light projecting section and the entrance pupil of the light receiving microlens are arranged at the same position in the optical axis direction of the optical system.
The image display device according to claim 1,
The eye sensing light receiving unit has a cut filter that cuts the image light,
Alternatively, the image display device has the light-receiving microlens for cutting image light.
The image display device according to claim 1,
Light projected from the eye-sensing light projecting unit passes through the optical system and is reflected by a pupil, and the eye-sensing light projecting unit and the entrance pupils of the light-receiving microlenses are at the same position in the optical axis direction of the optical system. image display device.
The image display device according to claim 1,
The image display device, wherein the eye-sensing light projector has a light-projecting microlens for increasing the amount of the eye-sensing light.
The image display device according to claim 4,
The image display device, wherein the projection microlens of the eye-sensing light projection unit is decentered so that the direction of the eye-sensing light coincides with the exit angle direction of the image light.
The image display device according to claim 1,
A radial direction of the effective image circle from the center of the image light projecting unit is defined as an α direction,
The direction orthogonal to the α direction is the β direction,
Let D be the distance from the center of the image light projection unit to the center of the eye sensing light projection unit,
B is the pupil diameter of the image light projection unit,
Let the emergence angles from D with respect to B be θ α+_D , θ α−_D , θ β+_D , θ β−_D , θ α_Center_D , where θ α_Center_D is the chief ray angle,
Let the projection range of the eye sensing light projection unit be α direction C αPRJ and β direction C βPRJ ,
Assuming that the light receiving range angles of the eye sensing light with respect to the light receiving range C are ξα +_PRJ , ξα −_PRJ , ξβ +_PRJ , and ξβ −_PRJ ,
The light-receiving microlens of the eye-sensing light projector is decentered ξα +_PRJ so as to satisfy the equations (1-1′), (1-2′), (1-3′) and (1-4′). −θ α_Center_D ≧C αPRJ /B・(θ α+_D −θ α_Center_D ) (1−1′)
ξ α−_PRJ −θ α_Center_D ≦C αPRJ /B・(θ α−_D −θ α_Center_D ) (1-2′)
ξ β＋_PRJ ≧C β_D /B・θ β_D (1−3′)
ξ β-_PRJ ≤ C β_D /B・θ β_D (1-4')
Image display device.
The image display device according to claim 1,
The image display device, wherein the light-receiving microlens of the eye-sensing light receiving section is decentered so as to form an image of the eye-sensing light reflected by the pupil at the center of the eye-sensing light-receiving section.
The image display device according to claim 1,
A radial direction of the effective image circle from the center of the image light projecting unit is defined as an α direction,
The direction orthogonal to the α direction is the β direction,
Let A be the distance from the center of the image light projecting unit to the center of the eye sensing light receiving unit,
B is the pupil diameter of the image light projection unit,
Let the emergence angles from A with respect to B be θ α+_A , θ α−_A , θ β+_A , θ β−_A , θ α_Center_A , where θ α_Center_A is the chief ray angle,
Let the light receiving range of the eye sensing light receiving portion be α direction C αDET and β direction C βDET ,
Assuming that the light receiving range angles of the eye sensing light with respect to the light receiving range C are ξα +_DET , ξα −_DET , ξβ +_DET , and ξβ −_DET ,
The light-receiving microlenses of the eye-sensing light-receiving unit are decentered ξ α+_DET −θ α_Center_A ≧ so as to satisfy the formulas (1-1), (1-2), (1-3) and (1-4). C αDET /B・(θ α+_A −θ α_Center_A ) (1-1)
ξ α−_DET −θ α_Center_A ≦C αDET /B・(θ α−_A −θ α_Center_A ) (1-2)
ξ β＋_DET ≧C β_A /B・θ β_A (1-3)
ξ β-_DET ≦ C β_A /B・θ β_A (1-4)
Image display device.
The image display device according to claim 1,
The image display device, wherein the image light projecting section is provided between the eye sensing light projecting section and the eye sensing light receiving section and the optical axis in a plane perpendicular to the optical axis of the optical system.
The image display device according to claim 1,
the image light projection unit has a rectangular shape on a plane perpendicular to the optical axis of the optical system,
The image display device, wherein the eye sensing light receiving section is positioned within an effective image circle of the optical system on a straight line connecting the center of the long side of the image light projecting section and the optical axis of the optical system.
The image display device according to claim 1,
The image display device, wherein the eye-sensing light receiving part includes a plurality of light-receiving unit sub-pixels separated from each other, the light-receiving microlenses are plural, and each microlens forms an image of the eye-sensing light on each light-receiving unit sub-pixel.
The image display device according to claim 11,
By providing a partition extending in the optical axis direction of the optical system between adjacent light-receiving unit sub-pixels, or by shielding the periphery of each light-receiving unit sub-pixel,
An image display device, wherein the plurality of light-receiving unit sub-pixels are separated from each other.
The image display device according to claim 11,
The plurality of light-receiving unit sub-pixels included in some of the light-receiving unit pixels and the plurality of light-receiving unit sub-pixels included in the other portion of the light-receiving unit pixels are offset from each other.
The image display device according to claim 1,
The image light projection unit has an inner area and an outer area,
the pixel size of the outer region is larger than the pixel size of the inner region;
The image display device, wherein the eye sensing light receiving section is provided as a pixel in the outer region of the image light projecting section.
15. The image display device according to claim 14,
The image display device, wherein the eye-sensing light projecting section is provided as a pixel in the outer region of the image light projecting section.
An electronic device comprising the image display device according to claim 1 .
an image light projection unit that emits image light that is imaged on the retina through an optical system;
an eye-sensing light projector that emits eye-sensing light that is reflected by the pupil;
an eye-sensing light receiving unit that receives the eye-sensing light reflected by the pupil via the optical system;
a light-receiving microlens for forming an image of the eye-sensing light reflected by the pupil and incident through the optical system on the eye-sensing light-receiving unit;
and
the light-receiving microlens is not included in the optical path of the image light and is not included in the optical path along which the eye sensing light reaches the pupil;
An integrated device in which the image light projecting section and the entrance pupil of the light receiving microlens are arranged at the same position in the optical axis direction of the optical system.