JPH0850256A - Display device provided with line-of-sight detection system - Google Patents

Abstract

PURPOSE:To obtain a head mounted display type display device provided with a line-of-sight detection system where video information displayed on a display means is controlled based on information on the line of sight of an observer. CONSTITUTION:This device is provided with an observation system for observing the virtual image of the video information by guiding the video information in a visible area displayed on the display means 4 to the eyeball of an observer without forming an image halfway by the use of an optical system having a reflection surface; and a line-of-sight detection system for detecting the line of sight of the eyeball of the observer 101 by the use of a signal from an image pickup means by making non-visible light from a light source means 102 incident on the eyeball of the observer, allowing the reflected luminous flux from the eyeball to pass through a part of the optical system, and then guiding the light to the surface of the image pickup means by an image-formation optical system 8 provided independently of the optical system. Then, the video information displayed on the display means 4 is controlled by utilizing the video information from the line-of-sight detection system.

Description

Detailed Description of the Invention

[0001]

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention relates to a display device having a line-of-sight detection system, and more particularly to image information (display information) displayed on display means by being mounted on the head of an information viewer (observer). The light is guided to the eyeball of the user to observe the image information.
Glasses type, which is called a so-called head mount display,
In a goggle-type or helmet-type display device, when the image information is displayed and observed by the display means, the image information is obtained by using a signal from a line-of-sight detection system that detects the movement of the eyeball of the observer, that is, the line of sight. And various controls.

[0002]

2. Description of the Related Art Conventionally, it is called a head mount display which is mounted on the head of an information viewer (observer) to guide image information displayed on a display means to an observer's eyeball for observation. Various display devices have been proposed. A line-of-sight detection system, that is, an axis of the direction of the gazing point observed by the observer, a so-called line-of-sight (visual axis), is reflected on the eyeball of the observer on such a display device. For example, Japanese Patent Application Laid-Open No. 3-1 / 1993 discloses a device that is provided with a visual axis detection system that is used for detection and controls visual information displayed on a display device by using visual axis information obtained by the visual axis detection system.
It is proposed in Japanese Patent No. 01709.

In this publication, image information displayed by a display means such as a CRT is imaged on a primary imaging surface by an imaging system, and the image information imaged on the primary imaging surface is passed through an eyepiece system. It uses an optical system designed to be observed. On the other hand, a light source that emits infrared light is provided, and the infrared light from the light source is incident on the eyeball of the observer by utilizing a part of the optical system. Then, the reflected light flux from the eyeball is transmitted to a part of the optical system and infrared light, and is guided to the image pickup device surface through a dichroic mirror that reflects visible light, and an output signal from the image pickup device is used. The line of sight (movement) of the eyeball is detected.

[0004]

In general, a display device as a head mounted display is mounted on the head of an information viewer for personal use, and therefore it is desirable that the entire device be small and lightweight.

The apparatus proposed in the above-mentioned Japanese Patent Laid-Open No. 3-101709 uses a primary image-forming optical system which forms an image of the image information displayed by the display means. For this reason, the visual axis detection system does not require a dedicated imaging system, but the optical system as a whole becomes complicated and tends to be large in size as a device to be mounted on the observer's head.

According to the present invention, the configuration of an observation system for observing image information displayed on the display means of a display device such as a head mounted display and a line-of-sight detection system for detecting the line of sight of an observer provided in a part thereof is appropriately set. Accordingly, it is an object of the present invention to provide a display device having a line-of-sight detection system capable of variously controlling the observation state of video information displayed on the display means of the observation system based on the line-of-sight information while downsizing the entire device And

[0007]

A display device having a line-of-sight detection system of the present invention is: (1-1) An observer using an optical system having a reflecting surface for video information in the visible range displayed by the display means. An observation system for observing a virtual image of the image information by guiding light without forming an image on the eyeball of
Invisible light from the light source means is incident on the eyeball of the observer,
After passing the reflected light flux from the eyeball through a part of the optical system,
An imaging optical system provided independently of the optical system is provided to guide the light onto the surface of the image pickup means, and a line-of-sight detection system for detecting the line of sight of the eyeball of the observer using a signal from the image pickup means is provided. The video information displayed on the display means is controlled by utilizing the line-of-sight information from the line-of-sight detection system.

In particular, (1-1-1) the optical system has a prism body for guiding a light beam from the image information displayed on the display means to an observer's eyeball, and the prism body has a curvature. Must have a reflective surface that performs total internal reflection.

(1-1-2) The prism body allows a light beam from the image information displayed on the display means to enter from an incident surface, and totally reflects the light beam from the incident surface on a front surface having a curvature, The light flux from the front surface is reflected by a concave surface having a curvature, and then passed through a part of the front surface to be guided to the eyeball of the observer.

(1-1-3) The front surface and / or the concave surface of the prism body has a different refractive power depending on the azimuth angle.

(1-1-4) The reflected light beam from the eyeball of the observer is guided to the image forming optical system of the visual axis detection system after passing through at least a part of the prism body and the dichroic mirror surface. .

(1-1-5) The reflected light beam from the eyeball of the observer is guided to the image forming optical system of the visual axis detection system after passing through the dichroic mirror surface.

(1-2) The image information in the visible range displayed by the display means is guided by an optical system having a reflecting surface without being formed on the eyeball of an observer halfway to form a virtual image of the image information. An observation system for observation and an invisible light from the light source means are made incident on the eyeball of the observer, and a light flux reflected from the eyeball is provided independently of the optical system after passing through a part of the optical system. A line-of-sight detection system that guides light onto the surface of the image-capturing means by the image-forming optical system and detects the line-of-sight of the eyeball of the observer using the signal from the image-capturing means, and video information supply means that sends video information to the display means. Has and
The video information supply means is characterized by controlling the video information displayed on the display means based on the visual line information from the visual line detection system.

(1-3) The image information in the visible range displayed by the display means is guided to the eyeball of an observer by using an optical system having a reflecting surface having a curvature and having a total reflection effect. And an observation system for observing the image of the observer, invisible light from the light source means is made incident on the eyeball of the observer, and a reflected light flux from the eyeball is guided to the surface of the image pickup means after passing through a part of the optical system. And a line-of-sight detection system for detecting the line-of-sight of the eyeball of the observer using the signal from the image-pickup unit, and controlling the image information displayed on the display unit by using the line-of-sight information from the line-of-sight detection system. It is characterized by having done.

(1-4) The image information in the visible range displayed by the display means is guided to the eyeball of the observer by using an optical system having a surface having a different refractive power depending on the azimuth angle. An observation system for observing an image, invisible light is made incident on the eyeball of the observer from the light source means, and a reflected light flux from the eyeball is guided to the image pickup means surface through a part of the optical system, A line-of-sight detection system for detecting the line-of-sight of the eyeball of the observer by using a signal from the image-pickup unit, and controlling the video information displayed on the display unit by using the line-of-sight information from the line-of-sight detection system. Is characterized by.

In particular, the line-of-sight detection system is characterized in that it has an image-forming optical system provided independently of the optical system, and that a surface having a different refractive power depending on the azimuth angle is a reflecting surface.

(1-5) An observation system for guiding image information in the visible range displayed by the display means to an eyeball of an observer by using an optical system having a reflecting surface to observe an image of the image information. The invisible light is made incident on the eyeball of the observer from the light source means, the reflected light flux from the eyeball is passed through a part of the optical system, and then the imaging means is provided by an imaging optical system provided independently of the optical system. A line-of-sight detection system that guides light onto a surface and detects the line-of-sight of the eyeball of the observer using the signal from the image-capturing unit, and displays on the display unit by utilizing the line-of-sight information from the line-of-sight detection system The image information to be controlled so that the imaging magnification of the imaging optical system from the eyeball of the observer to the imaging device is β, 0.02 <| β | <0.18 is satisfied. Is characterized by.

[0018]

1 and 2 are cross-sectional views of essential parts showing the optical paths of an observation system and a line-of-sight detection system according to a first embodiment of the present invention. FIG. 3 is a plan view of an essential part of FIG. 4 and 5 are schematic diagrams when the present invention is attached to the head of an observer.

In the figure, 101 is an observer, and 4 is a display means, which is composed of a liquid crystal display element or the like and displays image information in the visible range. The display means 4 displays the video information based on the signal from the video information supply means such as the CD-ROM 105 and the video camera 106. Reference numeral 10 is an optical member formed of a transparent plane parallel plate, and inside thereof is provided a dichroic mirror surface 7 as a beam splitter which transmits visible light and reflects infrared light. Instead of the dichroic mirror surface 7, a simple half mirror surface may be used.

Reference numeral 3 denotes a prism body, which is provided in the prism body 3 with a front surface 1 which is a toric aspherical surface and which utilizes total internal reflection, a rear surface 6 which is a transparent or non-transparent plane or a surface having a curvature. It has a concave surface 2 made of a semi-transparent or specular toric aspherical surface, and an incident surface 5. 104
Is the optical axis (center axis), which coincides with the optical axis of the eyeball 103 described later. Each element in the optical path from the display means 4 to the eyeball 103 constitutes an observation system for observing the virtual image of the image information displayed on the display means 4. Reference numeral 102 denotes a light source means, which emits infrared light (invisible light, wavelength near 880 nm) to the eyeball 103 in order to detect the line of sight of the eyeball 103 of the observer 101.

Reference numeral 8 denotes an image forming optical system (image forming lens), which observer 1 emits infrared light from the light source means 102 as shown in FIG.
When the eyeball 103 of No. 01 is irradiated, the corneal reflection image by the reflected light from the cornea of the eyeball 103 and the imaging position of the pupil and the like are imaged by the CCD or the like via the prism body 3 and the dichroic mirror surface 7 of the optical member 10. An image is formed on the surface of the element 9.
The imaging lens 8 is provided independently of the observation system for observing the virtual image of the image information on the display means 4. Each element in the optical path from the light source means 102 to the image sensor 9 via the eyeball 103 constitutes a visual axis detection system for detecting the visual axis of the eyeball 103 of the observer 101. In the present embodiment, as described above, by setting each element of the observation system and the line-of-sight detection system, the miniaturization of the entire optical system when using two systems is facilitated.

Next, the observation system for observing the virtual image of the video information displayed on the display means 4 will be described with reference to FIG. In this embodiment, a light flux (visible light flux) based on the image information displayed by the display means 4 is passed through the dichroic mirror surface 7 of the optical member 10 and introduced into the prism body 3 through its incident surface 5. Then, after being totally reflected by the front surface 1 of the prism body 3, it is reflected and condensed by the concave surface 2 and passed through the front surface 1 to allow the observer 101
The light is guided to the eyeball 103. At this time, the front surface 1 and the concave surface 2
By properly setting the curvature of the image information, the virtual image of the image information displayed on the display unit 4 is displayed in front of the observer 101 without intermediately forming the image information, that is, without providing the primary image formation surface. ing.

As described above, in this embodiment, the observation system is of the virtual image type so that the observer 101 can observe the virtual image of the image information. In the present embodiment, the concave surface 2 is a semi-transmissive surface, the rear surface 6 is a transmissive surface, and the curvature of the rear surface 6 is appropriately set, so that the image information of the outside scene and the virtual image of the video information of the display unit 4 are spatially separated. You may make it superimpose and observe both as the same diopter in the same visual field.

In the observation system of this embodiment, as shown in FIGS. 4 and 5, the image information from the image information supply means such as the CD-ROM 105 and the video camera 106 possessed by the observer 101 is displayed on the display means 4. At the time of display, using the line-of-sight information of the eyeball of the observer obtained by the line-of-sight detection system, for example, autofocus (focusing of the video camera), electronic zoom (electrically magnifying the line-of-sight direction information), zoom Drive (focal length f of the video camera so that the screen dimensions are extracted by the line of sight
Is calculated and adjusted to the focal length), and menu line-of-sight selection (photometry, strobe, panorama, etc.) is controlled.

Next, the eyeball 10 of the observer 101 will be described with reference to FIG.
A line-of-sight detection system for detecting line-of-sight 3 will be described. The infrared light from the light source means 102 illuminates the eyeball 103 of the observer 101. The infrared light reflected by the cornea of the eyeball 103 passes through the front surface 1 of the prism body 3 and is reflected by the concave surface 2 so that the front surface 1
After being totally reflected by the optical member 1,
Light is guided to 0. Then, the light is reflected by the dichroic mirror surface 7 of the optical member 10, and then the surface 10 of the optical member 10 is reflected.
After being totally reflected by a, the light is incident on the image sensor 9 by the imaging lens 8.

The image forming lens 8 is used because it has no image forming action because the observation system is of a virtual image type. As a result, a corneal reflection image of the eyeball 103 and an image of the eyeball 103 such as a pupil are formed on the surface of the image sensor 9. The line of sight of the eyeball 103 is detected using the signal from the image sensor 9.

As a method of detecting the line of sight of the eyeball in the present embodiment, for example, Japanese Patent Application Laid-Open No. 1-27 previously proposed by the present applicant.
The method disclosed in Japanese Patent No. 4736, Japanese Patent Laid-Open No. 3-11492, etc. is used.

FIGS. 6A and 6B are cross-sectional views of the main part of the prism body 3 used in this embodiment. FIG. 6 (A),
(B) shows the case where the prism body 3 is used as an observation system, but when it is used as a line-of-sight detection system, the optical paths are only reversed and the optical action is the same. The light beam 4a emitted perpendicularly from the display surface of the display means 4 passes through the incident surface 5 of the prism body 3 and enters the front surface 1 of the toric aspherical surface at an incident angle 43.
The light is made incident so as to be totally reflected on the front surface 1 at an angle of more than 1 degree. The light beam 4a totally reflected by the front surface 1 is reflected by the concave surface 2 formed of a toric aspherical surface at an incident angle of 43 degrees or less and emitted from the front surface 1.

The front surface 1 has a curvature so that a part of the front surface 1 has a total reflection effect and another part has a transmission effect. This is equivalent to having two curved surfaces, and the concave surface 2
Together with this, a reflective optical system having three curvatures as a whole is configured. As a result, the focal length of the entire optical system is shortened (20 to 25 mm in a numerical example described later), and the overall size of the optical system is reduced.

In this embodiment, a toric surface or a toric aspherical surface or an anamorphic aspherical surface having different refractive power depending on the azimuth angle, that is, a curvature different depending on the azimuth angle is used as the front surface 1, the concave surface 2 and the incident surface 5 in the observation system and the line-of-sight detection system. Applied. As a result, the angle formed between the light ray incident on the concave surface 2 and the reflected light ray is increased to favorably correct the decentering aberration that occurs when the overall size of the optical system is reduced.

The curvatures of the front surface 1 and the rear surface 6 are such that when light passes through both surfaces, it has a meniscus lens shape having a small refractive power. As a result, when observing the image information such as the outside scenery through the rear surface 6, the image information is satisfactorily observed.

The anterior surface 1 has a negative refractive power in the sagittal section to correct various aberrations generated by the positive refractive power of the concave surface 2. Here, the sagittal line is a plane perpendicular to the plane including the optical path from the image center of the display unit where the light is guided to the eyeball center of the design value (the direction perpendicular to the paper surface of FIG. 6).

In this embodiment, the generatrix section of the front surface 1 may also have a negative refracting power, and the same effect can be obtained as if the sagittal section has a negative refracting power. . Here, the generatrix cross section is a plane including the optical path from the image center of the display unit where the light is guided to the eyeball center of the designed value (inside the paper surface of FIG. 6).

As shown in FIG. 6 (B), the angle (tilde angle) formed by the close battle L in the generatrix section at the surface vertex of the front surface 1 and the line m perpendicular to the optical axis 104 of the eyeball and passing through the surface vertex of the front surface 1. ) = Α, | α | ≦ 20 ° (1) As in the conditional expression (1), the angle α is set to be smaller than 20 degrees, so that the virtual image of the image information of the display means 4 and the image information such as the outside scenery are spatially superposed and the distortion when observing both of them. And the prism thickness in the optical axis direction is reduced.

Next, from the display means 4 of this embodiment to the eyeball 103
Other features of the observation system and the line-of-sight detection system having the respective elements (incident surface 5, front surface 1, concave surface 2) provided in the optical path to the above will be described.

(2-1) Imaging lens 8 in this embodiment
The imaging magnification β from the eyeball 103 to the image sensor 9 is 0.02 <| β | <0.18 (2). Here, if the upper limit of conditional expression (2) is exceeded, the magnification of the eyeball image becomes too large and the effective diameter of the image pickup element increases, which is not preferable. If the lower limit of conditional expression (2) is exceeded, the focal length of the line-of-sight detection system will have to be made shorter, and as a result, various aberrations will occur more frequently and a good eyeball image will not be obtained.

(2-2) When the focal lengths of the whole system of the generatrix section and the sagittal section in the observation system are fy and fx, respectively, 0.9 <| fy / fx | <1.1・ (3) Make sure that the following condition is satisfied. As a result, the focal length of the entire system becomes substantially constant at any azimuth angle, and correction of the aspect ratio of the video information displayed on the display means in the generatrix direction and the sagittal direction is unnecessary.

(2-3) When the paraxial curvature radii of the generatrix section and the sagittal section of the concave surface 2 are Ry and Rx, respectively, | Rx | <| Ry | (4) To be satisfied. In order to reduce the size of the observation system, it is necessary to tilt the optical axis of the concave surface in the cross section of the generatrix largely clockwise from the optical axis of the eyeball. Then, many decentration aberrations occur. On the other hand, since the sagittal cross section is decentered at a small number of places, decentration aberrations are less likely to occur. Therefore, in the present embodiment, as indicated by the conditional expression (4), the radius of curvature Ry of the generatrix section is made larger than the radius of curvature Rx of the sagittal section, that is, the refracting power in the generatrix direction is compared with the refracting power in the sagittal direction. It is weakened to reduce the decentering aberration in the generatrix section.

Particularly in this embodiment, it is preferable to set the conditional expression (4) as follows: | Rx / Ry | <0.85 (5) for correction of decentration aberrations.

(2-4) When the entrance surface 5 of the prism body 3 is a toric surface or an anamorphic surface, | Ry5 | <when the paraxial curvature radii of the generatrix section and the sagittal section are Ry5 and Rx5, respectively. | Rx5 | ... (6) The eccentric aberration is relatively small in the generatrix cross section of the entrance surface 5. Therefore, the refracting power of the concave surface 2 and the front surface 1 cannot be made so strong, but the refracting power of the incident surface 5 is made strong, so that the focal length is almost the same at any azimuth angle in the entire observation system. I try to keep it constant.

(2-5) Within the sagittal section, when the light beam is totally reflected by the front surface 1, the refractive power in that area is negative, the refractive power of the concave surface 2 is positive, and that area when it is transmitted in the front surface 1. Good optical performance is obtained by making the refracting power to be negative. Further, when giving the refracting power to the incident surface 5, it is preferable to make the generatrix cross section positive, which makes it possible to compensate for the lack of the positive dioptric power in the generatrix cross section as a whole.

(2-6) In the cross section of the generatrix, when the light beam is totally reflected on the front surface 1, the refracting power in that region is negative and the refracting power of the concave surface 2 is positive, so that good optical performance is obtained. ing. Further, when the refracting power is applied to the entrance surface 5, it is preferable to make the refracting power positive in the sagittal section, and this can reduce the aberration in the sagittal section.

(2-7) In the sagittal section, when the radius of curvature of the concave surface 2 and the area of the front surface 1 when the light flux is totally reflected are Rx1 and Rx2, respectively, and the focal length of the entire system is fx, 0: .1 <| 2fx / Rx1 | <2.0 (7) 0.5 <| 2fx / Rx2 | <2.5 (8) The upper limit value of the conditional expressions (7) and (8) is a direction in which the refractive powers of the curvature radii Rx1 and Rx2 are strong, and conversely, the lower limit value is a direction in which the refractive powers are weak. If the upper limit of conditional expression (7) is exceeded, it becomes difficult to correct distortion. If the lower limit is exceeded, it will be difficult to satisfy the total reflection condition. If the upper limit of conditional expression (8) is exceeded, it becomes difficult to correct astigmatism. On the other hand, when the value goes below the lower limit, the entire optical system becomes large, and the thickness in the direction parallel to the optical axis becomes thick, which is not preferable.

(2-8) In the cross section of the generatrix, the radius of curvature of the concave surface 2 and the area where the light flux of the front surface 1 is totally reflected are Ry.
1, Ry2, when the focal length of the entire system is fy 0 <| 2fy / Ry1 | <1.0 (9) 0.2 <| 2fy / Ry2 | <2.5 ... ... (10). The upper limit value of the conditional expressions (9) and (10) is a direction in which the refractive powers of the curvature radii Ry1 and Ry2 are strong, and conversely the lower limit value is a direction in which the refractive power is weak. If the upper limit of conditional expression (9) is exceeded, it becomes difficult to correct decentering distortion, and if the lower limit is exceeded, it becomes difficult to satisfy the total reflection condition. If the upper limit of conditional expression (10) is exceeded, eccentric astigmatism will occur more often, and if the lower limit of this conditional expression is exceeded, the overall lens length will increase and the overall optical system will increase in size.

(2-9) The concave surface 2 is decentered parallel to the display means 4 side in the generatrix cross section (Y direction) from the optical axis 104 of the eyeball. Thereby, the eccentric distortion aberration within the cross section of the generatrix is suppressed to be small. Parallel eccentricity shift amount at this time (FIG. 6 (B))
When the distance from the optical axis 104 to the surface apex of the concave surface 2) is E, as shown in (5), the decentering distortion aberration is satisfactorily corrected so that 25 ≦ E (11) .

The tilt angle α in the equation (2-10) (1) is set to −15 ° ≦ α ≦ 5 ° (12). This effectively reduces the size of the entire optical system. If the lower limit of conditional expression (12) is exceeded, distortion of image information will increase, and if the upper limit is exceeded, the prism body 3 will be distorted.
This is not good because the thickness in the direction of the optical axis 104 increases.

7 to 10 are schematic views of the essential parts when a part of the visual axis detection system in the vicinity of the prism body 3 of Examples 2 to 5 of the present invention is changed.

In Example 2 of FIG. 7, as compared with Example 1, the optical member 10 is provided between the eyeball 103 of the observer and the prism body 3, and accordingly, the imaging lens 8 and the image pickup element 9 are provided. However, the other configurations are the same. The present embodiment has a feature that the visual axis can be detected with high accuracy because the visual axis detection system has no eccentric surface.

In Example 3 of FIG. 8, as compared with Example 1, the dichroic surface 7 is provided inside the prism body 3 with an inclination.
Along with that, the imaging lens 8 and the image pickup element 9 are different, and the other configurations are the same. In this embodiment, the number of parts is reduced and the overall size of the optical system is further reduced.

In Example 4 of FIG. 9, the optical member 10 is disposed farther from the eyeball 103 than the prism body 3 as compared with Example 1. The concave surface 2 is coated with a dichroic film that reflects visible light and transmits infrared light. Moreover, the optical member 1
0 is semi-transmissive, or is 100% reflective or is provided with a reflecting surface 11 on which a dichroic film is provided with an inclination, and the imaging lens 8 and the image pickup element 9 are provided accordingly, and other configurations are the same. . FIG. 11 shows a schematic diagram when the member of this example is attached to the head of an observer.

Compared with the first embodiment, the fifth embodiment shown in FIG.
A dichroic mirror 7 that transmits visible light and reflects infrared light is provided on the incident surface 5 of the prism body 3 without using the optical member 10 formed of a plane-parallel plate, and the imaging lens 8 and the image sensor 9 are provided accordingly. Are different, and other configurations are the same. FIG. 12 shows a schematic diagram when the member of this embodiment is attached to the observer's head.

The display device having the line-of-sight detection system of each of the above embodiments can be directly applied to a so-called head-up display device.

Next, numerical examples of this embodiment will be shown. In the numerical example, each element is shown as follows with reference to FIGS. (1) Eyeball 103 is the origin of the coordinate system (0, 0) (2) Rays are traced from the eyeball 103, and in the line-of-sight detection system, i = 1 eyeball i = 2 front surface 1 (transmissive surface) i = 3 concave surface 2 i = 4 Front surface 1 (total reflection surface) i = 5 Incident surface 5 i = 6 Incident surface of optical member 10 i = 7 Dichroic surface i = 8 i = 9 Exit surface of optical member 10 i = 10 Incident surface of imaging lens i = 11 exit surface of imaging lens i = 12 image sensor In observation system, i = 8 entrance surface of image information i = 9 display surface of image information (3) TAL is toric aspherical surface AAL is anamorphic aspherical surface There is.

The definition of TAL is that the cross section of the generatrix (Y, Z cross section) is the following aspherical expression.

[0055]

[Equation 1] The sagittal section (X, Z section) is a spherical surface.

The definition formula of AAL is

[0057]

[Equation 2] Is.

The AL used in the present invention is an aspherical surface (rotationally symmetric), and the definition formula of AL is

[0059]

(Equation 3) Is. The surface vertex coordinates Y and Z are (0,0) the surface vertex of the eyeball.
The absolute coordinates when. The meridian section tilt angle is the tilt angle of the optical axis of each surface with respect to the optical axis of the eyeball (the direction opposite to the clock is positive). The reflecting surface (including total reflection) is indicated by M. nd, ν
d is the refractive index of the d-line and the Abbe number. (Numerical Example 1) <Gaze detection system>

[0060]

[Outside 1] <TAL, AL data> TAL2,4: K = 460.670, A = -0.227E-5, B = 0.179E-7, C = -0.45
3E-10, D = 0.429E-13 TAL3: K = 1.105, A = -0.709E-6, B = -0.273E-8, C = -0.19
1E-11, D = 0.631E-15 AL10: K = -3.858, A = 0.851E-2, B = -0.101, C = 0.14
9, D = -0.755E-1 AL11: K = -0.113, A = 0.195, B = -0.590, C = 0.47
1, D = -0.138 (1) α = 0 (5) | Rx1 / Ry1 | = 0.10 (8) 2fx / Rx2 =
-1.09 (11) E = 26.3 (2) ｜ β ｜ = 0.10 | Rx2 / Ry2 | = 0.67 (9) 2fy / Ry1 =
-0.04 (3) ｜ fy / fx ｜ = 1.00 (7) 2fx / Rx1 = -0.88 (10) 2fy / Ry2 =
-0.36 (Numerical example 2) <Gaze detection system>

[0061]

[Outside 2] <TAL, AL data> TAL2,4: K = 460.670, A = -0.227E-5, B = 0.179E-7, C = -0.45
3E-10, D = 0.429E-13 TAL3: K = 1.105, A = -0.709E-6, B = 0.273E-8, C = -0.19
1E-11, D = 0.631E-15 AL6: K = -3.858, A = 0.851E-2, B = -0.101, C = 0.14
9, D = -0.755E-1 AL7: K = -0.113, A = 0.195, B = -0.590, C = 0.47
1, D = -0.138 (1) α = 0 (5) | Rx1 / Ry1 | = 0.10 (8) 2fx / Rx2 =
-1.09 (11) E = 26.3 (2) ｜ β ｜ = 0.05 | Rx2 / Ry2 | = 0.67 (9) 2fy / Ry1 =
-0.04 (3) ｜ fy / fx ｜ = 1.00 (7) 2fx / Rx1 = -0.88 (10) 2fy / Ry2 =
-0.36 (Numerical example 3) <Gaze detection system>

[0062]

[Outside 3] <AAL, AL data> AAL2,4: Ky = -13763.5, AR = -0.170E-4, BR = 0.406E-7, CR = -0.154E
-9, DR = 0.223E-12 Kx = -3.896, AP = -0.245, BP = 0.416E-1, CP = 0.870E-
1, DP = -0.203E-1 AAL3: Ky = 1.238, AR = -0.317E-5, BR = 0.248E-8, CR = -0.179E-
11, DR = 0.608E-15 Kx = 0.279, AP = -0.249, BP = 0.327E-2, CP = -0.192E
-1, DP = 0.181E-1 AAL5: Ky = 6.825, AR = -0.114E-4, BR = -0.402E-6, CR = 0.113E
-8, DR = -0.411E-10 Kx = -1.33E + 6, AP = 0.273E + 1, BP = 0.155E + 1, CP = 0.160E
+1, DP = -0.644 AL10: K = -3.858, A = 0.851E-2, B = -0.101, C = 0.149, D =
-0.755E-1 AL11: K = -0.113, A = 0.195, B = -0.590, C = 0.471, D =
-0.138 (1) α = -10.5 (5) | Rx1 / Ry1 | = 0.01 (8) 2fx / Rx2 =
-1.47 (11) E = 34.8 (2) ｜ β ｜ = 0.12 | Rx2 / Ry2 | = 0.52 (9) 2fy / Ry1 =
-0.02 (3) ｜ fy / fx ｜ = 0.96 (7) 2fx / Rx1 = -1.5 (10) 2fy / Ry2 =-
0.73 (Numerical example 4) <Gaze detection system>

[0063]

[Outside 4] <AAL, AL data> AAL2,4: Ky = -361850, AR = -0.183E-4, BR = 0.381E-7, CR = -0.114E
-9, DR = 0.153E-12 Kx = -13.802, AP = -0.317, BP = -0.602E-1, CP = 0.272E-
1, DP = -0.211E-1 AAL3: Ky = 1.227, AR = -0.209E-5, BR = 0.308E-8, CR = -0.190E-
11, DR = 0.505E-15 Kx = 0.172, AP = 0.472, BP = 0.553E-1, CP = -0.265E-
1, DP = 0.751E-2 AAL5: Ky = 987000, AR = -0.871E-5, BR = -0.264E-6, CR = 0.469E-
13, DR = 0.137E-11 Kx = -70.169, AP = 41.763, BP = -0.395, CP = 0.183E +
2, DP = -0.988 AL7: K = -3.858, A = 0.851E-2, B = -0.101, C = 0.149,
D = -0.755E-1 AL8: K = -0.113, A = 0.195, B = -0.590, C = 0.471,
D = -0.138 (1) α = 1.5 (5) | Rx1 / Ry1 | = 0.005 (8) 2fx / Rx2 =
-1.22 (11) E = 33.1 (2) ｜ β ｜ = 0.10 | Rx2 / Ry2 | = 0.56 (9) 2fy / Ry1 =
-0.46 (3) ｜ fy / fx ｜ = 1.00 (7) 2fx / Rx1 = -0.93 (10) 2fy / Ry2 =
-0.61 (Numerical Example 5) <Gaze detection system>

[0064]

[Outside 5] <AAL, AL data> AAL2,4: Ky = -387540, AR = -0.183E-4, BR = 0.378E-7, CR = -0.117E
-9, DR = 0.158E-12 Kx = -20.897, AP = -0.300, BP = -0.548E-1, CP = 0.326E-
1, DP = -0.228E-1 AAL3: Ky = 1.213, AR = -0.224E-5, BR = 0.305E-8, CR = -0.190E-
11, DR = 0.500E-15 Kx = 0.165, AP = -0.464, BP = 0.630E-1, CP = -0.251E-
1, DP = 0.380E-2 AAL5: Ky = 559.028, AR = -0.675E-5, BR = 0.182E-6, CR = 0.212E
-12, DR = -0.189E-10 Kx = -99429.4, AP = 0.486E + 1, BP = -0.125E + 1, CP = 0.111E
+2, DP = -0.789 AL11: K = -3.858, A = 0.851E-2, B = -0.101, C = 0.149, D =
-0.755E-1 AL12: K = -0.113, A = 0.195, B = -0.590, C = 0.471, D =
-0.138 (1) α = 0.28 (5) | Rx1 / Ry1 | = 0.005 (8) 2fx / Rx2 =
-1.26 (11) E = 33.0 (2) ｜ β ｜ = 0.11 | Rx2 / Ry2 | = 0.55 (9) 2fy / Ry1 =
-0.005 (3) ｜ fy / fx ｜ = 1.00 (7) 2fx / Rx1 = -0.95 (10) 2fy / Ry2 =
-0.69

[0065]

As described above, according to the present invention, the observation system for observing the image information displayed by the display means in the display device such as the head mount display and the line of sight for detecting the line of sight of the observer provided in a part thereof. By properly setting the configuration of the detection system, it is possible to reduce the size of the entire apparatus and to control various observation states of the video information displayed on the display means of the observation system based on the visual line information. It is possible to achieve a display device having.

[Brief description of drawings]

FIG. 1 is a schematic diagram showing an optical path of an observation system of Example 1 of the present invention.

FIG. 2 is a schematic diagram showing an optical path of the line-of-sight detection system according to the first embodiment of the present invention.

FIG. 3 is a schematic diagram showing an optical path of the line-of-sight detection system according to the first embodiment of the present invention.

FIG. 4 is an explanatory view when a display device of the present invention is attached to an observer.

FIG. 5 is an explanatory diagram when the display device of the present invention is attached to an observer.

FIG. 6 is an enlarged explanatory view of a part of FIG.

FIG. 7 is a schematic view of a main part near a prism body according to a second embodiment of the present invention.

FIG. 8 is a schematic view of a main part near a prism body according to a third embodiment of the present invention.

FIG. 9 is a schematic view of a main part near a prism body according to a fourth embodiment of the present invention.

FIG. 10 is a schematic view of a main part near a prism body according to a fifth embodiment of the present invention.

FIG. 11 is a schematic diagram showing optical paths of an observation system and a line-of-sight detection system according to a fourth embodiment of the present invention.

FIG. 12 is a schematic diagram showing optical paths of an observation system and a line-of-sight detection system according to a fifth embodiment of the present invention.

[Explanation of symbols]

 DESCRIPTION OF SYMBOLS 1 Front surface 2 Concave surface 3 Prism body 4 Display means 5 Incident surface 6 Rear surface 7 Dichroic mirror surface 8 Imaging optical system 9 Imaging element 10 Optical member 101 Observer 102 Light source means 103 Eyeball 104 Optical axis 105 Image information supplying means

Claims (15)

[Claims] 1. An observation for observing a virtual image of the image information displayed on the display means by guiding the image information of the image information in an eyeball of an observer without forming an image on the eyeball of the observer by using an optical system having a reflecting surface. System and imaging optics provided independently of the optical system after injecting invisible light from the light source means into the eyeball of the observer and passing the reflected light beam from the eyeball through a part of the optical system. A line-of-sight detection system that guides light onto the surface of the image-pickup unit by a system and detects the line-of-sight of the eyeball of the observer using a signal from the image-pickup unit, A display device having a line-of-sight detection system characterized by controlling video information displayed on a display means. 2. The optical system has a prism body for guiding a light flux from the image information displayed by the display means to an eyeball of an observer, and the prism body has a reflecting surface having a total reflection effect. A display device having the line-of-sight detection system according to claim 1. 3. The prism body makes a light beam from the image information displayed on the display unit incident from an incident surface, totally reflects the light beam from the incident surface on a front surface having a curvature, and The display device having a visual line detection system according to claim 2, wherein after being reflected by a concave surface having a curvature, the light is passed through a part of the front surface and guided to the eyeball of the observer. 4. A display device having a line-of-sight detection system according to claim 3, wherein the front surface and / or the concave surface of the prism body has a different refractive power depending on the azimuth angle. 5. A light flux reflected from an eyeball of an observer is guided to an image forming optical system of the visual axis detection system after passing through at least a part of the prism body and a dichroic mirror surface. A display device having the line-of-sight detection system of item 2. 6. The line-of-sight detection system according to claim 2, wherein a light beam reflected from an eyeball of an observer is guided to an image-forming optical system of the line-of-sight detection system after passing through a dichroic mirror surface. Display device. 7. The condition that 0.02 <| β | <0.18 is satisfied, where β is the imaging magnification of the imaging optical system from the eyeball of the observer to the imaging device. A display device having the line-of-sight detection system according to claim 1. 8. When the angle between the tangent of the generatrix cross section at the surface vertex of the front surface of the prism body and the line perpendicular to the optical axis of the eyeball and passing through the surface vertex of the front surface is α, | α | ≦ 20 ° 4. A display device having a line-of-sight detection system according to claim 3, which satisfies the following condition. 9. An observation for observing a virtual image of the image information displayed on the display means by using an optical system having a reflecting surface to guide the image information of the image information to the observer's eyeball without forming the image on the way. System and imaging optics provided independently of the optical system after injecting invisible light from the light source means into the eyeball of the observer and passing the reflected light beam from the eyeball through a part of the optical system. The system includes a visual line detection system that guides light onto the surface of the image pickup unit and detects a line of sight of the eyeball of the observer using a signal from the image pickup unit, and a video information supply unit that sends video information to the display unit. A display device having a line-of-sight detection system, wherein the image-information supply unit controls the image information displayed on the display unit based on the line-of-sight information from the line-of-sight detection system. 10. An image of the image information displayed on the display means is guided to an eyeball of an observer by using an optical system having a reflecting surface having a curvature and having a total reflection effect. An observation system for observing and invisible light from the light source means are made incident on the eyeball of the observer, and a reflected light flux from the eyeball is guided to the surface of the image pickup means after passing through a part of the optical system. A line-of-sight detection system for detecting the line-of-sight of the eyeball of the observer using a signal from the imaging unit is provided, and the visual information displayed on the display unit is controlled by using the line-of-sight information from the line-of-sight detection system. A display device having a characteristic line-of-sight detection system. 11. The image information in the visible range displayed by the display means is guided to an eyeball of an observer to observe an image of the image information by using an optical system having a surface having a different refractive power depending on the azimuth angle. And an invisible light from the light source means to the eyeball of the observer, and the reflected light flux from the eyeball is guided to the surface of the image pickup means after passing through a part of the optical system. And a line-of-sight detection system for detecting the line-of-sight of the eyeball of the observer using a signal from, and controlling the video information displayed on the display means using the line-of-sight information from the line-of-sight detection system. A display device having a line-of-sight detection system. 12. A display device having a line-of-sight detection system according to claim 1, wherein the line-of-sight detection system has an imaging optical system provided independently of the optical system. 13. A display device having a line-of-sight detection system according to claim 11, wherein the surface having a different refractive power depending on the azimuth angle is a reflecting surface. 14. An observation system for guiding image information in the visible range displayed by a display means to an eyeball of an observer to observe an image of the image information by using an optical system having a reflecting surface, and the observer. After invisible light is made incident on the eyeball from the light source means and the reflected light flux from the eyeball passes through a part of the optical system, the image is formed on the image pickup means surface by an imaging optical system provided independently of the optical system. A visual line detection system that guides light and detects a visual line of the eyeball of the observer using a signal from the image pickup unit, and image information displayed on the display unit by using visual line information from the visual line detection system Control the
A line-of-sight detection system that satisfies the condition 0.02 <| β | <0.18, where β is the imaging magnification of the imaging optical system from the eyeball of the observer to the imaging device. Having a display. 15. A head-up display device using the display device having the line-of-sight detection system according to claim 1, 9, 10, 11, or 14.