JPH11109278A - Video display device - Google Patents

Abstract

PROBLEM TO BE SOLVED: To display a video having the large angle of visibility and extremely high definition in an area gazed by an observer by displaying the 2nd video having high definition and expressing a part in the range of the 1st video by the scanning of light. SOLUTION: In a 1st display device 21; the aggregation of the light supplied from an optical fiber bundle 17 is emitted toward a half mirror 23 as light arrayed on a line and the emitted light scans in a direction perpendicular to the line direction of the light so as to obtain a two-dimensional image. In a 2nd display device 22; the light supplied from an optical fiber 18 is emitted toward the mirror 23 and the emitted light scans in two directions perpendicular to each other so as to obtain the two-dimensional image. The definition of the 2nd video displayed by the scanning of the light is decided according to the diameter and the step width of the scanning light, but it is easy to make the diameter of the light very minute and the step width of scanning small, so that the definition of the 2nd video is made extremely high.

Description

DETAILED DESCRIPTION OF THE INVENTION

[0001]

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention relates to an image display apparatus for displaying an image with a sense of realism, and more particularly, to displaying an image with a large viewing angle partially having a high-definition area, and The present invention relates to a video display device for moving an area with a high level according to the direction of a line of sight of an observer.

[0002]

2. Description of the Related Art A head mounted display (HMD) which is used by being mounted on a head and displaying an image in front of the eyes is widely used in the field of virtual reality and video games. HMD
It is desirable that the liquid crystal display device (L) be small and light in terms of its use form, and be relatively small and light.
In general, a configuration is provided in which a displayed two-dimensional image is enlarged through an observation optical system and provided to an observer.

The main purpose of virtual reality and video games is to provide a viewer with a high sense of realism, and an image display device used for this purpose needs to display images with a large viewing angle and high definition. is there. One way to achieve this is to display a wide-field image and a high-resolution image of a part of the image, combine them into the observer's eyes, and display the high-definition image It is proposed to move according to the direction of the line of sight. this is,
It utilizes the characteristic that the recognition accuracy of human vision is low except for a very narrow area (viewing angle of 3 to 4 °) including a gazing point although the visual field is wide.

A video display device using this method is disclosed in Japanese Patent Application Laid-Open Nos. 7-87374 and 7-236113. In these technologies, a high-definition image is synthesized by combining light of a wide-field image and a high-definition image displayed individually by a half mirror, and guiding the light of a high-definition image to the half mirror by two galvanometer mirrors. The image is moved in two different directions.

[0005]

However, in the conventional configuration in which a high-definition image is displayed as a two-dimensional image and the image light having the spread is guided to a half mirror for synthesis, the viewing angle is increased and the pupil diameter is increased. In order to increase the size, a relatively large galvanometer mirror must be used between the display element and the half mirror. For this reason, it is necessary to secure a large space for these mirrors, and the motor for driving the mirror is inevitably increased in size, making it difficult to reduce the size and weight of the apparatus.

Further, the definition of an image displayed as a two-dimensional image is limited by the pixel density of the display element, and therefore the definition cannot be increased without limit. In particular, when the combined image is enlarged by the observation optical system in order to increase the viewing angle of the entire image, a high-definition image is also enlarged and the definition is reduced. For this reason, it is difficult to achieve both a wide field of view and a high definition of an image at a high level.

SUMMARY OF THE INVENTION The present invention has been made in view of the above problems, and provides a small and lightweight image display apparatus which displays an image having a large viewing angle and an extremely high definition in an area watched by an observer. With the goal.

[0008]

In order to achieve the above object, according to the present invention, there is provided an image displaying a first image and a second image having a high definition representing a part of the first image. In the display device, the second image is displayed by scanning light.

The first image is larger than the second image and represents a wider area. The definition of the second image displayed by the light scanning is determined by the diameter of the light to be scanned and the step width. However, it is easy to make the diameter of the light small and to reduce the step width of the scanning. The definition of the second video can be extremely high. Therefore, even when the first image and the second image are enlarged and provided to the observer, the second image after the enlargement can have sufficiently high definition.

To achieve the above object, the present invention also provides a first image and a second image representing a part of the first image.
A first display means for displaying the first image, and a second display means for scanning the light, the first display means displaying the first image, and moving the second image in accordance with the direction of the line of sight of the observer. Second display means for displaying an image, observation optical system for superimposing the first image and the second image and projecting the image in front of the observer, detection means for detecting the direction of the line of sight of the observer, and detection means Control means for changing the direction of the second display means in accordance with the direction of the line of sight of the observer detected by the control means.

The second display means displays a high-definition second image by scanning light. If the direction of the second display means changes, the second image projected over the first image is displayed. 2
Also changes the position of the image. The detecting means detects the direction of the line of sight of the observer, and the control means directs the second display means in a direction corresponding to the direction of the detected line of sight of the observer, whereby the second
Can always be positioned on the observer's line of sight within the range of the first image.

[0012] The second display means is, specifically, an image 1
A second image is displayed by scanning light representing a point in two different directions. A one-dimensional image is formed by scanning light representing one point in one direction, and a two-dimensional image is formed by scanning the light in another direction.

[0013] The second display means may display the second image by scanning the light representing one line of the image in one direction. It is 1 by the light itself of one line
A two-dimensional image is formed by scanning the two-dimensional image.

The observation optical system preferably has a concave mirror. By projecting an image with a concave mirror, any part of the image intersects with the line of sight substantially perpendicularly, and the light of the second image always enters the observer's eyes from almost directly in front.

[0015]

DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS Hereinafter, an image display device according to the present invention will be described with reference to the drawings. The video display device of the present invention is configured as an HMD, and is used by being mounted on the head. FIG. 9 schematically shows the relationship when an observer observes an image displayed by the image display device of the present invention. The video display device displays the first video V1 having a large viewing angle, and partially displays the second video V1 having a high definition.
Display as 2. These images are virtual images generated by calculation.

The first image V1 is displayed at a position in front of the viewer at a predetermined distance from the observer U, and the display content of the first image V1 matches the orientation of the observer U's head. Change according to the direction. The definition of the first video is not particularly high. The second video V2 is changed according to the direction of the line of sight of the observer U so as to include the point at which the observer U gazes. The viewing angle of the second image is about two to three times the viewing angle at which a person can accurately identify two points.

FIG. 1 shows an image display apparatus 1 according to the first embodiment.
1 shows the entire configuration. The image display device 1 has left and right 1
It comprises a pair of display units 11, a half mirror 12, a concave mirror 13, a line-of-sight direction detection unit 14, a head position direction detection unit 15, and an image generation device 16.
Each part except the image generation device 16 is housed in a main body case (not shown), and the display unit 11 and the image generation device 16
Are connected by an optical fiber bundle 17 and an optical fiber 18. When the main body case is mounted on the observer's head, the left and right half mirrors 12 are located in front of the left and right eyes, and the observer can observe a stereoscopic image with both eyes via these.

[0018] The image generating device 16 includes a light of a first image matching the direction of the head of the observer and a second image of a region of the first image matching the direction of the line of sight of the observer. Generate light. The light of the first image is supplied to the display unit 11 line by line via an optical fiber bundle 17, and the light of the second image is supplied to the display unit 11 one point at a time via an optical fiber 18.

FIG. 2 shows a schematic configuration of the display unit 11 and an optical path from the display unit 11 to the eyes of the observer. The display unit 11 combines a first display device 21 for displaying a first image, a second display device 22 for displaying a second image, and light of the first image and light of the second image. A half mirror 23 is provided.

The first display device 21 includes the optical fiber bundle 1
7 is emitted toward the half mirror 23 as light aligned on a straight line, and the emitted light is scanned in a direction perpendicular to the line direction to emit light.
A two-dimensional image. The second display device 22 includes the optical fiber 1
8 is emitted toward the half mirror 23, and the emitted light is scanned in two directions perpendicular to each other to form a two-dimensional image.

The half mirror 23, the half mirror 12, and the concave mirror 13 form an observation optical system. Half mirror 2
3 reflects the light from the first display device 21 and transmits the light from the second display device 22 to guide them to the half mirror 12. The half mirror 12 reflects these lights toward the concave mirror 13, transmits the light reflected by the concave mirror 13, and guides the light to the eyes of the observer. With this configuration, the first video and the second video, each of which is a two-dimensional image, are combined and projected as a virtual image in front of the observer.

FIG. 3 shows the configuration of the video generation device 16. The image generation device 16 includes an output signal SE of the gaze direction detection unit 14 and an output signal SH of the head position / direction detection unit 15.
The control device 31 calculates the direction of the line of sight and the position and orientation of the head, and performs control related to display based on these, and outputs a signal of the first image having a wide field of view and a second image of the high definition. An image processing device 32 for generating a signal, a one-dimensional shutter array 33, a shutter array driving device 34, and an illumination device 3 for generating light of a first image
5, and red to generate light for the second image,
It has green and blue lasers 36R, 36G, 36B and a laser driver 37.

The image generating device 16 further includes an infrared laser 36I used for detecting the focus of the observer's eye,
Four lenses 38 for condensing the light emitted from each laser and four prisms 39 for guiding the condensed laser light to the optical fiber 18 are provided.

The video processing device 32 stores information on three-dimensional coordinates, colors, and brightness of all display objects in the virtual space, and performs an operation of projecting the objects within the display range onto a plane. Then, the color and brightness of each part of the two-dimensional image are calculated. For the first video, the display range is determined according to the detected orientation of the observer's head. With respect to the second image, the range to be displayed is determined according to the direction of the detected observer's line of sight within the range to be displayed as the first image.

The image processing device 32 generates a signal representing the image one line at a time from the calculated color and brightness of each part of the first image and sequentially supplies the signal to the shutter array driving device 34. In addition, a signal representing the video point by point is generated from the calculated color and brightness of each part of the second video, and is sequentially supplied to the laser driving device 37.

The one-dimensional shutter array 33 includes the illumination device 3
5 is modulated and guided to the optical fiber bundle 17. The shutter array driving device 34 controls the modulation of the individual shutters of the shutter array 33 based on the first video signal provided from the video processing device 32, and converts the light guided to the optical fiber bundle 17 into one line of the video. Light. The shutter array driving device 34 receives the signal S1 representing the scanning timing of the first display device 21 from the display unit 11, and performs the modulation of the shutter array 33 and the blinking of the illumination device 35 in synchronization with this signal. .

According to this control, the light of each line constituting the first image is supplied to the first display device 21 in order, and the supply and scanning are synchronized. It is displayed as a two-dimensional image.

The laser driving device 37 includes the video processing device 32
Laser 36 based on the second video signal provided by
The light emission of R, 36G, and 36B is individually controlled, and the mixed laser light guided to the optical fiber 18 is modulated so as to represent the color and brightness of each point of the image. The laser driving device 37
A signal S2 representing the timing of scanning of the second display device 22
From the display unit 11, and modulates the laser light in synchronization with the signal S2. As a result, the light of each point constituting the second image is sequentially supplied to the second display device 22, and scanning is performed in synchronization with the supply, whereby the second image is displayed as a two-dimensional image.

The laser driving device 37 also controls the emission of the infrared laser 36I, but this laser beam is not directly involved in the displayed image, and has a constant light amount. The detection of the focus of the observer's eye by the laser light will be described later.

FIG. 4 shows the configuration of the first display device 21 for displaying a first image having a wide field of view. The display device 21 includes a condenser lens 41, a half mirror 42, and a total reflection mirror 43.
Consisting of In this figure, the light beams emitted from the optical fiber bundle 17 are arranged in a direction perpendicular to the paper surface. The condensing lens 41 guides the individual light beams constituting the emitted light beam array to the half mirror 42 as convergent light. The half mirror 42 transfers the light from the condenser lens 41 to the mirror 43
, And transmits the light reflected by the mirror 43 to guide it to the combining half mirror 23.

The mirror 43 is a resonance mirror and swings around an axis perpendicular to the plane of the drawing. The direction of light reflection changes in accordance with the direction of the mirror 43, whereby the light beam array is scanned. The swing cycle of the mirror 43 is the frame cycle of the first video.

The projection distance of the first image, that is, the distance from the observer's eye to the virtual image to be observed is determined by the parallelism of each light beam transmitted through the condenser lens 41, and is 3 to 5 here.
m. Each light beam emitted from the optical fiber bundle 17 is set so as to enter the condenser lens 41 with a predetermined diameter spread.
The pupil diameter of the first image determined by the beam diameter after one transmission is
The size is such that a sufficient amount of light always passes through the pupil regardless of the orientation of the observer's eyeball.

FIG. 5 shows the configuration of the second display device 22 for displaying a high-definition second image. The display device 22 includes a condenser lens 51, a lens position adjusting device 52 that changes the position of the condenser lens 51, a circularly polarized light selection filter 53, a half mirror 54, and a scanning device 55 including two mirrors 55a and 55b. .

The laser light emitted from the optical fiber 18 is incident on the condenser lens 51 with a spread, and is converged by the condenser lens 51. The light transmitted through the condenser lens 51 enters the circular polarization selection filter 53. The circularly polarized light selection filter 53 transmits only clockwise circularly polarized light and guides it to the half mirror 54. The half mirror 54 reflects clockwise circularly polarized light from the filter 53 toward the scanning device 55. This reflection turns the light into left-handed circularly polarized light.

The first mirror 55a of the scanning device 55 has a characteristic of reflecting clockwise circularly polarized light and transmitting counterclockwise circularly polarized light, and deflecting the reflected light in a first direction. Further, the second mirror 55b has a characteristic of reflecting both clockwise and counterclockwise circular deflection, and deflecting the reflected light in a second direction perpendicular to the first direction. . The amount of light deflection by these mirrors 55a and 55b can be electrically adjusted.

The first mirror 55a is a half mirror 54
The left-handed circularly polarized light from is transmitted as it is. The second mirror 55b reflects light transmitted through the first mirror 55a and deflects the light in the second direction. Second mirror 5
The light reflected by 5b becomes clockwise circularly polarized light and is incident on the first mirror.
In the direction of. The light reflected by the first mirror 55a enters the second mirror 55b, is reflected again, and
In the direction of. Due to this reflection, the light becomes counterclockwise circularly polarized light and passes through the first mirror 55a. The half mirror 54 transmits the light transmitted through the first mirror 55a and guides the light to the half mirror 23 for synthesis.

The light constituting the second image is scanned by changing the amount of light deflection in two directions by the scanning device 55. Here, one of the deflections by the mirrors 55a and 55b is defined as a main scanning direction, and the other is defined as a sub-scanning direction. First, the main scanning is shifted by a small angle in the sub-scanning direction so that the image is made high definition in both the main scanning direction and the sub-scanning direction. The sub-scanning cycle is the frame cycle of the second video.

The parallelism of the light beam incident on the observer's eyes varies depending on the distance from the end face of the optical fiber 18 to the condenser lens 51, and the projection distance of the second image is adjusted by the lens position adjusting device 52. It can be adjusted by adjusting the position of. The amount of change in the position of the condensing lens 51 required for adjusting the projection distance
Since the distance between 1 and the optical fiber 18 is short, the distance is small.
Therefore, the luminous flux emitted from the optical fiber 18 is
Irrespective of the projection distance of the second image, the light enters the condenser lens 51 with a substantially constant diameter, and the pupil diameter of the second image determined by the diameter of the light beam transmitted through the condenser lens 51 is equal to the projection distance. It is always large enough without depending.

FIG. 6 shows the configuration of the line-of-sight direction detecting unit 14 for detecting the direction of the line of sight of the observer. The line-of-sight direction detection unit 14 includes a light-emitting diode (L
ED) 61, taking lens 62, and charge-coupled device (C)
CD) 63. The LED 61 emits infrared light toward the observer's eyeball, and the photographing lens 62 forms an image of the light reflected by the eyeball on the CCD 63. CCD63 is LED
It is sensitive only to the infrared light emitted by 61.

The output of the CCD 63 is given to the control device 31 of the image generation device 16, and the control device 31 detects the direction of the eyeball, that is, the direction of the line of sight, from the image of the eyeball represented by the output of the CCD 63. The line-of-sight direction detection unit 14 is disposed below the half mirror 12 located in front of the observer's eye, and photographs the eyeball from obliquely below. For this reason, the CCD 63 is disposed so as to be shifted from the optical axis of the taking lens 62 and perpendicular to the optical axis. This makes it possible to photograph the eyeball without distortion, as in the case of observation from the front.

An arrangement for moving the second image within the first image will be described. As shown in FIG. 5, the second display device 22 for displaying the second image is attached to the attitude control device 19, and the attitude control device 19 turns the display device 22 in an arbitrary direction. Is configured to obtain. Since the display device 22 displays an image by scanning light, the attitude control device 19 changes the direction of the scanning only by changing its direction.
Can be moved to an arbitrary position.

FIG. 7 shows an example of the configuration of the attitude control device 19. The attitude control device 19 holds the second display device 22 downward, and has a rotating portion 71 that can be rotated in the horizontal direction as indicated by an arrow R, and a motor 72 that rotates the rotating portion 71. And a motor 73 that is fixed to the rotating portion 71 and swings the display device 22 about a horizontal axis as shown by an arrow S. The rotation axis and the coaxial axis of the display device 22 intersect at the center point of the scanning device 55 built in the display device 22. The attitude control device 19 having such a configuration can freely tilt the display device 22 with respect to each of the X axis and the Y axis that define the horizontal plane.

FIG. 8 shows the setting of the optical system from the display device 22 to the observer's eyes. Now, it is assumed that the observer's eyeball E rotates about its center C, the intersection of the rotational axis and the swing axis of the display device 22 held by the attitude control device 19 is Q, and the observer turns his / her gaze straight ahead. Optical axis AX of eyeball E when
And the intersection of the half mirror 12 is represented by O. Concave mirror 13
Is a spherical surface centered on the center C of the eyeball E, the distance QO is equal to the distance CO, and the half mirror 12 is
The OQ is fixedly arranged so as to bisect the OQ. When the observer turns his / her gaze straight ahead, the display device 22 is turned to the point O.
Turn to.

With this setting, the relationship between the position of the display device 22 with respect to the concave mirror 13 via the half mirror 12 and the position of the eyeball E with respect to the concave mirror 13 becomes equal. Therefore, when the observer changes the direction of the line of sight by the angle θ, the second image can be positioned on the line of sight of the observer by simply changing the direction of the display device 22 by the same angle θ. It is very easy to control the position of the second video. In addition, even if the second image is moved, the line of sight of the observer is always perpendicular to the second image, and light enters the observer's eyes from directly in front, so that the pupil diameter does not change. .

The control of the direction of the display device 22 is performed simultaneously with the control of the generation of the second video signal by the control device 31 of the video generation device 16 which detects the direction of the visual line from the output of the visual line direction detection unit 14. .

The second image represents a part of the first image with high definition. When this area of the first image and the second image are displayed simultaneously, the first image with relatively low definition is displayed. The video impairs the definition of the second video, making it impossible to observe a high-definition video. Therefore, it is necessary to remove the first video from the display in the area where the second video is displayed.

When the entire area of the first image overlapping the second image is removed from the display, a clear boundary is created between the first image and the second image. Since the second image is set to be larger than the viewing angle at which the human eye has high identification accuracy, even if there is a clear difference in definition at the boundary, the observer recognizes that there is a difference in definition. There is no. However, regarding the brightness, the human eye is sensitive not only in the central part of the visual field but also in the peripheral part. Therefore, if there is a difference in the brightness at the boundary, it is recognized.

For example, if there is a slight error in assembling the apparatus, or even if a slight displacement occurs in the optical system due to an external impact during use, the first image and the second image may overlap. A gap may occur between the first image and the second image. The area where the two images overlap is about twice as bright, and conversely the gap is no longer bright, so the boundary is clearly observed as a bright or dark line, resulting in an unnatural image .

In order to prevent this inconvenience, the image display device 1 removes a range narrower than the second image from the first image, and removes a portion where the first image and the second image overlap each other. Gradually change the brightness of the image. This method is schematically shown in FIG. In FIG. 10, (a) and (b) show a first video V1 and a second video V2, respectively. The removed area E of the first video V1 is smaller than the second video V2,
And the second image V2 has a portion D overlapping each other.

The luminance of the first image V1 outside the part D and the luminance of the second image V2 inside the part D are set so that the same object has the same brightness. Also, the brightness of the part D is
In the first image V1, the voltage gradually decreases from the outside to the inside and becomes 0 at the inner edge. In the second image V2, the voltage gradually decreases from the inside to the outside and becomes 0 at the outer edge. In addition, the first video V1 and the second video V1 are set so as to be equal to the luminance of the portion other than the part D in an overlapping state.

With this setting, even if the first image V1 and the second image V2 are displaced and the width of the portion D is slightly changed, the brightness is kept substantially constant. Therefore, the first
There is no bright line or dark line between the image and the second image, and the image is recognized as a continuous natural image.

The image display device 1 is designed to give a sense of perspective to the image and reduce the load on the eyes of the observer.
The projection distance of the second image is changed according to the distance at which the observer focuses his or her eyes. That is, not only the direction of the line of sight of the observer but also the distance to the gazing point is detected, and the virtual image of the second video is positioned on the gazing point.

Further, the image display device 1 is provided in accordance with the difference between the distance at which the observer is focused on the eye and the distance of the display contents of each part of the second image in order to further enhance the perspective of the image. Then, defocus processing is performed on the second video to be displayed. That is, when the observer's eye is focused ahead of Xm (X meters), the portion of the second video that displays an object within the distance range of (X ± α) m In addition, a clear image without blur is displayed, and a blurred unclear image is displayed for a portion displaying an object outside this range.

In order to detect the focus of the observer's eye, FIG.
As shown in the figure, the second display device 22 has a half mirror 5
7. Cylindrical lens 58 and infrared sensor 5
9 is provided. The half mirror 57 is disposed between the optical fiber 18 and the condenser lens 51, but does not affect the second image because it transmits all the laser light emitted from the optical fiber 18.

The laser light transmitted through the half mirror 57 and emitted from the display device 22 is reflected by the retina of the observer, follows the optical path in the opposite direction, reaches the half mirror 57, and is reflected toward the infrared sensor 59. . The infrared sensor 59
Of the reflected light from the half mirror 57 that is transmitted through the cylindrical lens 58 and incident thereon, only the infrared light emitted by the infrared laser 36I of the image generation device 16 is responsive.

FIG. 11 shows the principle of focus detection of the observer's eye. The infrared sensor 59 has four light receiving elements 59a to 59d.
Are arranged symmetrically. In addition, the cylindrical lens 5
8 is the distance from the half mirror 57 to the half mirror 57
Are arranged at a position equal to the distance from the optical fiber 18 to the end face of the optical fiber 18 and in a direction parallel to a straight line connecting the two opposing light receiving elements.

When the observer's eye is focused on the projected virtual image, the light converged by the condenser lens 51 enters the retina as a point and has no spread on the retina. . The reflected light enters the cylindrical lens 58 as a light beam having a small diameter, and enters the infrared sensor 59 as a circle without being affected by the anisotropy of refraction of the cylindrical lens 58 as shown in B. Therefore, the light receiving amounts of the four light receiving elements 59a to 59d become equal.

On the other hand, when the focus of the observer's eye is in front of or far from the virtual image to be projected, that is, when the observer is in the state of the front focus and the back focus, the light incident on the observer's eye is on the retina. The reflected light becomes a relatively large-diameter light flux, enters the cylindrical lens 58, and is affected by the refraction anisotropy. For this reason, as shown in A and C, the light incident on the infrared sensor 59 becomes elliptical, and the light receiving amount is reduced by the pair of the light receiving elements 59a and 59c and the pair of the light receiving elements 59b and 59d at the opposing positions. There is a difference.

Therefore, based on the magnitude relationship between the amount of light received by the light receiving elements 59a and 59c and the amount of light received by the light receiving elements 59b and 59d, the observer's eyes can be focused on the virtual image, in any of the front focus state and the rear focus state. The light receiving element 59a,
From the magnitude of the difference between the amount of light received by 59c and the amount of light received by light receiving elements 59b and 59d, the degree of front focus or rear focus can be determined.
By adjusting the position of the condenser lens 51 based on this detection result, a virtual image of the second image is projected at a distance where the eye of the observer is in focus.

As described above, when generating a signal representing an image, the image processing device 32 of the image generating device 16 performs an operation of projecting an object within the display range onto a plane. For the second image, the coordinates in the vertical direction with respect to the projected plane are also calculated, and the distance from the observer's eye to the target at each point in the image is obtained. Then, a comparison is made with the in-focus distance of the observer's eye obtained from the output of the focus detection unit 56, and a signal at each point is generated according to the comparison result.

More specifically, when the difference between the distance from the eye to the object and the distance at which the eye is in focus is within a predetermined range, the point is set to the same color and brightness. Such points have a large difference in color and brightness from the surroundings, and a set of them forms a clear image. If the difference exceeds a predetermined range, a weighted average of the color and brightness of the point and the colors and brightness of surrounding points is taken as the color and brightness of the point. In such a point, the difference in color and brightness from the surroundings becomes small, and a set of them forms a blurred image.
By changing the weight at the time of averaging in accordance with the magnitude of the difference in distance, it is possible to adjust blurring.

In this way, a high-definition second image is displayed at a position corresponding to the focal point of the observer's eye, and the image is subjected to defocusing processing according to the distance of the displayed content, so that the observer's gazing point can be determined. The region including the image is regarded as a very natural image. Therefore, the observer can obtain a high sense of reality.

FIG. 12 shows a flow of control relating to video display of the video display device 1. First, based on the output of the head position / direction detection unit 15, the projection coordinate system of the first image with a wide field of view is determined based on the absolute coordinate system of the virtual space (step P1). Based on the outputs of the position direction detection unit 15 and the line-of-sight direction detection unit 14, a high-definition second
(P2). Here, the position of the eyeball determined from the position and orientation of the head is the viewpoint position, the direction of the head is the line of sight of the first image, and the direction of the eyeball is the line of sight of the second image.

Then, based on the calculated viewpoint position and line-of-sight direction and a predetermined projection distance, the contents of the first video and the second video to be displayed are extracted from the stored information of the display object in the virtual space. The calculation is performed (P3, P4) to obtain each two-dimensional video data. At this time, for the first video, a process of removing a range corresponding to the second video is performed, and for the second video, the distance from each viewpoint of the video to the viewpoint is also calculated.

In parallel with the processing of P1 to P4, the distance at which the observer's eyes are in focus is calculated based on the output of the focus detection unit 56 (P5), and this is calculated by projecting the second image. Distance. Then, the second video is subjected to defocus processing based on the difference between the distance from the viewpoint to each point and the projection distance (P
6).

Thus, the preparation for display is completed, the first image is displayed on the first display device 21 (P7), and at the same time, the second image is displayed on the second display device 22 (P7).
8). For the second video, the projection distance is adjusted by the display device 22 (P9), and the display position is adjusted by the posture control device 19 based on the output of the gaze direction detection unit 14 (P9).
Perform 10).

With the above control, the image display device 1 displays not only a wide-field image and a high-definition image of an area in which the observer is looking, but also observes a high-definition image. Is projected to the distance that the person is focusing the eye,
Defocus processing can be performed according to the distance of the display content. Therefore, the observer can always observe an image with a wide field of view, high definition, and perspective, and can obtain a high sense of realism.

The image display device 1 displays a high-definition image by scanning light and changes the position of the high-definition image by changing the direction of the display device, so that a large pupil diameter is secured. In addition, the apparatus can be made small and lightweight.

The detection of the focus of the observer's eye does not necessarily have to be performed independently for the left and right eyes. Either the left eye or the right eye is detected, and based on this, a high-definition image is obtained for both eyes. It is also possible to set the projection distance. Although the projection distance of the wide-field image is fixed here, the wide-field image may be projected at a distance where the observer focuses his or her eyes, similarly to the high-definition image. Further, defocus processing according to the distance of the displayed content may be performed on the wide field of view video.

A video display device according to the second embodiment will be described. In the video display device 1 according to the first embodiment, the high-definition second video is displayed by scanning light representing one point of the video in two directions.
As in the case of the first image having a wide field of view, a high-definition second image is displayed by scanning light representing one line of the image in one direction.

FIG. 13 shows the overall configuration of the video display device 2 of the present embodiment. The image display device 2 includes a pair of left and right display units 111, a half mirror 112, a concave mirror 113, a line-of-sight direction detection unit 114, a head position direction detection unit 115, and an image generation device 116.
Consisting of Among them, the half mirror 112, the concave mirror 113, the line-of-sight direction detection unit 114, and the head position direction detection unit 115 are respectively configured and function similarly to the corresponding parts of the video display device 1 of the first embodiment. Therefore, duplicate description will be omitted.

The display unit 111 and the image generating device 116
Are optical fiber bundles 117 and 118
Connected by The image generation device 116 generates light of the first image that matches the orientation of the observer's head and the first image light.
Of the area of the video that matches the direction of the observer's line of sight
, And supplies the light of the first image to the display unit 111 line by line through the optical fiber bundle 117, and similarly supplies the light of the second image line by line through the optical fiber bundle 118 It is supplied to the display unit 111.

FIG. 14 shows a schematic configuration of the display unit 111 and an optical path from the display unit 111 to the eyes of the observer. The display unit 111 includes a first unit for displaying a first image.
, A second display device 122 that displays a second image, and a half mirror 123 that combines the light of the first image and the light of the second image.

The first display device 121 emits a set of light supplied from the optical fiber bundle 117 toward the half mirror 123 as light arranged in a straight line, and outputs the emitted light with respect to the line direction. Scanning in a vertical direction to form a two-dimensional image. Similarly, the second display device 122
A group of light supplied from the optical fiber bundle 118 is emitted toward the half mirror 123 as light arranged in a straight line, and the emitted light is scanned in a direction perpendicular to the line direction to form a two-dimensional image. And

FIG. 15 shows the configuration of the video generation device 116. The image generation device 116 includes the gaze direction detection unit 11
4 and the output signal SH of the head position / direction detection unit 115, the direction of the line of sight and the position and orientation of the head are calculated, and a control device 131 that performs display-related control based on these is provided. A video processing device 132 for generating a first video signal and a high-definition second video signal of a visual field, a one-dimensional shutter array 133 for generating first video light, and a shutter array driving device 13
4 and a lighting device 135, and a one-dimensional shutter array 136, a shutter array driving device 137, and a lighting device 138 for generating light of a second image.

The image processing device 132 generates a signal representing each line of the image from the color and brightness of each part of the first image calculated according to the direction of the observer's head, and outputs the signal to the shutter array driving device. 134. In addition, a signal representing one line of the image is generated from the color and brightness of each part of the second image calculated according to the direction of the observer's line of sight, and the signal is sequentially supplied to the shutter array driving device 137. However,
The amounts and output periods of the first video signal and the second video signal are set individually because there is a difference in their definition.

The one-dimensional shutter array 133 modulates the light of the illumination device 135 and guides the light to the optical fiber bundle 117. The shutter array driving device 134 controls the modulation of each shutter of the shutter array 133 based on the signal of the first image provided from the image processing device 132 and converts the light guided to the optical fiber bundle 117 into one line of the image. Light. The shutter array driving device 134 receives the signal S1 representing the scanning timing of the first display device 121 from the display unit 111, and performs the modulation of the shutter array 133 and the blinking of the illumination device 135 in synchronization with this signal. .

The one-dimensional shutter array 136 modulates the light of the illumination device 138 and guides the light to the optical fiber bundle 118. The shutter array driving device 137 controls the modulation of each shutter of the shutter array 136 based on the signal of the second image provided from the image processing device 132, and converts the light guided to the optical fiber bundle 118 into one line of the image. Light. The shutter array driving device 137 receives the signal S2 representing the scanning timing of the second display device 122 from the display unit 111, and performs the modulation of the shutter array 136 and the blinking of the illumination device 138 in synchronization with this signal. .

By this control, the first display device 121 and the second display device 122 are provided with the first and second display devices, respectively.
Are sequentially supplied line by line, scanning is performed in synchronization with the supply, and the first and second images are displayed as a two-dimensional image.

The structure of the first display device 121 for displaying the first image having a wide field of view is the same as that shown in FIG. 4, and the description is omitted. FIG. 16 illustrates a configuration of the second display device 122 that displays a high-definition second image. Display device 122
Comprises a condenser lens 151, a half mirror 152, and a total reflection mirror 153. In this figure, the light beams emitted from the optical fiber bundle 118 are arranged in a direction perpendicular to the paper surface, and the arrangement interval of the light beams is set to be extremely small.

The condensing lens 151 converts the individual light beams constituting the emitted light beam array into convergent light by the half mirror 152.
The half mirror 152 reflects the light from the condenser lens 151 toward the mirror 153, transmits the light reflected by the mirror 153, and guides the light to the combining half mirror 123. The mirror 153 is a resonance mirror, and swings around an axis perpendicular to the paper surface. The direction of light reflection changes in accordance with the direction of the mirror 153, whereby the light beam array is scanned. Scanning by the display device 122 is performed in a very short cycle with a small step width in order to make the second image high definition.

The second display device 122 is the attitude control device 1
19. The attitude control device 119 changes the scanning position of the second display device 122 by changing the orientation of the display device 122 with respect to two axes perpendicular to each other by a configuration similar to that shown in FIG. The control of the direction of the display device 122 is performed by the control device 1 of the video generation device 116 that detects the direction of the line of sight from the output of the line of sight direction detection unit 114.
31 so that the second image always matches the direction of the observer's line of sight.

The image display device 2 of the present embodiment can detect the focal point of the observer's eye and change the projection distance of the second image, and can perform defocus processing according to the distance of the display content. Not performed. For this reason, the control flow omits steps P5 and P6 in FIG. 12, and does not calculate the distance from the viewpoint to each point of the video at P4.

By fixing the projection distance of the second image and not performing defocus processing, the high-definition image itself does not have a perspective. In addition, since one direction of the second image is formed by a set of lights representing individual points, there is an upper limit in making the second image high definition.

However, depending on the type of image to be displayed, it is possible to obtain a sufficient sense of realism only by displaying an image with a wide field of view and by setting the line of sight to a certain degree of definition. By giving parallax to the video, the perspective can be expressed, and the usefulness of the video display device 2 is high. Moreover, by displaying the second image by scanning of light representing one line of the image, it is possible to realize, with a simple configuration, a region in the direction of the observer's line of sight in a wide field of view image with high definition. In addition, control for display is easy.

[0086] An image display device according to the third embodiment will be described. This image display device displays a first image with a wide field of view as L
This is displayed as a two-dimensional image on a CD, and a high-definition second image is displayed by scanning light. FIG. 17 shows a schematic configuration and an optical path of the video display device 3 of the present embodiment, and FIG.

The image display device 3 includes an LCD 211, a laser irradiation unit 212, a prism 213, an observation optical system 214 including an eyepiece, and a screen 215.
The LCD 211, the laser irradiation unit 212, the prism 213, and the screen 215 are arranged on a straight line orthogonal to the optical axis of the observation optical system 214.

The LCD 211 is of a transmissive type and is illuminated with a backlight from behind. The surface of the screen 215 is a translucent surface and is translucent. Screen 215 is LC
D211 is disposed substantially in contact with the liquid crystal panel, and directly receives the backlight of the LCD 211 that has passed through the liquid crystal panel, and directly displays the image displayed on the LCD 211. The image displayed on the screen 215 by the LCD 211 is the first image with a wide field of view.

The laser irradiating section 212 includes an emitting section 222 for emitting a laser beam guided by an optical fiber 221 from a light source (not shown) as a parallel beam having a small diameter, and three total reflection mirrors 223, 224, and 225. Injection unit 2
The point-like laser light from 22 is reflected by mirrors 223, 224, and 2
Irradiate the screen 215 through 25. The first mirror 223 is provided to simply bend the optical path to reduce the size of the laser irradiation unit 212, and is fixed.

Second mirror 224 and third mirror 2
Reference numeral 25 denotes a resonance mirror. The former swings about an axis in the vertical direction in the plane of the paper, and the latter swings about an axis perpendicular to the plane of the paper. The irradiation position of the laser beam on the screen 215 changes according to the directions of the second and third mirrors 224 and 225. By swinging these mirrors, a two-dimensional scan of the screen 215 by the laser beam is performed. A high-definition second image is displayed on 215.

The prism 213 has two prisms 213
a and 213b are joined, and the joining surface 213c is a half mirror. The bonding surface 213c transmits the laser light from the laser irradiation unit 212 and
5 is set so as to reflect the laser light reflected by 5, and both the light of the first image and the light of the second image displayed on the screen 215 are reflected toward the observation optical system 214. These image lights are guided to the observer's eyes by the observation optical system 214. The observer observes the real image on the screen 215 via the observation optical system 214 to observe the enlarged virtual image.

The image display device 3 has a line-of-sight direction detection unit 216 for detecting the direction of the observer's eyeball. The line-of-sight direction detection unit 216 includes a light-emitting unit that emits infrared light and a light-receiving unit, receives light from the light-emitting unit reflected by the eyeball at the light-receiving unit, and determines the direction of the eyeball from the position of the pupil appearing in the image. To detect. A prism 217 is provided between the observation optical system 214 and the observer's eyeball for guiding light from the light emitting unit of the gaze direction detection unit 216 to the eyeball and guiding light reflected by the eyeball to the light receiving unit.

The image display device 3 is supplied with an output signal of the line-of-sight direction detection unit 216 indicating the direction of the eyeball of the observer, that is, the direction of the line of sight.
And a control unit for controlling display of the second image and display of the second image by the laser irradiation unit 212. The control unit sets the direction of the center of the swing of the mirrors 224 and 225 so as to scan a part of the area of the screen 215 centered on the line of sight of the observer, and outputs the laser light emitted from the light source to the video signal of the area. And the scanning is performed in synchronization with the video signal.

The control unit also removes the video signal in the area of the screen 215 scanned by the laser irradiation unit 212 from the video signal supplied to the LCD 211,
Blank out part of the video. This is performed according to the method described with reference to FIG.

The screen 215 diffuses the light from the LCD 211 and the light from the laser irradiation unit 212. Therefore, light from each point forming an image on the screen 215 is incident on the observation optical system 214 with a spread, and becomes a light beam having a spread even after passing through the observation optical system 214. For this reason, the image light guided to the observer's eyes has a large pupil diameter, and a sufficient amount of light passes through the pupil even when the center of the pupil is not on the optical axis of the observation optical system 214. Therefore,
The observer can observe a bright image as a whole regardless of the direction in which the observer looks.

In the image display device 3 of this embodiment, light is scanned by changing the directions of the mirrors 224 and 225. Since the pupil diameter is increased by the screen 215, these mirrors are fine. Small-diameter luminous flux 2
Any size is acceptable as long as it can perform dimensional scanning. Therefore,
Small mirrors 224 and 225 can be used, and the device can be made compact.

Although the second image is moved by changing the direction of the mirrors 224 and 225, the laser irradiation unit 212 is held by the above-described attitude control device 19,
The second image may be moved by changing the overall direction of the laser irradiation unit 212. Further, scanning may be performed by the display device 22 of the first embodiment instead of the laser irradiation unit 212.

[0098]

According to the video display device of the first aspect,
In the case where the first video and the second video are provided to the observer without being enlarged, as well as in the case where the first video and the second video are provided in an enlarged manner, the observed second video can have high definition.
That is, since it is possible to expand the entire range of the image without losing the definition of the image range to be high definition,
In order to increase the viewing angle of the first image, it is not necessary to use a large display device as the first display means. Therefore, the device is small and lightweight, and can give a high realism to the observer.

According to the image display device of the second aspect, it is possible to change the position of the second image in the first image simply by changing the direction of the second display means. There is no need to arrange any optical element on the optical path to change the position of the image. In addition, there is no need to move the second display means, and the space occupied by the second display means is small, so that the apparatus can be made compact.

In the image display device according to the third aspect, since the scanning is performed in two directions, the definition of the second image can be arbitrarily set in both the vertical and horizontal directions, and the definition is not uneven. Images can be provided.

In the image display device of the fourth aspect, since scanning is performed in only one direction, the configuration for scanning is simple and control is extremely easy.

In the image observation apparatus according to the fifth aspect, since the light of the second image always enters the observer's eyes from almost directly in front, even when the observer gazes in any direction, a high-definition image of the gazing destination is obtained. There is no blemishes or partial darkening. Therefore, it is possible to provide an image with extremely high sense of reality, which has not only high definition but also unnatural brightness.

[Brief description of the drawings]

FIG. 1 is a diagram illustrating an overall configuration of a video display device according to a first embodiment.

FIG. 2 is a diagram illustrating a schematic configuration of a display unit of the video display device according to the first embodiment and an optical path from the display unit to an eye of an observer.

FIG. 3 is a diagram showing a configuration of a video generation device of the video display device according to the first embodiment.

FIG. 4 is a diagram illustrating a configuration of a display device that displays a wide-field image of the video display device according to the first embodiment.

FIG. 5 is a diagram showing a configuration of a display device for displaying a high-definition video of the video display device according to the first embodiment.

FIG. 6 is a diagram illustrating a configuration of a gaze direction detection unit of the video display device according to the first embodiment.

FIG. 7 is a diagram illustrating an example of a configuration of a posture control device of the video display device according to the first embodiment.

FIG. 8 is a view showing settings of an optical system from a display device for displaying a high-definition video of the video display device of the first embodiment to an eye of an observer.

FIG. 9 is a diagram schematically showing a relationship between an image displayed by the image display device of the present invention and an observer.

FIG. 10 is an exemplary view showing a method of removing a part of a wide-field image corresponding to a high-definition image in the image display device according to the first embodiment.

FIG. 11 is a view showing the principle of detecting the focus of an observer's eye in the video display device according to the first embodiment.

FIG. 12 is a diagram showing a flow of control relating to video display in the video display device of the first embodiment.

FIG. 13 is a diagram illustrating an overall configuration of a video display device according to a second embodiment.

FIG. 14 is a diagram illustrating a schematic configuration of a display unit of an image display device according to a second embodiment and an optical path from the display unit to an eye of an observer.

FIG. 15 is a diagram illustrating a configuration of a video generation device of the video display device according to the second embodiment.

FIG. 16 is a diagram showing a configuration of a display device for displaying a high-definition video of the video display device according to the second embodiment.

FIG. 17 is a diagram illustrating a schematic configuration and an optical path of a video display device according to a third embodiment.

FIG. 18 is a diagram showing an arrangement position of each component of the video display device according to the third embodiment.

[Explanation of symbols]

1, 2, 3 Image display device 11, 111 Display unit 12, 112 Half mirror (observation optical system) 13, 113 Concave mirror (observation optical system) 14, 114 Line-of-sight direction detection unit (detection means) 15, 115 Head position Direction detection unit 16, 116 Image generation device 17, 117, 118 Optical fiber bundle 18 Optical fiber 19, 119 Attitude control device (Control means) 21, 121 First display device (First display device) 22, 122 Second display Apparatus (second display means) 23, 123 Half mirror 31, 131 Controller (detection means, control means) 32, 132 Video processing apparatus 33, 133, 136 One-dimensional shutter array 34, 134, 137 Shutter array drive 35 , 135, 138 Lighting device 36R, 36G, 36B Laser 36I Infrared ray 37 Laser driving device 41, 151 Condensing lens 42, 152 Half mirror 43, 153 Resonating mirror 51 Condensing lens 52 Lens position adjusting device 53 Circular polarization selection filter 54 Half mirror 55 Scanning device 55a, 55b Mirror 56 Focus detecting unit 57 Half Mirror 58 cylindrical lens 59 infrared sensor 61 light-emitting diode 62 photographing lens 63 charge-coupled device 71 rotating unit 72, 73 motor 211 liquid crystal display device (first display means) 212 laser irradiation unit (second display means) 213 prism 213c Prism bonding surface 214 Observation optical system 215 Screen 216 Line-of-sight direction detection unit (detection unit) 217 Prism 221 Optical fiber 222 Emission unit 224, 225 Resonant mirror

Claims (5)

[Claims] 1. An image display device for displaying a first image and a second image having a high definition representing a part of the first image, wherein the second image is displayed by scanning light. A video display device for displaying. 2. An image display apparatus for displaying a first image and a second image representing a part of a range of the first image, and moving the second image in accordance with a direction of a line of sight of an observer. First display means for displaying the first image; second display means for displaying the second image by scanning light; and superimposing the first image and the second image. An observation optical system for projecting the observer in front of the observer, a detector for detecting the direction of the observer's line of sight, and the second display unit A video display device comprising: a control unit for changing a direction. 3. The apparatus according to claim 2, wherein the second display means displays the second image by scanning light representing one point of the image in two different directions. Video display device. 4. The video display according to claim 2, wherein the second display unit displays the second video by scanning light representing one line of the video in one direction. apparatus. 5. The image display device according to claim 2, wherein the observation optical system has a concave mirror.