JP2023032250A - Video display device and control method of the same - Google Patents

Abstract

To provide a smaller and lighter video display device that allows diopter adjustment according to point of fixation.SOLUTION: A video display device 101 comprises: first and second video display part 201a, 201b displaying videos corresponding respectively to a right eye and a left eye of a user; and first and second display optical systems 202a, 202b. First line of sight detection parts 203a, 204a and second line of sight detection parts 203b, 204b detect the line of sight of the user without the first and second display optical system 202a, 202b. The video display device 101 determines a position of a point of fixation of the user from the line of sight detection result, and calculates diopter adjustment amount corresponding to video parallax on the position of the point of fixation. First and second diopter adjustment parts 205a, 205b perform, on the basis of the diopter adjustment amount, diopter adjustment related to the first and second video display part 201a, 201b.SELECTED DRAWING: Figure 2

Description

The present invention relates to a technique for presenting images by guiding different images to the left and right eyeballs of a user.

In a video display device having a user's line of sight detection function, it is possible to adjust the visibility by calculating a visibility adjustment amount corresponding to the parallax of the gaze point position. In Patent Literature 1, line-of-sight detection is performed via a display optical system using a prism. A special prism is employed to ensure that changes in optical power due to diopter adjustment do not affect line-of-sight detection. In addition, since a configuration is adopted in which diopter is adjusted according to the user's gaze point, discomfort during stereoscopic viewing is reduced, and motion sickness during viewing can be suppressed.

JP-A-8-234141

In the conventional technology, when a prism is used, the optical system becomes large and heavy. When a user wears an image display device such as a head-mounted display on his or her head to view an image, it is difficult to wear a device that is heavy and has a large optical system. Even if it is possible to wear it, the fatigue of the user is great, and it is difficult to watch for a medium to long time.
SUMMARY OF THE INVENTION It is an object of the present invention to provide a smaller and lighter image display device while realizing dioptric power adjustment according to the position of the gaze point.

An image display device according to an embodiment of the present invention includes first and second image display units for displaying first and second images to the right eye and left eye of a user, respectively; and first and second display optical systems respectively corresponding to the image display units of and line-of-sight detection related to the first and second image display units without passing through the first and second display optical systems, respectively. and the parallax between the first image and the second image at the position of the gaze point obtained from the detection results of the first and second line-of-sight detection means. and first and second diopter adjustment means for performing dioptric adjustment of the first and second image display units according to the dioptric adjustment amount. .

According to the present invention, it is possible to provide a smaller and lighter image display device while realizing dioptric power adjustment according to the position of the gaze point.

1 is a schematic diagram showing the appearance of a video display device according to an embodiment; FIG. FIG. 2 is a cross-sectional view of the image display device according to the first embodiment when viewed from the top side; FIG. 4 is a schematic diagram illustrating the convergence angle and diopter of an eyeball when gazing; It is a block diagram which shows the structure of the whole system of a video display apparatus. 4 is a flowchart for explaining operations from line-of-sight detection to diopter adjustment; FIG. 11 is a cross-sectional view of the image display device according to the second embodiment when viewed from the upper surface side; FIG. 11 is a cross-sectional view of the image display device according to the third embodiment when viewed from the upper surface side;

Preferred embodiments of the present invention are described in detail below with reference to the accompanying drawings. In the embodiments, a head-mounted display is shown as an example of an image display device capable of displaying a pair of images having parallax on each of a plurality of image display units and performing stereoscopic display via a plurality of display optical systems. .

[First embodiment]
FIG. 1 is a schematic diagram showing the appearance of a video display device according to this embodiment. The image display device 101 has a belt portion 101a to be worn on the head 102 of the user. The user observes the image by the image display device 101 with the right eye 103a and the left eye 103b.

A video display device according to the present embodiment will be described with reference to FIG. FIG. 2 is a schematic cross-sectional view of the video display device 101 viewed from above. In FIG. 2, the reference numerals of the components corresponding to the user's right eye 103a are appended with a, and the reference numerals of the components corresponding to the user's left eye 103b are appended with b.

The image display unit includes an image display unit 201a for the right eye and an image display unit 201b for the left eye. The image display unit is configured by a liquid crystal display, an organic EL (Electro Luminescence) display, or the like. The display optical system 202a is an optical system that guides the image on the image display unit 201a to the right eye 103a, and its optical axis is denoted as La. The display optical system 202b is an optical system that guides the image on the image display unit 201b to the left eye 103b, and its optical axis is denoted as Lb. For the sake of simplification, the display optical systems 202a and 202b in FIG. 2 are both illustrated with a single lens, but various optical systems such as an optical system with a plurality of lenses and an optical system made thin by a Fresnel element can be used. There is a system

The line-of-sight detection camera 203a detects the line of sight of the right eye 103a, and its optical axis is denoted as LGa. The line-of-sight detection camera 203b detects the line of sight of the left eye 103b, and its optical axis is denoted as LGb. The line-of-sight detection cameras 203a and 203b are arranged near the side surfaces of the display optical systems 202a and 202b, respectively, and can capture the user's line of sight without passing through the display optical systems 202a and 202b. The optical axis LGa of the line-of-sight detection camera 203a is inclined at an angle θ1a with respect to the optical axis La of the display optical system 202a that displays an image in front of the right eye 103a. Similarly, the optical axis LGb of the line-of-sight detection camera 203b is inclined at an angle θ1b with respect to the optical axis Lb of the display optical system 202b that displays an image in front of the left eye 103b. Thus, the line-of-sight detection cameras 203a and 203b detect the line of sight without using the display optical systems 202a and 202b. Therefore, the display optical systems 202a and 202b have a simpler structure than the conventional structure, and a lighter and smaller display optical system can be realized.

The infrared illumination unit 204a performs illumination for the line-of-sight detection camera 203a in the right eye 103a to form a corneal reflection image. Similarly, the infrared illumination unit 204b performs illumination for the line-of-sight detection camera 203b in the left eye 103b to form a corneal reflection image. Infrared LEDs (light emitting diodes) or the like are used for the infrared illumination units 204a and 204b. Although only one infrared illumination unit 204a, 204b is shown in FIG. 2, a plurality of infrared illumination units 204a, 204b are actually arranged so as to surround the display optical systems 202a, 202b.

In this way, it is possible to detect the sight lines of the user's right eye and left eye by using the sight line detection camera and the infrared illuminator. That is, the line-of-sight detection cameras 203a and 203b and the infrared illumination units 204a and 204b constitute line-of-sight detection means.

As for the sight line detection method, a known method using a pupil image of an eyeball and a corneal reflection image obtained by infrared illumination is used, so detailed description thereof will be omitted. Also, the line-of-sight detecting means of this embodiment captures an eyeball image from a position tilted with respect to the user's eyes. Therefore, the pupil and the corneal reflection image captured by the line-of-sight detection camera are distorted due to perspective, but the line of sight can be detected satisfactorily by performing correction.

The visibility adjustment unit 205a adjusts visibility by driving the image display unit 201a in the direction of the optical axis La. The visibility adjustment unit 205b adjusts visibility by driving the image display unit 201b in the direction of the optical axis Lb. Dioptric power is adjusted according to the visual acuity of the left and right eyes of the user. The visibility adjustment units 205a and 205b have motors capable of linear driving. Although the motor driving method is not limited, a motor with high quietness is preferable because it is positioned near the user's ear when the image display device 101 is used. A coil motor is used.

The diopter adjustment units 205a and 205b perform diopter adjustment based on the diopter adjustment amount at the position of the gaze point determined from the detection result by the sight line detection means. Diopter is adjusted by changing the respective distances between the image display units 201a and 201b and the display optical systems 202a and 202b. The dioptric power adjustment is performed in real time each time the gaze point is changed, which has the effect of reducing sickness and fatigue of the user when viewing the video. Details thereof will be described with reference to FIG.

FIG. 3 is a schematic diagram for explaining the angle of convergence and diopter of the eyeball during gaze. FIG. 3A shows the state of gaze in the real world, and FIG. 3B shows the state of gaze on the image display device 101 . In FIG. 3A, the observer is gazing at an object point 301 at a distance A1 in the real world. The line of sight Ga1 of the observer's right eye 103a and the line of sight Gb1 of the observer's left eye intersect at the object point 301 to form an angle B1. The angle B1 corresponds to the convergence angle when the observer gazes at the object point 301 . At this time, with respect to the diopter of the observer's eyeball, the crystalline lens changes so that the focus is on the distance A1, and as indicated by rays Da1 and Db1 in the figure, rays emitted from the object point 301 are projected onto the retina in the eyeball. Converge at roughly one point. That is, in the real world, the distance corresponding to the convergence angle B1 and the distance A1 corresponding to diopter always match.

On the other hand, FIG. 3B shows a state in which the observer is gazing at an object point 301 at a position separated by a distance A1 on the image display device 101 . The convergence angle B1 is the same as in the real world shown in FIG. 3(A). The virtual image position 302 is the virtual image position when the observer observes the first image display unit 201a and the second image display unit 201b through the first display optical system 202a and the second display optical system 202b, respectively. is. A virtual image position 302 is a position away from the eyes 103a and 103b by a distance A2 (>distance A1). As for the diopter of the eyeball, the lens changes so that the focus is on the distance A2, and the light rays emitted from the virtual image position 302 converge at approximately one point on the retina in the eyeball, as indicated by light rays Da2 and Db2 in the figure. That is, this is the principle of stereoscopic vision by the video display device 101. By giving the convergence angle B1 to the virtual image position 302, the observer visually perceives as if an object were on the point 301. FIG. However, in the video display device 101, unlike the real world, the distance corresponding to the angle of convergence B1 and the distance A2 corresponding to diopter do not match. This is the so-called contradiction between vergence and diopter adjustment, and is the main cause of sickness and fatigue of observers.

Therefore, in this embodiment, as shown in FIG. 2, by changing the distance between the image display units 201a and 201b and the display optical systems 202a and 202b, the virtual image position 302 in FIG. 3B can be changed. . For example, it is assumed that the line-of-sight detection means detects that the observer is gazing at the point 301 . The distance between the image display unit and the display optical system is changed so that the virtual image position 302 at the distance A2 is at the distance A1. As a result, as shown in FIG. 3(A), the distance corresponding to the angle of convergence and the distance corresponding to the diopter match, enabling more natural observation, and at the same time reducing motion sickness and fatigue. be. When changing the distance between the image display section and the display optical system, at least one of the image display section and the display optical system should be driven. That is, there are embodiments in which the image display section or the display optical system is driven, and embodiments in which the image display section and the display optical system are driven.

Next, the operation of the video display device 101 will be described with reference to FIGS. 4 and 5. FIG. FIG. 4 is a block diagram showing the configuration of the entire system of the image display device 101 of this embodiment. An arithmetic processing unit 401 of the video display device 101 includes a CPU (Central Processing Unit) and the like, and controls the entire system. Image display units 201a and 201b, line-of-sight detection cameras 203a and 203b, infrared illumination units 204a and 204b, and visibility adjustment units 205a and 205b are connected to arithmetic processing unit 401 .

The video display device 101 includes a power supply 402, a radio section 403, and speakers 404a and 404b. By connecting to the network via the wireless unit 403 , the user can view moving image content or the like on the video display device 101 .

Operations from line-of-sight detection to diopter adjustment will be described with reference to the flowchart of FIG. A program corresponding to the flowchart of FIG. First, in S501, sight lines of the user's right eye 103a and left eye 103b are detected using sight line detection cameras 203a and 203b and infrared illumination units 204a and 204b.

Next, in S502, the arithmetic processing unit 401 executes processing for determining a gaze point from the line of sight of the right eye 103a and the line of sight of the left eye 103b. An average value of the sight lines of the right eye and the left eye is used to calculate the gaze point. In S503, the arithmetic processing unit 401 executes processing for calculating the diopter adjustment amount at the position of the gaze point of the images displayed on the image display units 201a and 201b. Specifically, the parallax between the image at the point-of-regard position for the right eye and the image at the point-of-regard position for the left eye is calculated, and the parallax adjustment amount is derived from this parallax. For example, the image display device 101 has a data table that associates parallax and visibility adjustment amount, and can calculate the visibility adjustment amount corresponding to the parallax. Alternatively, the arithmetic processing unit 401 can calculate the diopter adjustment amount from the parallax using a mathematical expression representing the relationship between the parallax and the diopter adjustment amount.

In S504, the motors constituting the visibility adjustment units 205a and 205b are driven based on the calculated visibility adjustment amount. As a result, the operation of changing the distance between the image display units 201a and 201b and the display optical systems 202a and 202b is performed. Since the above operation is performed each time the gaze point is changed, it is possible to change the diopter by adjusting the diopter in real time.

By the way, the human line of sight contains high-frequency micro-movements called fixational eye micro-movements. In S501, smooth line-of-sight detection can be performed by removing the fine movement component. In this embodiment, it is possible to drive the dioptric power adjusters 205a and 205b independently for the right eye and the left eye, so it is possible to accommodate users with different visual acuity in the left and right eyes. Also, the line-of-sight detection cameras 203a and 203b perform line-of-sight detection without using the display optical systems 202a and 202b. Therefore, it is possible to suppress an increase in the size of the display optical system, and achieve line-of-sight detection with a lightweight and compact configuration that is less susceptible to diopter adjustment related to the display optical system.

According to the present embodiment, it is possible to provide a smaller and lighter image display device while realizing dioptric power adjustment according to the position of the gaze point.

[Second embodiment]
A video display apparatus according to a second embodiment of the present invention will be described with reference to FIG. In this embodiment, in order to reduce the angle between the optical axis of the display optical system and the optical axis of the line-of-sight detection camera, a reflecting member that functions as an optical path changing means is arranged between the display optical system and the eye. indicate. This configuration enables line-of-sight detection with higher accuracy. By using the same reference numerals as those of the first embodiment, detailed description thereof will be omitted, and differences will be mainly described. This method of omitting descriptions is the same for the embodiments described later.

FIG. 6 is a schematic cross-sectional view of the video display device 101 of this embodiment as viewed from above. The reflecting members 601a and 601b are arranged between the user's eyes 103a and 103b and the display optical systems 202a and 202b, respectively, and serve as optical path changing means.

In FIG. 6, the optical axis of the optical system of the line-of-sight detection camera 203a for the user's right eye 103a is denoted by LGa2. LGb2 denotes the optical axis of the optical system of the line-of-sight detection camera 203b for the user's left eye 103b. The line-of-sight detection cameras 203a and 203b are arranged at positions closer to the user than the side surfaces of the display optical systems 202a and 202b, respectively, and can catch the line of sight of the user via the reflecting members 601a and 601b. As in the first embodiment, the line-of-sight detection cameras 203a and 203b can capture the user's line of sight without using the display optical systems 202a and 202b.

In this embodiment, the angle formed by the optical axis LGa2 and the optical axis La is denoted by θ2a, and the angle formed by the optical axis LGb2 and the optical axis Lb is denoted by θ2b. The line-of-sight detection camera 203a captures the eye 103a from a position inclined at an angle θ2a with respect to the optical axis La. Similarly, the line-of-sight detection camera 203b captures the eye 103b from a position inclined at an angle θ2b with respect to the optical axis Lb. The angles .theta.2a and .theta.2b are significantly smaller than the angles .theta.1a and .theta.1b shown in the first embodiment. Therefore, the perspective distortion generated in the pupil and corneal reflection image captured by the line-of-sight detection camera is greatly reduced. By doing so, it is possible to obtain an image that is closer to the eyeball seen from the front, so that it is possible to detect the line of sight with higher accuracy. In particular, when the eyeball rotates away from the line-of-sight detection camera 203a or 203b, distortion of the pupil due to perspective increases. According to the configuration provided with the reflecting members 601a and 601b as in the present embodiment, the effect of improving the line-of-sight detection accuracy is large.

The line-of-sight detection cameras 203a and 203b perform line-of-sight detection using infrared images obtained by the infrared illumination units 204a and 204b. The reflecting members 601a and 601b have the property of reflecting infrared rays and transmitting visible light. Accordingly, it is possible to present images to the user while suppressing optical loss in the image display units 201a and 201b. Also in this embodiment, since the line-of-sight detection cameras 203a and 203b perform line-of-sight detection without the intervention of the display optical systems 202a and 202b, the same effect as in the first embodiment is obtained.

[Third embodiment]
A video display device according to a third embodiment of the present invention will be described with reference to FIG. Differences from the second embodiment will be described below. In this embodiment, a configuration using a liquid crystal type optical element capable of changing the focus by electrical switching in the display optical system is shown. With this configuration, the display optical system itself also serves as the diopter adjustment means, which is effective in reducing the size of the image display device.

FIG. 7 is a schematic cross-sectional view of the video display device 101 of this embodiment as viewed from above. The optical elements 701a and 701b constitute a display optical system, and it is possible to change the focal length by changing the alignment characteristics based on the switching of the liquid crystal. Such an optical element includes an element using a Fresnel structure, an element having a combination of polarization characteristics, and the like.

The optical elements 701a and 701b constituting the display optical system not only guide the images of the image display units 201a and 201b to the user's eyes, but also serve as diopter adjusting means. Therefore, since a linear driving motor for changing the distance between the image display units 201a and 201b and the optical elements 701a and 701b is not required, the image display device 101 can be further miniaturized. In this embodiment, the line-of-sight detection cameras 203a and 203b perform line-of-sight detection without using the optical elements 701a and 701b.

According to this embodiment, in addition to the effects of the second embodiment, by using the flat optical elements 701a and 701b whose focal lengths can be changed by electrical signals, a more compact image display device 101 can be realized.

In the above-described embodiment, unlike the conventional example, the optical system does not become large-sized, the structure is lighter and more compact, and line-of-sight detection can be realized without being affected by the diopter adjustment of the display optical system. . In other words, it is possible to realize reduction in size and weight while realizing diopter adjustment according to the position of the gaze point. Although preferred embodiments of the present invention have been described, the present invention is not limited to these embodiments, and various modifications and changes are possible within the scope of the gist.

[Other embodiments]
The present invention supplies a program that implements one or more functions of the above-described embodiments to a system or device via a network or a storage medium, and one or more processors in the computer of the system or device reads and executes the program. It can also be realized by processing to It can also be implemented by a circuit (for example, ASIC) that implements one or more functions.

101 Image display device 201a, 201b Image display unit 202a, 202b Display optical system 203a, 203b, 204a, 204b Line of sight detection unit 205a, 205b Visibility adjustment unit 601a, 601b Reflecting member (optical path changing means)
701a, 701b optical elements

Claims (8)

first and second image display units for displaying first and second images to the right eye and left eye of the user, respectively;
first and second display optical systems respectively corresponding to the first and second image display units;
first and second line-of-sight detection means for performing line-of-sight detection related to the first and second image display units, respectively, without passing through the first and second display optical systems;
a calculating means for calculating a diopter adjustment amount corresponding to the parallax between the first image and the second image at the position of the point of gaze obtained from the detection results of the first and second line-of-sight detecting means;
and first and second visibility adjusting means for adjusting the visibility of the first and second image display units according to the visibility adjustment amount. The first and second image display units display the first and second images having parallax, respectively, and perform stereoscopic display via the first and second display optical systems. The video display device according to claim 1. The first visibility adjustment means adjusts visibility by driving at least one of the first image display unit and the first display optical system,
The second diopter adjustment means adjusts the diopter by driving at least one of the second image display unit and the second display optical system. 3. The video display device according to 2. First and second optical path changing means arranged on the user's side with respect to the first and second display optical systems,
The first line-of-sight detection means performs line-of-sight detection via the first optical path changing means,
The image display device according to any one of claims 1 to 3, wherein the second line-of-sight detection means performs line-of-sight detection via the second optical path changing means. The first and second line-of-sight detection means comprise an infrared illuminator and a line-of-sight detection camera,
5. The image display device according to claim 4, wherein the first and second optical path changing means have characteristics of reflecting infrared rays and transmitting visible light. The first and second display optical systems are composed of flat optical elements whose focal lengths can be changed by electric signals,
3. The image display apparatus according to claim 1, wherein said first and second display optical systems also serve as said first and second visibility adjusting means, respectively. 7. The image display device according to any one of claims 1 to 6, wherein the image display device can be attached to the head of a user. first and second image display units that respectively display first and second images to the right eye and left eye of a user; and first and second image display units corresponding to the first and second image display units, respectively. A control method executed by a video display device comprising a second display optical system,
a line-of-sight detection step of performing line-of-sight detection for each of the first and second image display units without passing through the first and second display optical systems;
a calculation step of calculating a visibility adjustment amount corresponding to the parallax between the first image and the second image at the position of the point of gaze obtained from the detection result of the sight line detection step;
and a visibility adjustment step of performing visibility adjustment for the first and second image display units according to the visibility adjustment amount.

Images