JP2018049039A - Image display apparatus - Google Patents

Abstract

PROBLEM TO BE SOLVED: To provide an image display apparatus capable of allowing an observer to observe images after satisfactorily correcting optical distortion depending on the position of eyes.SOLUTION: The image display system includes: a pair of image display elements 107 and 108; a pair of eyepiece optical systems 105 and 106 that guide light from an original image displayed on the image display element to observer's eyes 103 and 104; and image processing means 102 that carries out a series of correct processing to correct the original image using a correction value depending on optical aberration of the eyepiece optical systems. The image processing means uses the correction value depending on the position of the eyes with respect to the eyepiece optical system for the correct processing.SELECTED DRAWING: Figure 2

Description

  The present invention relates to an image display device such as a head mounted display (HMD).

  In an image display apparatus such as an HMD, an eyepiece optical system that guides light from an original image displayed on an image display element such as an LCD to an observer's eye in order to cause an observer to observe an observation image without distortion or color shift It is desirable that there is little optical aberration such as distortion aberration and lateral chromatic aberration. As a method for allowing an observer to observe an observation image in which the influence of optical aberration of an eyepiece optical system is reduced, Patent Document 1 performs correction processing so as to have a distortion opposite to the distortion caused by the distortion aberration of the eyepiece optical system. An image display device that displays an original image on an image display element is disclosed. In Patent Document 1, the chromatic aberration of magnification is also reduced by performing image processing on the original image according to the distortion aberration for each of the red, green, and blue color components.

  Patent Document 2 discloses an image display device that changes a correction value for an original image related to distortion and lateral chromatic aberration of an eyepiece optical system according to the aberration of an optical system for correcting vision such as an eyeglass of an observer. Yes.

JP 10-327373 A JP 2010-096864 A

  However, the optical aberration of the eyepiece optical system varies depending on the position of the observer's eyes. For example, different amounts and shapes of optical aberrations appear in an observer's eye located near the eyepiece optical system and an observer's eye located far from the eyepiece optical system. In both of the image display devices disclosed in Patent Document 1 and Patent Document 2, correction processing according to the position of the eye of the observer is not performed on the original image, so that an optical according to the position of the eye of the observer is used. An observer cannot observe an observation image in which aberrations are corrected favorably.

  The present invention provides an image processing system that enables observation of an observation image in which the optical distortion in accordance with the position of the observer's eye is well corrected.

  An image display system according to one aspect of the present invention includes an image display element, an eyepiece optical system that guides light from an original image displayed on the image display element to an observer's eye, and optical aberrations of the eyepiece optical system. Image processing means for performing correction processing for correcting the original image using a corresponding correction value. The image processing means uses a correction value corresponding to the eye position with respect to the eyepiece optical system for the correction processing.

  An image processing program according to another aspect of the present invention includes an image display element, an eyepiece optical system that guides light from an original image displayed on the image display element to an eye of an observer, and a computer. In the image display system, there is provided a computer program for causing the computer to perform a correction process for correcting an original image using a correction value corresponding to an optical aberration of the eyepiece optical system. The correction process is performed using a correction value corresponding to the position.

  According to the present invention, it is possible to cause an observer to observe an observation image in which the optical aberration according to the position of the eye of the observer is favorably corrected.

1 is a diagram showing an HMD system that is Embodiment 1 of the present invention. FIG. FIG. 2 is a top view showing an optical configuration in Example 1. FIG. 6 is a diagram for explaining distortion aberration correction processing according to the first embodiment. The figure explaining the case where an eye width differs in Example 1. FIG. The figure explaining the case where eye relief differs in Example 1. FIG. The figure which shows the HMD system which is Example 2 of this invention. FIG. 6 is a side sectional view showing an optical configuration in Example 2. FIG. 6 is a side cross-sectional view showing another optical configuration in Example 2. 3 is a flowchart illustrating image processing performed by the PC in the first embodiment.

  Embodiments of the present invention will be described below with reference to the drawings.

  FIG. 1 shows an HMD 101 and a personal computer (PC) 102 constituting an image display system that is Embodiment 1 of the present invention. The HMD 101 is connected to an image processing unit and a PC 102 serving as an image processing apparatus via a cable (or wireless communication) so as to communicate with each other. The PC 102 generates a right-eye original image and a left-eye original image as image signals to be displayed on the right-eye and left-eye image display elements to be described later in the HMD 101, and communicates these original images to the HMD 101 by communication. Output.

  FIG. 2 shows the optical configuration of the HMD 101. Reference numeral 107 denotes a right eye image display element, and reference numeral 108 denotes a left eye image display element. Reference numeral 105 denotes an eyepiece optical system for the right eye, and reference numeral 106 denotes an eyepiece optical system for the left eye. 103 is an observer's right eye placed on the exit pupil (right exit pupil) of the right eyepiece optical system 105, and 104 is placed on the exit pupil (left exit pupil) of the left eyepiece optical system 106. The left eye of the observer.

  Each of the right-eye image display element 107 and the left-eye image display element 108 is composed of an organic EL element, and displays a right-eye original image and a left-eye original image according to an image signal input from the PC 102. The right-eye eyepiece optical system 105 enlarges a virtual image formed by light from the right-eye original image displayed on the right-eye image display element 107 and guides it to the right eye 103. On the other hand, the left-eye eyepiece optical system 106 enlarges a virtual image formed by light from the left-eye original image displayed on the left-eye image display element 108 and guides it to the left eye 103.

  At this time, when the original image 110 without distortion shown in FIG. 3A is displayed on the image display elements 107 and 108, the observation image 111 that the observer sees through the eyepiece optical systems 105 and 106 is displayed on the eyepiece optical systems 105 and 106. As shown in FIG. 3B, the image is distorted into a barrel shape due to distortion. When an original image including a white line is displayed on the image display elements 107 and 108, the observation image viewed by the observer through the eyepiece optical systems 105 and 106 is red, green, and blue due to the magnification chromatic aberration of the eyepiece optical systems 105 and 106. Are images of lines separated from each other.

  Therefore, in this embodiment, the PC 102 performs image processing (correction processing) so that the original image 110 has a pincushion-shaped distortion opposite to the distortion generated in the eyepiece optical systems 105 and 106 as shown in FIG. To generate a corrected original image 112. Then, the corrected original image 112 is output to the HMD 101 and displayed on the image display elements 107 and 108. Accordingly, the observation image that the observer sees through the eyepiece optical systems 105 and 106 is an image in which distortion due to distortion aberration of the eyepiece optical systems 105 and 106 is reduced. Further, correction processing is performed on the red, green, and blue components in the original image in accordance with the amount of distortion in the eyepiece optical systems 105 and 106 for each color light of red, green, and blue, thereby generating in the eyepiece optical systems 105 and 106. Color shift due to lateral chromatic aberration can also be reduced.

  When the distortion aberration of the eyepiece optical systems 105 and 106 is so small as to be negligible, a correction process for correcting only the lateral chromatic aberration may be performed on the original image. If the chromatic aberration of magnification of the eyepiece optical systems 105 and 106 is small enough to be ignored, a correction process for correcting only distortion aberration may be performed on the original image.

  Here, the relationship between the position of the observer's eyes with respect to the HMD 101 (the eyepiece optical systems 105 and 106) and optical aberrations (distortion aberration and lateral chromatic aberration) will be described. Parameters relating to the position of the observer's eyes with respect to the HMD 101 include an eye width and an eye relief.

  First, the case where the eye width, which is the distance between the right eye 103 and the left eye 104 of the observer, changes with respect to the eyepiece optical systems 105 and 106 is described. FIG. 4A shows a case where the eye width WE of the observer is narrower than the interval (hereinafter referred to as the reference eye width) WA between the optical axes of the eyepiece optical systems 105 and 106, and FIG. Indicates a case where the eye width WE of the observer is wider than the reference eye width WA. 4A and 4B are different from each other in the positions through which light beams guided to the right eye 103 and the left eye 104 of the observer pass in the eyepiece optical systems 105 and 106. When optical aberrations occur in the eyepiece optical systems 105 and 106, the optical aberrations observed by the observer's right eye 103 and left eye 104 are different because the positions where light rays pass through the eyepiece optical systems 105 and 106 are different. The amount and shape change. Therefore, even if the correction process of the original image is performed based on the amount and shape of the optical aberration when the reference eye width WA and the observer's eye width WB are the same as shown in FIG. ) And (b), it is impossible for the observer to observe an observation image in which the optical aberration is corrected satisfactorily.

  Therefore, in this embodiment, a correction value for generating a corrected original image in which distortion or color misregistration due to optical aberrations of the eyepiece optical systems 105 and 106 is corrected in the memory in the PC 102 according to the eye width of the observer. A plurality of correction values are stored in advance. A correction value corresponding to the eye width of the observer can be selected from the plurality of correction values. Thereby, when the eye widths of a plurality of observers are different from each other, it is possible to perform an original image correction process corresponding to the optical aberration corresponding to the eye widths of the observers. For this reason, each of a plurality of observers having different eye widths can observe an observation image in which the influence of optical aberration is further reduced.

  As a method for selecting a correction value corresponding to the eye width of the observer, there are the following methods. First, an observer measures his / her eye width in advance, and information indicating the eye width (information on the eye position) is input devices such as a keyboard and a touch panel provided on the PC 102 (eye position acquisition means and input means). ) To the PC 102. Second, the corrected original images generated by the correction process using different correction values for each eye width are displayed on the image display elements 107 and 108 sequentially or simultaneously. Then, the correction value corresponding to the correction original image selected by the observer who feels that the influence of the optical aberration is the smallest among the correction original images is selected as the correction value used for the subsequent correction processing. In this case, the observer does not have to grasp his / her own eye width. At this time, the correction value used for the correction process may be continuously changed or may be changed stepwise.

  Third, as will be described in detail in the second embodiment, the eye width of the observer is measured (detected) in the HMD 101, and information indicating the measured eye width is automatically input to the PC 102. The PC 102 selects a correction value corresponding to the input eye width information.

  Next, the case where the eye relief, which is the distance between the eyepiece optical systems 105 and 106 and the right eye 103 and the left eye 104 of the observer, changes will be described. FIG. 5A shows a case where the eye relief ER is shorter than the reference eye relief ER1 set in the HMD 101, and FIG. 5B shows a case where the eye relief ER is longer than the reference eye relief ER1. 5A and 5B are different from each other in the positions where the light beams guided to the right eye 103 and the left eye 104 of the observer pass in the eyepiece optical systems 105 and 106. When optical aberrations occur in the eyepiece optical systems 105 and 106, the optical aberrations observed by the observer's right eye 103 and left eye 104 are different because the positions where light rays pass through the eyepiece optical systems 105 and 106 are different. The amount and shape change. For this reason, even if correction processing is performed on the original image based on the amount and shape of the optical aberration observed in the reference eye relief ER1, the optical aberration is corrected satisfactorily in the cases shown in FIGS. The observed image cannot be observed by the observer.

  Therefore, in this embodiment, a plurality of corrections corresponding to eye relief are used as correction values for generating a corrected original image in which distortion or color misregistration due to optical aberrations of the eyepiece optical systems 105 and 106 is corrected in the memory in the PC 102. The value is stored in advance. A correction value corresponding to the eye relief during actual observation can be selected from the plurality of correction values. As a result, when the eye reliefs observed by a plurality of observers are different from each other, it is possible to perform a correction process on the original image corresponding to the optical aberration corresponding to each eye relief. For this reason, each of a plurality of observers observing with different eye reliefs can observe an observation image in which the influence of optical aberration is further reduced.

  As a method for selecting a correction value corresponding to the eye relief, there are the following methods. First, an observer measures eye relief when he / she observes in advance, and inputs information on the eye relief (information on the position of the eye) to the PC 102 through the input device described above. Second, the corrected original images generated by the correction process using different correction values for each eye relief are displayed on the image display elements 107 and 108 sequentially or simultaneously. Then, the correction value corresponding to the correction original image selected by the observer who feels that the influence of the optical aberration is the smallest among the correction original images is selected as the correction value used for the subsequent correction processing. In this case, the observer does not need to grasp the eye relief when he / she observes. At this time, the correction value used for the correction process may be continuously changed or may be changed stepwise. Third, as will be described in detail in the second embodiment, the eye relief when the observer observes in the HMD 101 is measured (detected), and information indicating the measured eye relief is automatically input to the PC 102. To. The PC 102 selects a correction value corresponding to the input eye relief information.

  Here, it is known that the average value of the eye width and eye relief varies depending on the race. For example, Japanese people tend to have wider eyes than Westerners. In addition, Westerners tend to have longer eye relief than Japanese people because Western people have deeper face sculptures than Japanese people. For this reason, it is also possible to store an average eye width or eye relief correction value for each race in advance in the PC 102 and change the correction value according to the race of the observer. At this time, the observer may input his / her race into the PC 102 through the input device described above. Further, the color of the observer's skin and eyes are detected by the color sensor provided in the HMD 101, and the correction value corresponding to the race estimated by the PC 102, that is, the eye position is selected according to the detection result. May be.

  As described above, the information on the eye position is not limited to the above-described eye width and eye relief itself, but includes various information such as race and the like that can specify or estimate the eye position. Further, the method (eye position acquisition means) for acquiring information related to the position of the eye is not limited to the above-described example, and various methods such as acquisition from a photographed image obtained by photographing an observer by image recognition processing. The method can be used.

  In addition, for both eye width and eye relief, a correction value to be used is not selected from a plurality of correction values as described above, but a correction value corresponding to a reference eye width or a reference eye relief is input. A correction value to be used may be generated by correcting according to the eye width or eye relief. Selecting or generating a correction value in this way is also referred to as setting a correction value.

  When the observer uses a vision correction optical system such as eyeglasses, the observer observes not only the eyepiece optical systems 105 and 106 but also an observation image affected by the optical aberration of the vision correction optical system. . In this case, even if the original image is corrected using the correction value for correcting only the optical aberration of the eyepiece optical systems 105 and 106, the observer is allowed to observe an observation image in which the influence of the optical aberration of the vision correction optical system is reduced. I can't.

  Therefore, in this case, a plurality of correction values corresponding to the diopter of the vision correction optical system are stored in the memory in the PC 102, and different correction values are set according to the diopter of the vision correction optical system used by the observer. It is desirable to be able to use it. This allows an observer who uses the vision correction optical system to observe an observation image in which the influence of optical aberration generated in both the eyepiece optical systems 105 and 106 and the vision correction optical system is well corrected. In this case, the observer may measure or hold the diopter of the glasses used by the observer in advance and input it to the PC 102. When the observer does not know the diopter of his / her glasses, the corrected original images generated by the correction process using different correction values for each diopter are displayed on the image display elements 107 and 108 sequentially or simultaneously. . Then, a correction value corresponding to the correction original image selected by the observer who feels that the influence of the optical aberration is least may be selected from among the correction original images. Further, a correction value to be used may be generated by correcting the correction value corresponding to the diopter used as a reference by the PC 102 according to the input diopter.

  In the present embodiment, the correction of the original image displayed on the image display element in the HMD has been described. However, the original image displayed on the image display element in the electronic viewfinder of the camera may be similarly corrected.

  In addition, the eyepiece optical systems 105 and 106 shown in FIGS. 2 and 5 are configured by a single spherical lens, but an aspherical single lens or a plurality of lenses may be used. In this embodiment, the case where an organic EL element is used as the image display elements 107 and 108 has been described. However, other image displays such as a transmissive liquid crystal display element, a reflective liquid crystal display element, and a digital micromirror device (DMD) are used. An element may be used. In these cases, a light source and an illumination optical system for illuminating the image display element are required.

  By using a rotationally symmetric lens as an eyepiece optical system, distortion and lateral chromatic aberration also have a rotationally symmetric shape. Therefore, the correction value used for correcting the original image by the PC 102 is the distance from the center of the image display element. It may be a function as a parameter. Further, when the distortion aberration and the lateral chromatic aberration are not rotationally symmetric due to a manufacturing error or the like, a correction table having a correction value for each pixel of the image display element may be used.

  FIG. 9 shows the flow of original image correction processing performed by the PC 102. The PC 102 executes this correction processing according to an image processing program that is a computer program installed therein. Here, a case will be described in which the PC 102 acquires information on the position of the observer's eyes (eye width, eye relief, or race information: hereinafter referred to as eye position information) through input or measurement from the input device described above.

  First, in step S101, the PC 102 acquires eye position information. Next, in step S102, the PC 102 sets a correction value according to the eye position information acquired in step S101. At this time, when the observer uses the vision correction optical system, a correction value corresponding to the diopter of the vision correction optical system may be further set.

  In step S103, the PC 102 performs a correction process on the original image using the correction value set in step S102 to generate a corrected original image. Finally, in step S104, the PC 102 outputs the corrected original image generated in step S103 to the HMD 101. Accordingly, the observer can observe an observation image in which distortion, color shift, and the like due to optical aberrations of the eyepiece optical systems 105 and 106 corresponding to the eye width and eye relief are favorably corrected.

  FIG. 6 shows an HMD 201 and a personal computer (PC) 202 constituting an image display system that is Embodiment 2 of the present invention. The HMD 201 is connected to an image processing unit and a PC 202 as an image processing apparatus via a cable (or wireless communication) so as to communicate with each other. The PC 202 generates a right-eye original image and a left-eye original image as image signals to be displayed on the right-eye and left-eye image display elements to be described later in the HMD 201, and communicates these original images to the HMD 201 by communication. Output.

  The HMD 201 is provided with a right eye position sensor 203 and a left eye position sensor 204 as eye position acquisition means and detection means, respectively. The right eye position sensor 203 detects the position of the right eye of the observer, and the left eye position sensor 204 detects the position of the left eye of the observer.

  In the present embodiment, a plurality of correction values are stored in the memory in the PC 202 so that different optical aberrations can be favorably corrected according to the position of the eyes of the observer. When the observer wears the HMD 201, the PC 202 detects the positions of the right eye and the left eye of the observer through the right eye and left eye position sensors 203 and 204. And the information (eye position information) regarding the eye position such as the eye width of the observer, the eye relief, and the deviation of the left and right eyes from the reference position is acquired from the detection result. Further, the PC 202 selects a correction value corresponding to the eye position information from a plurality of correction values stored in the memory.

  As described above, in this embodiment, the positions of the left eye and the right eye of the observer are detected by the right eye and left eye position sensors 203 and 204, and correction values corresponding to the detection results are set. This allows the observer to observe not only the individual differences in eye width and eye relief, but also an observation image in which distortion due to optical aberration and color misregistration are well corrected even when the eye is misaligned when the HMD is worn. be able to.

  FIG. 7 shows an eyepiece optical system 206 for the right eye of the HMD 201 in this embodiment. The right-eye eyepiece optical system 206 is composed of an optical element (prism) filled with an optical medium such as glass or plastic having a refractive index larger than 1 inside surrounded by three surfaces. The optical element includes a first surface 206a through which light from the original image displayed on the right-eye image display element 207 is transmitted, and a second surface 206b that reflects light incident on the optical element from the first surface 206a. And a third surface 206c that reflects the reflected light from the second surface 206b. The second surface 206b also functions as a transmission surface that transmits the reflected light from the third surface 206c. The light emitted from the third surface 206c is guided to the observer's right eye 205 located at the exit pupil (right exit pupil) of the right-eye eyepiece optical system 206. Thereby, an enlarged virtual image of the original image is presented to the observer.

  Thus, the light from the right-eye image display element 207 is reflected twice in the right-eye eyepiece optical system 206 and guided to the right eye 205. Since the second surface 206b is an optical surface that reflects and transmits light, it is desirable that the reflection here is total internal reflection in order to reduce a light amount loss. Further, the right eyepiece optical system 206 is thinned by folding the optical path using the decentered reflecting surfaces (206b, 206c). By making each surface constituting the right-eye eyepiece optical system 206 into a free-form surface, it is possible to satisfactorily correct decentration aberrations and allow an observer to observe an observation image with good image quality.

  In the present embodiment, the left-eye eyepiece optical system is configured in the same manner as the right-eye optical system 206.

  -By using an optical element having a decentered free curved surface as an eyepiece optical system as in this embodiment, distortion and lateral chromatic aberration are not rotationally symmetric. For this reason, it is desirable that the correction value used for the correction process in the PC 202 is a correction table having a correction value for each pixel of the image display element.

In addition, the eyepiece optical system in the present embodiment does not have an intermediate image plane, but as shown in FIG. 8, an intermediate image plane IM is formed in the optical path from the image display element 307 to the eye 305 of the observer. An eyepiece optical system 306 may be used. This makes it possible to present to the observer an observation image obtained by enlarging the original image while making the eyepiece optical system (that is, the HMD) compact.
(Other examples)
The present invention supplies a program that realizes one or more functions of the above-described embodiments to a system or apparatus via a network or a storage medium, and one or more processors in a computer of the system or apparatus read and execute the program This process can be realized. It can also be realized by a circuit (for example, ASIC) that realizes one or more functions.

  Each embodiment described above is only a representative example, and various modifications and changes can be made to each embodiment in carrying out the present invention.

101,201 HMD
102, 202 Personal computer (image processing apparatus)
103, 205 Right eye 104 Left eye 105, 206 Right eye eyepiece optical system 106 Left eye eyepiece optical system 107, 207 Right eye image display element 108 Left eye image display element

Claims (10)

An image display element;
An eyepiece optical system for guiding light from the original image displayed on the image display element to the eyes of the observer;
Image processing means for performing correction processing for correcting the original image using a correction value corresponding to the optical aberration of the eyepiece optical system;
The image processing system, wherein the image processing means uses the correction value corresponding to the position of the eye with respect to the eyepiece optical system for the correction processing. Eye position obtaining means for obtaining information on the eye position;
The image display system according to claim 1, wherein the image processing unit uses the correction value set based on information on the position of the eye. The eye position acquisition means is an input means for allowing the observer to input information regarding the position of the eye,
The image display system according to claim 2, wherein the image processing unit sets the correction value based on information regarding the position of the eye input from the input unit. The eye position acquisition means is a detection means for detecting the position of the eye;
The image display system according to claim 2, wherein the image processing unit sets the correction value based on the position of the eye detected by the detection unit. Storage means for storing a plurality of correction values corresponding to different eye positions,
The image processing means generates a plurality of correction original images by the correction processing using the plurality of correction values for the original image, and the correction original selected by the observer among the plurality of correction original images. The image display system according to claim 1, wherein the correction value corresponding to an image is used for the subsequent correction processing.   The image display system according to claim 1, wherein the optical aberration is a distortion aberration.   The image display system according to claim 1, wherein the optical aberration is a chromatic aberration of magnification. As the eyepiece optical system, there are a right eyepiece optical system and a left eyepiece optical system that guide light from the original image to the right exit pupil and the left exit pupil where the right eye and the left eye of the observer are respectively positioned. And
8. The right-eye and left-eye eyepiece optical systems each include an optical element having an optical surface that reflects and transmits the light and at least one reflective surface. The image display system according to one item. An image display device having the image display element and the eyepiece optical system;
The image display system according to claim 1, further comprising an image processing device as the image processing unit that outputs the original image to the image display device through communication. An image display system comprising: an image display element; an eyepiece optical system that guides light from an original image displayed on the image display element to an observer's eye; and a computer; an optical element that the eyepiece optical system includes in the computer A computer program for performing correction processing for correcting the original image using a correction value according to aberration,
An image processing program that causes the computer to perform the correction process using the correction value corresponding to the position of the eye with respect to the eyepiece optical system.