JP2017026943A - Image display device and image processing device - Google Patents

Abstract

PROBLEM TO BE SOLVED: To provide an image display device capable of providing an easy-to-see image, and an image processing device.SOLUTION: According to one embodiment, there is provided an image display device including optical sections, reflection sections and image processing sections. The optical sections emit light including image information of a display image. The reflection sections reflect at least parts of the light emitted from the optical sections toward pupils of an observer. The image processing sections generate the display image by changing at least one of a size and a shape of an object region being at least a part of the object image based on a positional relation between the reflection sections and the pupils.SELECTED DRAWING: Figure 1

Description

  Embodiments described herein relate generally to an image display device and an image processing device.

  A display unit that displays an image, an optical unit that projects an image displayed on the display unit using an optical element such as a lens, and a reflection unit that reflects the image projected from the optical unit toward the eyes of the viewer There is an image display device having a (combiner). In such an image display device, optical distortion or chipping may occur in an image viewed by the viewer depending on the pupil position of the viewer. Even if the pupil position changes, it is desirable to display an easy-to-view image with little distortion.

JP 2007-65899 A

  Embodiments of the present invention provide an image display device and an image processing device capable of providing an easy-to-see image.

  According to the embodiment of the present invention, an image display device including an optical unit, a reflection unit, and an image processing unit is provided. The optical unit emits light including image information of a display image. The reflection part reflects at least a part of the light emitted from the optical part toward the pupil of the viewer. The image processing unit generates the display image by changing at least one of a size and a shape of at least a part of the target region of the target image based on a positional relationship between the reflection unit and the pupil.

1 is a schematic view illustrating an image display device according to a first embodiment. 1 is a block diagram illustrating an image display device according to a first embodiment. FIG. 3A to FIG. 3C are schematic views illustrating images related to the image display apparatus according to the first embodiment. It is a schematic diagram which illustrates the image display apparatus which concerns on a reference example. 5A to 5D are schematic views illustrating an image display device according to a reference example. It is a block diagram which illustrates the image amendment part of the image display device concerning a 1st embodiment. FIG. 7A and FIG. 7B are schematic views illustrating image processing in the image display device according to the first embodiment. FIG. 8A to FIG. 8G are schematic views illustrating correction coefficients used in the image display apparatus according to the first embodiment. 3 is a flowchart illustrating processing of the image display apparatus according to the first embodiment. FIG. 10A to FIG. 10F are schematic views illustrating processing of the image display device according to the first embodiment. It is a block diagram which illustrates the image amendment part of the image display device concerning a 2nd embodiment. 10 is a flowchart illustrating processing in an image correction unit of an image display device according to a second embodiment. It is a block diagram which illustrates the image correction part of the image display apparatus which concerns on 3rd Embodiment. 10 is a flowchart illustrating processing in an image correction unit of an image processing apparatus according to a third embodiment. FIG. 15A to FIG. 15C are schematic views illustrating processing in the image correction unit of the image processing apparatus according to the third embodiment. It is a block diagram which illustrates the image correction part of the image display apparatus which concerns on 4th Embodiment. 14 is a flowchart illustrating processing in an image correction unit of an image display device according to a fourth embodiment. FIG. 18A to FIG. 18C are schematic views illustrating processing in the image correction unit of the image processing apparatus according to the fourth embodiment. FIG. 19A and FIG. 19B are schematic views illustrating an image display device according to the fifth embodiment. FIG. 20A and FIG. 20B are schematic views illustrating the image display device according to the fifth embodiment. FIG. 21A and FIG. 21B are schematic views illustrating an image display device according to the fifth embodiment. FIG. 22A and FIG. 22B are schematic views illustrating an image display device according to the fifth embodiment. FIG. 23A and FIG. 23B are schematic views illustrating the position control unit. FIG. 24 is a block diagram illustrating an image correction unit of the image display device according to the fifth embodiment. 14 is a flowchart illustrating processing in an image correction unit of an image display device according to a fifth embodiment. FIG. 26A and FIG. 26B are block diagrams illustrating an image display device according to the embodiment. It is a schematic diagram which illustrates the image display apparatus which concerns on embodiment.

Each embodiment will be described below with reference to the drawings.
The drawings are schematic or conceptual, and the relationship between the thickness and width of each part, the size ratio between the parts, and the like are not necessarily the same as actual ones. Further, even when the same part is represented, the dimensions and ratios may be represented differently depending on the drawings.
Note that, in the present specification and each drawing, the same elements as those described above with reference to the previous drawings are denoted by the same reference numerals, and detailed description thereof is omitted as appropriate.

(First embodiment)
FIG. 1 is a schematic view illustrating an image display device according to the first embodiment.
FIG. 2 is a block diagram illustrating the image display apparatus according to the first embodiment.
As illustrated in FIGS. 1 and 2, the image display apparatus 101 according to the present embodiment includes a projection unit 125 (display unit 110 and optical unit 120), a reflection unit 130, and an image processing device 200.

  The image display device 101 is, for example, a glasses-type head mounted display. The projection unit 125 acquires image information from the image processing apparatus 200 and emits light including the image information. The light is projected onto the eyes (pupil 150) of the viewer 60 (user) via the reflection unit 130. Thereby, a virtual image corresponding to the image information is displayed on the viewer 60.

  The image processing apparatus 200 includes an image processing unit 140 (circuit unit). The image processing unit 140 acquires data of a target image that is a display target and information on the pupil position (eyeball position) of the viewer 60. For example, the image processing unit 140 is connected to an external storage medium, a network, or the like by wire or wireless, and receives data of a target image (input image). The image processing unit 140 performs a correction process on the target image based on the pupil position of the viewer 60 and generates a display image (corrected image). The correction process will be described later.

  The image processing unit 140 is electrically connected to the display unit 110 by a bendable cable 145, for example. As a result, the display image generated by the image processing unit 140 is input to the display unit 110.

  The display unit 110 is a display that displays video and images, for example. For the display, for example, liquid crystal, organic EL, or Liquid Crystal On Silicon is used. However, the embodiment is not limited to these. The display unit 110 includes a plurality of pixels 110e arranged on a plane.

  The display unit 110 displays a display image. That is, the display unit 110 emits light L1 (image light) including image information of a display image. The light L1 is emitted toward the optical unit 120 from the plurality of pixels 110e.

  The optical unit 120 is provided between the display unit 110 and the reflection unit 130 on the optical path of the light L1 emitted from the plurality of pixels 110e. The optical unit 120 includes at least one or more optical elements. The optical unit 120 projects the incident light L1. As the optical element, a lens, a prism, a mirror, or the like can be used. For example, the optical unit 120 changes the traveling direction of at least part of the light L1. In this way, the optical unit 120 (projection unit 125) emits light (correction light) including image information toward the reflection unit 130. When a plurality of optical elements are used, they do not have to be arranged on a straight line.

  The reflection unit 130 includes a reflection surface provided along the first surface 11p. For example, a plurality of fine reflecting surfaces can be arranged in parallel on the first surface 11p to form the reflecting unit 130. The first surface 11p may be a flat surface or a curved surface. Each of the reflecting surfaces is, for example, a half mirror, and reflects at least a part of incident light. Each of the reflection surfaces is inclined with respect to the first surface 11p, and a step is formed between the reflection surfaces. The reflecting unit 130 has, for example, a Fresnel shape formed by a plurality of reflecting surfaces and a plurality of steps. The angle formed between each of the reflection surfaces and the first surface 11p is determined by the positional relationship between the optical axis of the optical unit 120 and the assumed pupil 150. By adjusting the angle formed between each of the reflection surfaces and the first surface 11p, for example, the light reflection angle can be adjusted.

  However, in the embodiment, the reflection unit 130 is not limited to such a half mirror. As the reflection unit 130, a normal half mirror may be used, or another member capable of adjusting the reflection angle may be used. In addition, as an example, an example is described in which a half mirror with the same reflectance and transmittance is applied, but the embodiment is not limited to the example with the same ratio. The material used for the reflecting surface may be any material as long as it is a material that transmits part of light and reflects part of light.

  The reflection unit 130 reflects at least a part of the light L1 emitted from the optical unit 120. For example, the reflecting unit 130 reflects light that has passed through the optical unit 120 toward the pupil 150 of the viewer 60 of the image display apparatus 101. A part of the light L1 reflected by the reflecting unit 130 forms an image (hereinafter referred to as “viewed image”). For example, a part of the light L <b> 1 forms the viewing image 170 as a virtual image when viewed from the pupil 150. In this way, the viewer 60 can see a viewing image (observation image) corresponding to the target image (and display image). In this example, the viewing image is displayed as a virtual image, but the reflection unit 130 may be separated from the pupil 150 and displayed as a real image.

  In addition, for example, a half mirror is used as the reflection unit 130, so that the reflection unit 130 transmits a part of light incident on the reflection unit 130 from the outside. Thereby, the human viewer 60 can view the outside world through the reflector 130. In this example, the viewing image 170 is displayed in front of the pupil 150, but an image may be displayed at the end of the field of view of the viewer 60 like the viewing image 171. Thereby, the visual field of the viewer 60 is not obstructed.

  For example, the relative arrangement of the display unit 110 and the optical unit 120 in the projection unit 125 is fixed. Note that the relative arrangement of the display unit 110 and the optical unit 120 may be variable so as not to impair the function of projecting an image. For example, in the projection unit 125, the display unit 110 and the optical unit 120 are attached with screws. By adjusting the tightening degree of the screw, the relative distance between the display unit 110 and the optical unit 120 and the angle formed with each other can be adjusted. By adjusting the distance between the display unit 110 and the optical unit 120, the distance from the viewer 60 to the virtual image viewed can be changed. For example, an image that was seen 1 m ahead of the face can be moved 2 m away.

  Note that the block diagram shown in FIG. 2 is an example, and may not necessarily match an actual module. For example, a part of each block may be provided separately from the image display device.

  A specific example of the image display apparatus 101 will be further described with reference to FIG. The image display device 101 is, for example, a glasses-type image display device, and further includes a holding unit 320 and a spectacle lens 160.

  The holding unit 320 is, for example, a spectacle frame (a temple or a vine). The image display apparatus 101 can be attached to the head of the viewer 60 by the holding unit 320. The holding unit 320 further includes a nose pad 321 and a bridge 322. The bridge 322 connects one spectacle lens 160 and the other spectacle lens 160. A rim of the spectacle lens 160 (a frame for holding the spectacle lens 160) or the like may be provided as necessary. In the present application, a configuration similar to that of normal correction glasses will be described, but a configuration in which the left and right lenses are integrated may be used.

  The spectacle lens 160 (reflecting unit 130) is held by the holding unit 320. For example, as in a normal spectacle frame, the angle between the holding unit 320 and the spectacle lens 160 may be variable.

For example, the relative arrangement of the nose pad 321 and the spectacle lens 160 is fixed. The reflector 130 is included in the spectacle lens 160 (provided integrally with the spectacle lens 160). That is, a combiner-integrated spectacle lens 160 is used, and the relative positional relationship between the reflector 130 and the spectacle lens 160 is fixed.
The spectacle lens 160 has a first surface 161 and a second surface 162. The second surface 162 is separated from the first surface 161. The reflection unit 130 is provided between the first surface 161 and the second surface 162. In addition, the position of the reflection part 130 is not limited to the above, For example, the structure which distribute | arranges the reflection part 130 on the 2nd surface 162 may be sufficient.

  When the image display apparatus 101 is used, the human viewer 60 places the nose pad 321 on the nose and places one end 320e of the holding unit 320 on the ear. In this manner, the position of the holding unit 320 and the relative position of the spectacle lens 160 (and the reflecting unit 130) are defined according to the position of the nose and ear of the viewer 60. When the image display apparatus 101 is used, the relative arrangement of the reflection unit 130 with respect to the holding unit 320 is substantially fixed. In addition, the position of the pupil 150 with respect to the reflecting unit 130 moves with eye movement.

  The nose pad 321 may be replaceable. That is, the holding part 320 may have a structure in which the nose pad can be replaced. For example, the nose pad 321 is replaced with another nose pad having a shape different from that of the nose pad 321. Thereby, for example, the relative arrangement of the pupil position and the reflecting unit 130 can be adjusted in the Z-axis direction shown in FIG.

In FIG. 1, a binocular head mounted display (HMD) using two image display devices 101 is illustrated. One image display device displays an image for the right eye of the viewer 60, and the other display device displays an image for the left eye. The embodiment may be a monocular HMD that uses one image display device 101 and displays an image for one eye.
In this example, one circuit unit is provided for each image display apparatus 101. When two image display devices 101 are used, the image processing unit 140 (circuit unit) may be integrated as much as possible.

FIG. 3A to FIG. 3C are schematic views illustrating images related to the image display apparatus according to the first embodiment.
FIG. 3A illustrates the target image A1 in the image display apparatus 101. The image processing unit 140 corrects the target image A1 based on the pupil position of the viewer 60, that is, the positional relationship between the reflecting unit 130 and the pupil 150.
FIG. 3B illustrates a display image A2 generated from the target image A1 by correction processing. The display image A2 is an image displayed on the display unit 110.
FIG. 3C illustrates a viewing image A3 formed by light including information on the display image A2 and viewed by the viewer 60.

  For example, the target image A1 is a color image. That is, the target image A1 includes a plurality of pixels, and each pixel includes a plurality of color components. The plurality of color components is information on three different colors (first to third colors), for example. For example, the first color, the second color, and the third color are red, blue, and green, respectively. Each pixel 110e of the display unit 110 emits light corresponding to each pixel of the target image. For example, each pixel 110e includes first to third sub-pixels, and the first to third sub-pixels emit light of first to third colors, respectively. A color image is displayed by superimposing the three colors of light. In other words, each pixel of the target image is represented by a first color pixel corresponding to the first color, a second color pixel corresponding to the second color, and a third color pixel corresponding to the third color. .

The image processing unit 140 performs a correction process for changing the scale (size) of at least a part of the target region (first region R1) of the target image A1 based on the positional relationship between the reflecting unit 130 and the pupil 150. The first region R1 is the whole or a part of the target image A1.
In this example, the first region R1 is a portion that displays a specific display object 80 in the target image A1. The first region R1 is, for example, a rectangle. The display object is, for example, information on at least one of specific characters, character strings, figures, and photographs in the image. In this example, the display object 80 is information of a character string “01234456789ABCDE”. A plurality of display objects may be arranged in one image.

  For example, the image processing unit 140 generates the display image A2 by changing at least one of the size of the first region R1 in the target image A1 and the shape of the first region R1. Changing the shape includes changing the width or changing the aspect ratio along a certain direction.

  For example, the image processing unit 140 changes the width in the horizontal direction of the first region R1. Here, the direction and ratio regarding the change of the width of the image are determined according to “optical distortion” and “missing” of the screen, which will be described later. Specifically, the direction and ratio regarding the change in the width of the image are determined based on the pupil position of the viewer 60, the position of the optical unit 120, the position of the reflecting unit 130, and the like. As a result, for example, optical distortion and screen defects can be suppressed.

  When the target image is a color image, the image processing unit 140 performs correction processing for each light color (first to third colors). In the correction process, the size information (vertical width and horizontal width) of the target image is taken into consideration. Thereby, for example, the aspect ratio of the target image is maintained in the viewing image A3.

  Note that the size of an image (or part thereof) corresponds to the number of pixels that display the image (or part thereof). Further, the width (length) along a certain direction of the image (or part thereof) corresponds to the number of pixels along the direction of the image (or part thereof).

  The display image A2 generated in this way includes a second region R2 generated based on the first region R1 (target region). The second region R2 is, for example, a rectangle. In this example, the second region R2 is a portion that displays the above-described display object 80 in the display image A2.

For example, the aspect ratio of the display image A2 is different from the aspect ratio of the target image A1.
Here, the aspect ratio of the target image A1 is the length Ln2 along the second direction D2 (vertical direction) of the target image A1 with respect to the length Ln1 along the first direction D1 (horizontal direction) of the target image A1. It is a ratio. Similarly, the aspect ratio of the display image A2 is a ratio of the length Ln4 along the second direction D2 of the display image A2 to the length Ln3 along the first direction D1 of the display image A2.

For example, the size of the second region R2 is different from the size of the first region R1. For example, the aspect ratio of the second region R2 is different from the aspect ratio of the first region R1.
Here, the aspect ratio of the first region R1 is a ratio of the length Ln6 along the second direction D2 of the first region R1 to the length Ln5 along the first direction D1 of the first region R1. Similarly, the aspect ratio of the second region R2 is the ratio of the length Ln8 along the second direction D2 of the second region R2 to the length Ln7 along the first direction D1 of the second region R2.

  The display image A2 further includes a first image region S1 and a second image region S2. The first image area S1 is an area for displaying image information of the target area of the target image A1. That is, the second region R2 is arranged in the first image region S1. The second image region S2 is a region that is not used for displaying the target image A1 in the display image A2. The first image region S1 and the second image region S2 are determined based on the positional relationship between the pupil of the viewer 60 and the reflecting unit 130, the position of the optical unit 120, and the like. By providing the first image area S1 and the second image area S2 as described above, it is possible to suppress a lack of a screen described later.

  By providing the second image region S2, the position of the first region R1 in the target image A1 is different from the position of the second region R2 in the display image A2. In this example, the center position of the second region R2 in the display image A2 (or the display screen of the display unit 110) is decentered compared to the center position of the first region R1 in the target image A1. In the embodiment, the display image A2 may not include the second image region S2.

  A viewing image A3 is formed by the light emitted from the display unit 110 based on the display image A2. The viewing image A3 includes a first image Q1 formed based on the first region R1 and the second region R2. In this example, the first image Q1 is a portion that displays the display object 80 described above. The first image Q1 is, for example, a rectangle.

  The aspect ratio of the first image Q1 is substantially equal to the aspect ratio of the first region R1. Here, the aspect ratio of the first image Q1 is a ratio of the length Ln10 along the second direction D2 of the first image Q1 to the length Ln9 along the first direction D1 of the first image Q1.

Next, optical distortion, color breakup, and screen defects that occur in the image display apparatus will be described. FIG. 4 is a schematic view illustrating an image display device according to a reference example.
The image display device 109 according to the reference example includes a reflection unit 130b and a projection unit 125b. A configuration similar to that of the reflection unit 130 can be applied to the reflection unit 130b, and a configuration similar to that of the projection unit 125 can be applied to the projection unit 125b. The image display device 109 is different from the image display device 101 according to the embodiment in that it does not include the image processing unit 140. That is, the image display device 109 does not perform the correction process for the target image described above.

  A viewing image 170b is formed by the light emitted from the projection unit 125b. For example, in the image display device, the optical design is made on the assumption that the pupil of the observer is located within a certain range. FIG. 4 exemplifies that there is a range of pupil positions where the viewing image 170b can be observed. This range is called an eye range 180 and is a circular region having a diameter of about several millimeters.

  In such a glasses-type image display device, the position of the reflection unit 130 with respect to the position of the eye is determined according to the arrangement of the ears, nose, and eyes of the viewer 60. For example, when the viewer 60 changes, the position of the reflection unit 130 with respect to the eye changes. For this reason, when the viewer 60 changes, the position of the image seen from the viewer 60 may change, and the image may not be displayed at an appropriate position. If the pupil 150 is within the eye range 180, the viewer can see the entire image 170. However, when the pupil 150 is outside the eye range 180, the screen is missing.

  The optical distortion and color breakup that occur in the image display device 109 will be described. Light including image information emitted from the display unit travels to the pupil via an optical element such as a lens or a half mirror that constitutes the optical unit or the reflection unit 130b. For example, each time light passes through the optical element or is reflected by the optical element, aberration occurs. For this reason, degradation such as optical distortion and color breakup occurs in the viewing image 170b.

  The optical distortion is an aberration when light emitted from the display unit passes through the optical element. The optical distortion is caused when the image of the light passing through the optical element loses similarity to the image of the light emitted from the display unit.

  In color image display, color breakup is an aberration caused by a difference in light wavelength. The size of the image of light after passing through the optical element depends on the wavelength of the light. For example, as the wavelength is shorter, refraction by a lens or the like is likely to occur, and the eye range is likely to be narrowed. Also, for example, the center position of the image depends on the wavelength of light and varies depending on the color.

5A to 5D are schematic views illustrating an image display device according to a reference example.
FIG. 5A shows how the light emitted from the projection unit 125b reaches the pupil 150a (position PA) or the pupil 150b (position PB) at a different position. The difference in pupil position is caused by individual differences among the viewers 60, for example.

  FIG. 5B is an example of an image PI displayed on the display unit of the image display device 109. The image PI includes a white square lattice image Pw. White is displayed by overlapping red, blue and green.

  FIG. 5C is an example of a virtual image that can be seen from the pupil 150a (position PA) when the image PI is displayed on the display unit. FIG. 5D is an example of a virtual image that can be seen from the pupil 150b (position PB) when the image PI is displayed on the display unit.

  As shown in FIGS. 5C and 5D, the shape of the displayed virtual image is distorted with respect to the shape of the image PI. This is due to the phenomenon caused by the above-described optical distortion and color breakup. Furthermore, due to color breakup, the size and position of the area Pb in which blue is displayed, the area Pg in which green is displayed, and the area Pr in which red is displayed are different from each other, and the displayable areas are different for each color. I can confirm. In FIG. 5C, a part of the leftmost virtual image is missing. This is a phenomenon that occurs when the position PB exists outside the eye range 180 of the image display device 109.

In this way, the displayed image is distorted according to the relative arrangement of the pupil and the projection unit. Further, when a pupil is present outside the eye range, a screen defect occurs.
On the other hand, for example, a method of preparing a plurality of pieces of unique information (for example, distortion correction coefficients) indicating individual differences between image display apparatuses is conceivable. In this method, a distortion correction coefficient corresponding to each individual is selected from a plurality of distortion correction coefficients, and a display image in which the target image is corrected is generated. Thereby, the above-mentioned optical distortion etc. can be suppressed. However, as described above, optical distortion, color breakup, screen loss, and the like also depend on the pupil position of the viewer. For this reason, in the above method, depending on the viewer 60, optical distortion and color breakup may occur. In addition, when a pupil is present outside the eye range, a screen defect occurs. Furthermore, since the displayable area differs for each color due to color breakup, an area where some colors are not displayed may occur.

  On the other hand, in the image display apparatus 101 according to the embodiment, the image processing unit 140 includes at least one target image based on the pupil position of the viewer 60 (the positional relationship between the pupil 150 and the reflecting unit 130). The size or shape of the part (first region) is changed. Further, when the target image is a color image, correction processing is performed for each of the first to third color pixels. Thereby, even if the pupil position of the viewer 60 changes, optical distortion and color breakup can be suppressed. Further, at least part of the target image is changed based on the size information (aspect ratio) of the target image. Thereby, the aspect ratio can be maintained in the viewing image. Further, the image processing unit 140 sets the first image region S1 and the second image region S2 described above based on the pupil position of the viewer 60. The second image region S2 corresponds to a region where the image cannot be viewed from the viewer 60 when the pupil position is outside the eye range. Thereby, for example, even if the pupil position of the viewer 60 changes, it is possible to avoid a lack of a screen. In addition, about the Z-axis direction, the nose pad 321 may be replaced and the height of the reflection part 130 may be adjusted. Thereby, screen chipping in the Z-axis direction may be suppressed.

  As described above, the image processing unit 140 adaptively corrects the target image based on the pupil position. As a result, it is possible to display an easy-to-see virtual image with less image distortion, color breakup, and screen chipping.

Hereinafter, a specific example of the image processing unit 140 will be further described.
As illustrated in FIG. 2, the image processing unit 140 includes an image correction unit 141 and a setting unit 142. The image correction unit 141 and the setting unit 142 are incorporated in, for example, the image processing unit 140 (circuit unit).

  The setting unit 142 acquires information on the pupil position regarding the positional relationship (relative arrangement) between the pupil 150 and the reflecting unit 130, and outputs the information to the image correction unit 141.

  For example, the image display apparatus 101 further includes a first detection unit 182 (see FIG. 1) that detects the pupil position. The first detection unit 182 is provided in the holding unit 320, for example. Arbitrary sensors, such as an infrared sensor and a visible camera, can be used for the 1st detection part 182. FIG. The pupil position can be measured (estimated) based on information such as an image obtained from the sensor. An electrooculogram or the like may be used for measuring the pupil position. For example, a change in electrooculogram when light enters the pupil 150 is measured. Information on the pupil position can be obtained based on the incident direction of light and changes in electrooculogram.

  The pupil position information set by the setting unit 142 may be based on information input by the viewer 60. For example, the viewer 60 inputs information such as a numerical value (for example, a scale value) to the image display apparatus 101 as information regarding the position of the pupil 150. Thereby, for example, the viewer 60 can change the correction in the image correction unit 141. For example, software (application) is installed in a computer or a mobile terminal, and information is input from the viewer 60 to the setting unit 142 via the computer or the mobile terminal.

  The image correction unit 141 performs distortion correction on the target image based on the pupil position information output from the setting unit 142. The image correction unit 141 outputs a display image (corrected image) generated by distortion correction to the display unit 110.

FIG. 6 is a block diagram illustrating an image correction unit of the image display apparatus according to the first embodiment.
FIG. 7A and FIG. 7B are schematic views illustrating image processing in the image display device according to the first embodiment.
As illustrated in FIG. 6, the image correction unit 141 includes a coordinate conversion unit 410, a scale update unit 420, and a correction coefficient output unit 430. In this example, the correction coefficient output unit 430 further includes a correction coefficient selection unit 431 and a correction coefficient storage unit 435. Note that the block diagram shown in FIG. 6 is an example, and may not necessarily match an actual module. For example, a part of each block (for example, the correction coefficient storage unit 435) may be provided separately from the image display device.

The correction coefficient storage unit 435 stores a plurality of predetermined correction coefficients. Each of the plurality of correction coefficients corresponds to each of a plurality of pupil positions.
In this example, the plurality of correction coefficients are a first correction coefficient C1 corresponding to the first positional relationship between the reflecting unit 130 and the pupil 150, and a second correction corresponding to the second positional relationship between the reflecting unit 130 and the pupil 150. And a coefficient C2. FIG. 6 shows a coefficient (position PA correction coefficient 436) when the pupil is arranged at the position PA as the first correction coefficient C1. As the second correction coefficient C2, a coefficient (position PB correction coefficient 437) when the pupil is arranged at the position PB is shown.

  In distortion correction, a display image is generated by correcting a target image using a correction coefficient. As already described, the target image is represented by a plurality of pixels. The position of each pixel in the image is represented by coordinates in a coordinate system (image coordinate system). For example, the image correction unit 141 generates a display image by converting the coordinates of a plurality of pixels using the first correction coefficient C1.

  As shown in FIG. 7A, the correction coefficient stores a correspondence relationship between each pixel of the display image and each pixel of the target image. The image correction unit 141 refers to this correspondence relationship, acquires the position of each pixel of the target image corresponding to the display image, and generates a display image.

In the case where the target image is an image represented by three primary colors such as red, green, and blue, for example, as described with reference to FIG. 5C and FIG. In this case, a display image is generated using three correction coefficients set for each color.
For example, the first correction coefficient C1 includes a first color coefficient for the first color pixel, a second color coefficient for the second color pixel, and a third color coefficient for the third color pixel. The image correction unit 141 converts the coordinates of the first color pixel in the image coordinate system based on the first color coefficient, converts the coordinates of the second color pixel in the image coordinate system based on the second color coefficient, and The coordinates in the image coordinate system of the third color pixel are converted based on the color coefficient.

  In FIG. 7A, the display object of the target image is represented by a white image Iw. The white image Iw is a superposition of the red image Ir, the blue image Ib, and the green image Ig. In the target image, the positions of the red image Ir, the blue image Ib, and the green image Ig are the same. On the other hand, in the display image, the positions of the red image Ir, the blue image Ib, and the green image Ig are different from each other and shifted. In this way, coordinate conversion is performed for each color component.

  When there is a pupil position outside the eye range, a correction coefficient is used so that the screen is not lost. FIG. 7B shows a state where a display image is generated using the correction coefficient when the pupil is located outside the eye range. The correction coefficient used in this case includes information regarding a region in the display image that the viewer 60 cannot see. That is, the correction coefficient used in this case includes information on the first image area S1 and the second image area S2 described with reference to FIG. In FIG. 7A, distortion correction of the target image is applied using a correction coefficient in which the correspondence of pixels of the entire image is stored. In contrast, in FIG. 7B, the display image is generated without using the leftmost region of the display image. Accordingly, as shown in FIG. 5C, when the pupil position is at a position where the display image at the left end is missing, it is possible to avoid the lack of the screen.

FIG. 8A to FIG. 8G are schematic views illustrating correction coefficients used in the image display apparatus according to the first embodiment.
When setting the correction coefficient, for example, a visual image is captured using a camera. The correction coefficient can be determined from the correspondence relationship between each pixel of the visual image and each pixel of the display image. In the case where the display image and the target image are images represented by three primary colors such as red, green, and blue, the pixel correspondence is stored for each color.

FIG. 8A shows an image Ra represented by the first color (red “R” in this example) in the display image. This image is generated from the first color pixel of the target image, for example.
FIG. 8B shows an image Ga represented by the second color (green “G” in this example) in the display image. This image is generated from the second color pixel of the target image, for example.
FIG. 8C shows an image Ba represented by the third color (blue “B” in this example) in the display image. This image is generated from, for example, the third color pixel of the target image.

The viewing image viewed by the viewer 60 includes a plurality of color images corresponding to a plurality of color components. For example, the viewing image is an overlay of a first color image corresponding to the first color pixel, a second color image corresponding to the second color pixel, and a third color image corresponding to the third color pixel. is there.
FIG. 8D shows a first color image Rb formed by red light including the information of the image Ra in FIG.
FIG. 8E shows a second color image Gb formed by green light including the information of the image Ga in FIG.
FIG. 8F shows a third color image Bb formed by blue light including information on the image Ba in FIG.

  As described above, the correspondence between the viewing image and the display image is different for each color due to the influence of color breakup. For example, as shown in FIGS. 8D to 8F, the areas where the first to third color images are displayed are different from each other. In the viewing image, color display is possible in a region where the first to third color images overlap each other, but color display cannot be performed in other regions. Therefore, the display areas of the three primary colors are integrated into areas capable of color display. As shown in FIGS. 8D to 8F, the inscribed rectangles Rc, Gc, and Bc of the reference position area of each color are obtained. Then, as shown in FIG. 8G, the product area of each inscribed rectangle is obtained. The reference position of each color is normalized by the size of the product area. Thus, a correction coefficient that can refer to a color displayable area (aspect ratio a: 1) is generated.

  It is assumed that the aspect ratio a: 1 is stored in the correction coefficient. For example, the first correction coefficient C1 includes information regarding the overlapping region Sa (product region) in the viewing image in which a plurality of color images (first to third color images) overlap each other. Specifically, the first correction coefficient C1 includes information on the ratio (a: 1) of the length along the horizontal direction of the overlapping region Sa and the length along the vertical direction of the overlapping region Sa. Thereby, for example, the aspect ratio of the target image can be maintained at the time of distortion correction by a process described later.

  With the above processing, the correspondence between the coordinate position on the display image and the position on the normalized coordinate is obtained for each of the three colors RGB.

  It should be noted that areas other than the color displayable area that cannot be expressed simultaneously with three colors may be defined as areas capable of displaying a single color or two colors. The correction coefficient may include information on an area that can be displayed in a single color or two colors. That is, for example, the first correction coefficient C1 may further include information on the non-overlapping region Sb in the viewing image in which a plurality of color images (first color image and second color image) do not overlap each other.

  For example, in the case of red, as shown in FIG. 8D, information on a region Rd capable of displaying a single color may be stored in the correction coefficient, in addition to the region capable of color display. The correction coefficient may include information on an area where a single color other than red can be displayed, or information on an area where a combination of two colors can be displayed.

  The correction coefficient may include a curve expression parameter. Thereby, the correspondence between the pixel of the corrected image and the pixel of the observation image can be calculated.

  In FIG. 6, the correction coefficient selection unit 431 acquires position information related to the relative arrangement of the pupil and the reflection unit 130 from the setting unit 142. The correction coefficient selection unit 431 selects a correction coefficient according to the acquired position information.

  The scale update unit 420 acquires the correction coefficient selected from the correction coefficient output unit 430. The scale update unit 420 scales the correction coefficient based on the size of the target image and the aspect ratio of the reference coordinate area of the acquired correction coefficient while maintaining the aspect ratio of the target image. Then, the scale update unit 420 outputs the correction coefficient whose scale has been updated to the coordinate conversion unit 410. Details of the processing of the scale update unit 420 will be described later. The coordinate conversion unit 410 generates a correction image using the target image and the correction coefficient obtained from the scale update unit 420.

  For example, the correction coefficient output unit 430 selects and outputs the first correction coefficient C1 from the plurality of correction coefficients based on the first positional relationship between the reflection unit 130 and the pupil 150. The scale update unit 420 updates the first correction coefficient C1 based on the size (size information) of the target image. The coordinate conversion unit 410 converts the coordinates of each pixel of the target image using the updated first correction coefficient C1.

  FIG. 9 is a flowchart illustrating the processing of the image display apparatus according to the first embodiment. In step S <b> 510, information regarding the pupil position is input to the setting unit 142. The setting unit 142 outputs the input information to the image correction unit 141 as pupil position information. The pupil position varies depending on, for example, race, region, age, and the like. The information on the pupil position is expressed by a relative arrangement with respect to a reference position where the position is uniquely determined, such as a nose pad. The relative arrangement may be expressed by three-dimensional coordinates. The information on the pupil position may include not only the position of the pupil but also the posture.

  In step S511, a target image is input. The image format and format type are arbitrary. For example, the image is a bitmap image. The image may be either a color image or a gray scale image.

In step S520, the correction coefficient selection unit 431 selects a correction coefficient registered in the correction coefficient storage unit 435 according to the pupil position information. The correction coefficient selection unit 431 outputs the selected correction coefficient to the scale update unit 420.
For example, the correction coefficient for the position PA and the correction coefficient for the position PB are registered in the correction coefficient storage unit. When the pupil position of the viewer 60 is PA, the correction coefficient for the position PA is selected. If the correction coefficient corresponding to the pupil position of the viewer 60 is not registered in the correction coefficient storage unit, the correction coefficient selection unit 431 uses the evaluation value such as the Euclidean distance and is closest to the pupil position. Select the correction coefficient corresponding to the position.

  In step S530, the scale update unit 420 calculates the scale used by the coordinate conversion unit 410 using the image size of the target image, and updates the correction coefficient. The correction coefficient is normalized in a rectangular area. Therefore, when applying distortion correction to the target image, a scale from the normalized coordinate system to the image coordinate system is obtained. At this time, the scale is calculated so that the target image fits in the rectangular area of the correction coefficient that can be displayed in color. As a result, the image can be displayed while maintaining the aspect ratio of the target image.

FIG. 10A to FIG. 10F are schematic views illustrating processing of the image display device according to the first embodiment.
FIG. 10A illustrates the target image A4. The width of the target image A4 (length along the horizontal direction) is W _IN, the height of the target image A4 (length along the longitudinal direction) is H _IN. At this time, the scale s from the normalized coordinate system to the image coordinate system of the correction coefficient can be calculated by the following equation (1).
s = max (W _IN / a, H _IN ) (1)
a is the aspect ratio of the rectangular area capable of color display described above. max represents the maximum value among a plurality of arguments. The correction coefficient is updated by scaling the normalized coordinates of the correction coefficient using the obtained scale s.

  FIG. 10B illustrates the correction coefficient Ca updated using the obtained scale s. As shown in FIG. 10B, the correction coefficient is scaled while maintaining the aspect ratio. In the example of FIG. 10B, due to the difference between the aspect ratio of the target image and the aspect ratio of the color displayable area, an area that is not used for distortion correction occurs when the correction coefficient is scaled. The areas not used for distortion correction are, for example, an upper rectangular area 920 and a lower rectangular area 930. Information different from the target image may be presented in an area that is not used for distortion correction. For example, information including the current time and date is presented.

  Thereafter, in step S540, the coordinate conversion unit 410 calculates a display image obtained by correcting the target image using the target image and the updated correction coefficient. FIG. 10C illustrates a display image A5 obtained by converting the coordinates of the pixel of the target image A4 in FIG. 10A using the updated correction coefficient Ca.

  FIG. 10D illustrates the same target image A4 as that in FIG. FIG. 10E illustrates the correction coefficient Cb updated by scaling similar to that described with reference to FIG. The pupil position corresponding to the correction coefficient Cb is different from the pupil position corresponding to the correction coefficient Ca. The pupil position corresponding to the correction coefficient Ca exists in the eye range, and the pupil position corresponding to the correction coefficient Cb exists outside the eye range.

  FIG. 10F illustrates the display image A6 obtained by converting the pixel coordinates of the target image A4 in FIG. 10D using the updated correction coefficient Cb. Unlike the display image A5 of FIG. 10C, the display image A6 includes a region 940. The area 940 is an unused area on the display image, for example, and corresponds to the second image area S2 described with reference to FIG. Information regarding the region 940 is held by the correction coefficient Cb. For this reason, the process using the correction coefficient Cb is the same as the process using the correction coefficient Ca in both step S530 and step S540.

  According to the first embodiment described above, the image display apparatus 101 corrects the target image by the image correction unit 141 so as to suppress the influence of the screen defect and the aberration caused according to the pupil position. As a result, it is possible to display an easy-to-see virtual image with less image distortion, color breakup, and missing screen.

(Second Embodiment)
FIG. 11 is a block diagram illustrating an image correction unit of the image display apparatus according to the second embodiment.
Similar to the image display device 101 according to the first embodiment, the image display device 102 according to the present embodiment includes a display unit 110, an optical unit 120, a reflection unit 130, and the like. These are the same as those in the first embodiment. The second embodiment differs from the first embodiment in the image processing unit.

  The image processing unit of the image display apparatus 102 includes an image correction unit 141 b and a setting unit 142. As illustrated in FIG. 11, the image correction unit 141b includes a coordinate conversion unit 410, a scale update unit 420, and a correction coefficient output unit 430b. The correction coefficient output unit 430b includes a correction coefficient calculation unit 432, a correction coefficient selection unit 433, and a correction coefficient storage unit 435. The same description as in the first embodiment can be applied to the coordinate conversion unit 410, the scale update unit 420, and the correction coefficient storage unit 435. Note that the block diagram shown in FIG. 11 is an example, and may not necessarily match an actual module.

FIG. 12 is a flowchart illustrating processing in the image correction unit of the image display apparatus according to the second embodiment.
In step S510, as in the first embodiment, the image correction unit 141b acquires pupil position information.
In step S511, as in the first embodiment, the image correction unit 141b acquires a target image.

  In step S521, the correction coefficient selection unit 433 selects a plurality of correction coefficients from the correction coefficient storage unit 435 according to the pupil position. For example, the correction coefficient selection unit selects a correction coefficient based on the evaluation value related to the pupil position. When the pupil position is expressed by three-dimensional coordinates, the Euclidean distance can be used as the evaluation value. The correction coefficient selection unit 433 acquires a correction coefficient corresponding to a position close to the pupil position acquired in step S510 from the plurality of correction coefficients. For example, the correction coefficient selection unit 433 selects a plurality of correction coefficients in order from the shortest Euclidean distance. The number of correction coefficients selected may be any number greater than or equal to two.

  In step S525, the correction coefficient calculation unit 432 generates a correction coefficient at the pupil position by a linear interpolation method such as bilinear interpolation based on the plurality of correction coefficients selected by the correction coefficient selection unit 433. The generated correction coefficient is output to the scale update unit 420.

  Thereafter, in step S530, as in the first embodiment, the scale update unit 420 updates the correction coefficient. In step S540, as in the first embodiment, the coordinate conversion unit 410 converts the coordinates of the pixels of the target image to generate a display image.

The above processing will be described as an example where the pupil 150 is in the first position. At this time, it is assumed that the positional relationship between the pupil 150 and the reflecting unit 130 is the first positional relationship.
The correction coefficient storage unit 435 includes, for example, a second correction coefficient and a third correction coefficient. The second correction coefficient corresponds to the second position of the pupil 150 and corresponds to the second positional relationship between the pupil 150 and the reflecting unit 130. The third correction coefficient corresponds to the third position of the pupil 150 and corresponds to the third positional relationship between the pupil 150 and the reflecting unit 130. For example, among the pupil positions corresponding to the respective correction coefficients stored in the correction coefficient storage unit 435, the second position is the position closest to the first position, and the third position is the second closest to the first position. Position.

  At this time, the correction coefficient selection unit 433 selects the second correction coefficient and the third correction coefficient from the correction coefficient storage unit 435 based on the first positional relationship. Then, the correction coefficient calculation unit 432 calculates a first correction coefficient corresponding to the first positional relationship using the second correction coefficient and the third correction coefficient. Thereafter, the scale update unit 420 updates the calculated first correction coefficient, and the coordinate conversion unit 410 performs coordinate conversion of the target image using the updated first correction coefficient.

  As described above, a correction coefficient corresponding to the pupil position may be calculated based on a plurality of predetermined correction coefficients. As a result, as in the first embodiment, it is possible to display an easy-to-see virtual image with less image distortion, color breakup, and screen loss.

(Third embodiment)
FIG. 13 is a block diagram illustrating an image correction unit of an image display device according to the third embodiment.
Similar to the image display device 101 according to the first embodiment, the image display device 103 according to the present embodiment includes a display unit 110, an optical unit 120, a reflection unit 130, and the like. These are the same as those in the first embodiment. The third embodiment is different from the first embodiment in the image processing unit.

  The image processing unit of the image display device 103 includes an image correction unit 141 c and a setting unit 142. As illustrated in FIG. 13, the image correction unit 141c includes a coordinate conversion unit 410, a scale update unit 421, and a correction coefficient output unit 430c. The coordinate conversion unit 410 is the same as that in the first embodiment. The correction coefficient output unit 430c is the same as the correction coefficient output unit 430 or 430b described above. Note that the block diagram shown in FIG. 13 is an example, and may not necessarily match an actual module.

FIG. 14 is a flowchart illustrating processing in the image correction unit of the image processing apparatus according to the third embodiment.
In step S510, as in the first embodiment, the image correction unit 141c acquires pupil position information.
In step S511, as in the first embodiment, the image correction unit 141c acquires a target image.

  In the present embodiment, the setting unit 142 sets information regarding the display area in addition to the pupil position information described in the first embodiment. The display area is an area in which information of the target image is displayed in the display image (corrected image). In step S512, the image correction unit 141c acquires information (display area information) regarding the set display area.

  In step S526, as in the first or second embodiment, the correction coefficient output unit 430c outputs the correction coefficient to the scale update unit 421.

  In step S531, the scale update unit 421 scales the correction coefficient output from the correction coefficient output unit 430c based on the size information (width and height) of the target image and the display area information. In step S540, as in the first embodiment, coordinate conversion is performed and a display image (corrected image) is generated.

An example of the processing in steps S531 and S540 will be further described.
FIG. 15A to FIG. 15C are schematic views illustrating processing in the image correction unit of the image processing apparatus according to the third embodiment.
FIGS. 15A to 15C illustrate a method of arranging the target image in the color displayable area of the correction coefficient.
FIG. 15A illustrates the target image A7. FIG. 15B illustrates the correction coefficient Cc scaled to the image coordinate system by the scale update unit 421. FIG. 15C illustrates a display image A8 (corrected image) obtained by converting the coordinates of the pixel of the target image A7 using the correction coefficient Cc.

  In step S531, an area used as a display area is set according to the display area information acquired in step S512, and the information is output to the coordinate conversion unit 410. Specifically, the display area information is, for example, an enlargement ratio (or reduction ratio) of the size of the target image and an offset (such as a display start position in a color displayable area). FIGS. 15A to 15C show how the target image is associated with the coordinate system of the scaled correction coefficient based on the preset enlargement ratio and offset.

  Similar to the scale update unit 420 in the first embodiment, the correction coefficient is scaled by a constant s. Further, the target image is associated with the correction coefficient based on the enlargement factor and the offset. In the example of FIGS. 15A to 15C, the target image is reduced to 0.5 times, and the display start position is associated with the position (0, 0) in the image coordinate system of the correction coefficient. Yes. Accordingly, as illustrated in FIG. 15C, the display image A <b> 8 includes a display area 1410 including information on the target image and a non-display area 1411. The non-display area 1411 is an area other than the display area 1410 in the display image A8. In the non-display area 1411, information on the target image is not displayed. By setting the display area and applying distortion correction, an image can be presented at the top of the screen.

  When no information regarding the display area is set, the scale update unit 421 performs the same process as the scale update unit 420 of the first embodiment.

  By changing the display area information (magnification and offset of the target image), the size of the display area 1410 relative to the size of the display image A8 and the position of the display area 1410 in the display image A8 can be changed.

  As described in the first embodiment, it is desirable to display an image to which distortion correction is applied according to the pupil position. There are various appropriate display positions for the viewer 60, and the appropriate display position may differ depending on the purpose. For example, when a user performs an operation while looking at a virtual image, if an image is displayed at the center of the screen (for example, the center of the user's field of view), the hand may be difficult to see. For this reason, work efficiency may fall.

  On the other hand, in the present embodiment, the position and size of the display area used for displaying the target image are variable depending on the display area information. Thereby, for example, after the display position of the video is appropriately set, the screen is controlled by the image correction unit 141, and a desired image can be displayed to the user.

  The display area information set by the setting unit 142 may be based on information input by the viewer 60. For example, the viewer 60 inputs information indicating the display area (for example, a scale value) to the image display device 103. Thereby, for example, the viewer 60 can change the image display position in the image correction unit 141c. For example, software (application) is installed in a computer or a mobile terminal, and information is input from the viewer 60 to the image processing unit (circuit unit) via the computer or the mobile terminal. A plurality of display methods may be stored in the memory in advance, and the viewer 60 may select one from the stored display methods. For example, a set value is input corresponding to the selected display method. Examples of the display method include a method of displaying only at the right end and a method of displaying only at the lower end. As described above, the viewer 60 can appropriately set the size and position of the display area.

  According to the third embodiment, similarly to the first embodiment, it is possible to display an easy-to-see virtual image with less image distortion, color breakup, and missing screen. Furthermore, an image (virtual image) can be displayed at an appropriate position according to the purpose.

(Fourth embodiment)
FIG. 16 is a block diagram illustrating an image correction unit of an image display device according to the fourth embodiment.
Similar to the image display device 101 according to the first embodiment, the image display device 104 according to the present embodiment includes a display unit 110, an optical unit 120, and a reflection unit 130. These are the same as those in the first embodiment. The fourth embodiment differs from the first embodiment in the image processing unit.

  The image processing unit of the image display device 104 includes an image correction unit 141d and a setting unit 142. As illustrated in FIG. 16, the image correction unit 141d includes a coordinate conversion unit 410, a scale update unit 422, and a correction coefficient output unit 430d. The coordinate conversion unit 410 is the same as that in the first embodiment. The correction coefficient output unit 430d is the same as the correction coefficient output unit 430 or 430b described above. Note that the block diagram shown in FIG. 16 is an example, and may not necessarily match an actual module.

FIG. 17 is a flowchart illustrating processing in the image correction unit of the image display apparatus according to the fourth embodiment.
In step S510, as in the first embodiment, the image correction unit 141d acquires pupil position information.
In step S511, the image processing unit (image correction unit 141d) acquires a target image.

  In the present embodiment, the setting unit 142 sets information related to the use area in addition to the pupil position information described in the first embodiment. The use area is an area used as an object of correction and display among the target images acquired by the image correction unit 141d. In other words, in the present embodiment, the correction and display target is, for example, a part (use area) of the target image. In step S513, the image correction unit 141d acquires information on the used area (used area information).

  In step S526, as in the first or second embodiment, the correction coefficient output unit 430d outputs the correction coefficient to the scale update unit 422.

  In step S532, the range of the used area in the target image is set according to the used area information acquired in step S513. Then, the scale update unit 422 scales the correction coefficient output from the correction coefficient output unit 430d based on the set use area. In step S540, as in the first embodiment, coordinate conversion is performed, and a display image (corrected image) is generated.

An example of the processing in steps S542 and S540 will be further described.
FIG. 18A to FIG. 18C are schematic views illustrating processing in the image correction unit of the image processing apparatus according to the fourth embodiment.
FIG. 18A to FIG. 18C illustrate a method of setting a use area by clipping a target image.

  In the scale update unit 422, the use area of the target image is set according to the acquired use area information. The used area information is information indicating the used area in the target image, and specifically, for example, the image size and offset of the used area in the target image. The use area information may be, for example, coordinate values of four vertices of the rectangular area. For example, as shown in FIG. 18A, the use area 1710 in the target image A9 is designated by the use area information.

In this example, when the display image is generated by correcting the target image, the size information (width and height) of the used area in the target image is considered. The correction coefficient is updated based on the size information (width and height) of the used area, and correction processing (coordinate conversion) is performed using the updated correction coefficient.
For example, as shown in FIG. 18A, the width of the used area obtained based on the used area information is _W′IN, and the height of the used area is _H′IN . At this time, the scale s ′ of the correction coefficient from the normalized coordinate system to the image coordinate system can be calculated by Expression (2).
s ′ = max (W ′ _IN / a, H ′ _IN ) (2)
a is the aspect ratio of the rectangular area capable of color display described above. The correction coefficient is updated by scaling the normalized coordinate of the correction coefficient using the obtained scale s ′. As a result, an updated correction coefficient Cd is obtained as shown in FIG. Then, by using the updated correction coefficient Cd to convert the coordinates of the pixels of the target image A9, a display image A10 (corrected image) as shown in FIG. 18C is obtained.

  Note that when no information regarding the use area is set, the scale update unit 422 performs the same processing as the scale update unit 420 of the first embodiment.

  The target image A9 includes the use area 1710 and the non-use area 1711 described above. The non-use area 1711 is an area other than the use area 1710 in the target image A9. The display image A10 illustrated in FIG. 9C includes information based on the use area 1710, but does not include information based on the non-use area 1711. In this way, by setting the use area and applying the distortion correction, the video included in the set area can be presented to the viewer 60 except for the unnecessary area.

  As described in the first embodiment, it is desired to display an image to which distortion correction according to the pupil position is applied. The input image data is often image data in accordance with a standard. For example, when image data is input via a communication interface according to a standard, the image size may be limited to 1280 × 720. For this reason, distortion correction may not be applied to an arbitrary image.

  On the other hand, in the present embodiment, the distortion correction is applied by the image correction unit 141d after the use area is set in the target image. Thereby, for example, an image in which distortion correction is applied to a desired region of the target image can be generated.

  The use area information set by the setting unit 142 may be based on information input by the viewer 60. For example, the viewer 60 inputs information indicating the use area (for example, coordinate values of four corners of the rectangular area) to the image display device 104. Thereby, the viewer 60 can change the use area of the target image in the image correction unit 141d. For example, software (application) is installed in a computer or a mobile terminal, and information is input from the viewer 60 to the image processing unit (circuit unit) via the computer or the mobile terminal. In addition, a plurality of use areas may be stored in the memory in advance, and the viewer 60 may select one from the stored use areas. For example, the selected use area becomes the set value. Examples of the setting method of the use area include a method of specifying the width and height with respect to the origin of the coordinates of the target image, and a method of specifying the width and height with respect to the center of the target image. As described above, the viewer 60 can appropriately set the use area.

  According to the fourth embodiment, as in the first embodiment, it is possible to display an easy-to-see virtual image with less image distortion, color breakup, and screen chipping. Furthermore, a desired image can be displayed without being affected by the standard.

(Fifth embodiment)
Also in the image display device 105 according to the present embodiment, the display unit 110, the optical unit 120, the reflection unit 130, the holding unit 320, and the like are provided as in the image display device 101 according to the first embodiment. About these, the description similar to the description about 1st Embodiment is applicable.

The image display apparatus according to the present embodiment further includes a position control unit 126 (see FIG. 1). The fifth embodiment differs from the first embodiment in the position control unit 126 and the image processing unit.
In the following, first, the position control unit 126 will be described.

  The projection unit 125 (the display unit 110 and the optical unit 120) is held by the holding unit 320 via the position control unit 126. The position control unit 126 is fixed to the holding unit 320. The relative arrangement of the projection unit 125 and the reflection unit 130 is variable by the position control unit 126.

  In FIG. 1, the extending direction of the holding unit 320 is a Y-axis direction. One direction perpendicular to the Y-axis direction is taken as the X-axis direction. A direction perpendicular to the X-axis direction and perpendicular to the Y-axis direction is taken as a Z-axis direction. For example, the X-axis direction corresponds to the left-right direction (lateral direction) of the viewer 60, the Y-axis direction corresponds to the front-rear direction of the viewer 60, and the Z-axis direction corresponds to the up-down direction of the viewer 60. Corresponds to (vertical direction). In FIG. 1, the holding portion 320 has a side extending linearly in the Y-axis direction, but the embodiment includes a case where the shape of the holding portion 320 is gently bent. The shape of the holding part 320 is appropriately changed in consideration of design and convenience in use.

FIG. 19A and FIG. 19B are schematic views illustrating an image display device according to the fifth embodiment.
In FIG. 19A and FIG. 19B, a position control unit 126a is used as an example of the position control unit 126. In the example shown in FIGS. 19A and 19B, the distance between the projection unit 125 and the reflection unit 130 is variable by the position control unit 126a. For example, the distance is variable along the optical axis of the optical unit 120.

  In this example, the long hole 31 is provided in the position control unit 126 a along the optical axis of the optical unit 120. The projection unit 125 is provided with a movable shaft 51. The movable shaft 51 is fixed to the projection unit 125. The movable shaft 51 can pass through the long hole 31 and slide in the long hole 31. Thereby, the position of the projection unit can be adjusted.

  19A shows a state where the distance between the projection unit 125 and the reflection unit 130 is long, and FIG. 19B shows a state where the distance between the projection unit 125 and the reflection unit 130 is short. Yes. 19A and 19B show an optical path of the light L2 emitted from one end of the projection unit 125 and an optical path of the light L3 emitted from another end of the projection unit 125.

  In the example of FIG. 19A, the light L <b> 3 is reflected by the reflecting unit 130 and enters the pupil 150. On the other hand, a part of the light L <b> 2 reflected by the reflecting unit 130 does not enter the pupil 150. For this reason, the human viewer 60 cannot see the right end of the image, for example. On the other hand, as shown in FIG. 19B, the distance between the reflection unit 130 and the projection unit 125 is shortened. Thereby, the spread of the light L2 in the reflecting portion 130 is suppressed. Since the light emitted from the end of the projection unit enters the pupil, a correct virtual image can be seen.

FIG. 20A and FIG. 20B are schematic views illustrating the image display device according to the fifth embodiment.
20A and 20B, a position control unit 126b is used as an example of the position control unit 126. In the example shown in FIGS. 20A and 20B, the relative arrangement between the projection unit 125 and the reflection unit 130 is variable by the position control unit 126b.

  For example, the reflection unit 130 is provided along the first surface 11p. In the direction D5 along the first surface 11p, the arrangement of the projection unit 125 is variable. For example, the direction D5 is parallel to a plane including the incident direction (for example, DL1) and the reflection direction (DL2) of the light emitted from the projection unit 125 (display unit 110) in the reflection unit 130. In this example, the direction D5 is parallel to the X-axis direction. In the left-right direction of the viewer 60, the relative arrangement of the projection unit 125 and the reflection unit 130 is variable.

  The position controller 126b is provided with a long hole 32 along the X-axis direction. The movable shaft 52 fixed to the projection unit 125 can pass through the long hole 32 and slide in the long hole 32. Thereby, the position of the projection part 125 can be adjusted in the left-right direction of the human viewer 60.

  20A shows a state where the projection unit 125 is arranged on the right side, and FIG. 20B shows a state where the projection unit 125 is arranged on the left side. For example, the distance between the projection unit 125 and the holding unit 320 in FIG. 20A is shorter than the distance between the projection unit 125 and the holding unit 320 in FIG.

  As shown in FIG. 20A, in this example, a part of the light L <b> 2 emitted from the projection unit 125 is not incident on the pupil 150. For this reason, the viewer 60 cannot see the right end of the image, for example. On the other hand, the projection unit 125 is moved to the left as shown in FIG. Thereby, the light L2 enters the pupil 150. Since the light emitted from the end of the projection unit enters the pupil, a correct virtual image can be seen. As the projection unit 125 moves in the left-right direction, the image viewed from the viewer 60 also moves in the left-right direction.

FIG. 21A and FIG. 21B are schematic views illustrating an image display device according to the fifth embodiment. FIGS. 21A and 21B are side views of the human viewer 60 viewed from the lateral direction.
21A and 21B, a position control unit 126c is used as an example of the position control unit 126. In the example shown in FIG. 21A and FIG. 21B, the relative arrangement of the projection unit 125 and the reflection unit 130 is variable by the position control unit 126c.

  For example, the reflection unit 130 is provided along the first surface 11p. In the direction D6 along the first surface 11p, the arrangement of the projection unit 125 is variable. The direction D6 is a direction perpendicular to the direction D5 described in FIG. In this example, the direction D6 is parallel to the Z-axis direction. In the vertical direction of the viewer 60, the relative arrangement of the projection unit 125 and the reflection unit 130 is variable.

  The position controller 126c is provided with a long hole 33 along the Z-axis direction. The movable shaft 53 fixed to the projection unit 125 can pass through the long hole 33 and slide in the long hole 33. Thereby, the position of the projection part 125 can be adjusted in the up-down direction of the human viewer 60.

  FIG. 21A shows a state where the projection unit 125 is arranged on the lower side, and FIG. 21B shows a state where the projection unit 125 is arranged on the upper side.

  As shown in FIG. 21A, in this example, a part of the light L <b> 2 emitted from the projection unit 125 is not incident on the pupil 150. For this reason, the viewer 60 cannot see the lower end of the image, for example. On the other hand, the projection unit 125 is moved upward as shown in FIG. Thereby, the light L2 enters the pupil 150. Since the light emitted from the end of the projection unit 125 enters the pupil, a correct virtual image can be seen. As the projection unit 125 moves in the vertical direction, the image viewed from the viewer 60 also moves in the vertical direction.

  FIG. 22A and FIG. 22B are schematic views illustrating an image display device according to the fifth embodiment. 22A and 22B, a position control unit 126d is used as an example of the position control unit 126. The relative arrangement of the projection unit 125 and the reflection unit 130 is variable by the position control unit 126d.

  For example, the optical unit 120 has an optical axis 120a. The angle between the optical axis 120a and the first surface 11p is variable by the position controller 126d. In other words, the incident direction DLa of the light L1 including the image information in the reflection unit 130 is variable by the position control unit 126d.

  In this example, the position control unit 126 d has a rotation shaft unit 54. The projection unit 125 is held by the rotation shaft unit 54. The projection unit 125 can be rotated around the rotation shaft unit 54. For example, the projection unit 125 can be rotated in the XY plane. 22A shows a state where the incident angle of the light L1 to the reflecting unit 130 is large, and FIG. 22B shows a state where the incident angle of the light L1 to the reflecting unit 130 is small. Thus, by rotating the projection unit 125, the incident direction DLa and the reflection direction DLb of the light L1 in the reflection unit 130 can be adjusted. Thereby, the direction in which an image can be seen can be adjusted.

FIG. 23A and FIG. 23B are schematic views illustrating the position control unit.
In the example shown in FIG. 23A, the incident direction DLa of the light L1 including the image information in the reflecting unit 130 is variable by the position control unit 126d. The projection part 125 is provided with an attachment part 55. For example, the attachment portion 55 has a shape of a part of a sphere. An opening 35 is provided in the position controller 126d. For example, the opening 35 covers at least a part of the attachment portion 55. The attachment portion 55 is held by the position control portion 126d, and the attachment portion 55 can be rotated in the opening 35. Thereby, the projection part 125 can be rotated in the up-down direction and the left-right direction, and the direction in which an image can be seen can be adjusted.

  For example, there is a reference method for adjusting the position of the virtual image by changing the position of the reflection unit 130 without changing the arrangement of the projection unit 125. However, in the eyeglass-type image display device, the relative arrangement of the eyeglass frame and the eyeglass lens (reflecting unit 130) is substantially fixed during use. For this reason, as described above, the relative arrangement of the pupil 150 of the viewer 60 and the reflecting unit 130 is substantially fixed, and it may be difficult to change the position of the reflecting unit 130. On the other hand, in the embodiment, the position control unit 126 changes the arrangement of the projection unit 125. Thereby, for example, the degree of freedom is increased in adjusting the relative arrangement of the reflection unit 130 and the projection unit 125.

  The mechanism of the position control unit 126 described above is an example, and the embodiment includes any form that can similarly adjust the position of the projection unit. Furthermore, a plurality of mechanisms of the position control unit 126 described above may be combined. For example, the position control unit 126e shown in FIG. 23B is an example in which a rotation mechanism in the XY plane and a position adjustment mechanism in the left-right direction are combined. In the embodiment, the mechanism and the number of mechanisms used for the combination are arbitrary. Thereby, the projection part 125 can be arrange | positioned in an appropriate position, the position of an image can be adjusted, and an easy-to-see display can be obtained.

Next, the image processing unit of the image display device 105 will be described.
FIG. 24 is a block diagram illustrating an image correction unit of the image display device according to the fifth embodiment.
The image display device 105 according to the present embodiment includes an image correction unit 141e. The image correction unit 141e includes a coordinate conversion unit 410, a scale update unit 420, and a correction coefficient output unit 430e. The coordinate conversion unit 410 and the scale update unit 420 are the same as those in the first embodiment. Note that the block diagram shown in FIG. 24 is an example, and may not necessarily match an actual module.

  The correction coefficient output unit 430e stores a plurality of predetermined correction coefficients as in the first or second embodiment. Each of the plurality of correction coefficients corresponds to each of a plurality of pupil positions. Furthermore, in the present embodiment, each of the plurality of correction coefficients corresponds to each of a plurality of projection unit positions. That is, the correction coefficient according to the present embodiment is generated according to at least two pieces of information, that is, pupil position information and projection unit position information. The structure and generation method of the correction coefficient are the same as those in the first embodiment. Note that the position of the projection unit is represented by, for example, the relative arrangement of the optical unit 120 (projection unit 125) and the reflection unit 130. That is, each correction coefficient is determined based on the positional relationship between the reflecting unit 130 and the pupil 150 and the relative arrangement of the optical unit 120 and the reflecting unit 130.

FIG. 25 is a flowchart illustrating the processing in the image correction unit of the image display device according to the fifth embodiment.
The setting unit 142 sets the pupil position information as in the first embodiment.
In step S510, the image correction unit 141e acquires pupil position information as in the first embodiment.
In step S511, the image correction unit 141e acquires the target image as in the first embodiment.

  In the present embodiment, the setting unit 142 sets information on the projection unit position in addition to the pupil position information described in the first embodiment. In step S514, the image correction unit 141e acquires information on the position of the projection unit.

  The setting unit 142 acquires (sets) projection unit position information related to the relative arrangement of the projection unit 125 (the optical unit 120) and the reflection unit 130 separately from the pupil position information when the reflection unit and the projection unit are variable. And output to the image correction unit 141e. Arbitrary sensors, such as a camera and a potentiometer, can be used for detection of a projection part position. The sensor is provided, for example, in the projection unit 125, the position control unit 126, or a circuit unit. For example, the relative arrangement of the optical unit 120 and the reflecting unit 130 can be adjusted steplessly.

  The projection unit position information may be used for estimating the pupil position of the viewer. For example, in a glasses-type display device, the relative arrangement of the reflection unit 130 and the optical unit 120 changes according to the arrangement of the viewer's 60 ears, nose, eyes, and the like. For this reason, it is also possible to estimate the pupil position of the human viewer 60 to some extent based on information on the relative arrangement of the optical unit 120 and the reflecting unit 130.

  The position control unit 126 may be a mechanical mechanism that adjusts the position of the projection unit 125 in stages. For example, a scale such as a dial indicating position information is provided, and the position of the projection unit 125 is adjusted step by step. The setting unit 142 outputs information corresponding to the scale value to the image correction unit 141e.

  The projection unit position information set by the setting unit 142 may be based on information input by the viewer 60. For example, the viewer 60 inputs information indicating the position of the projection unit 125 (for example, a scale value) to the image display device 105. Accordingly, for example, the viewer 60 can change the correction in the image correction unit 141e. For example, software (application) is installed in a computer or a mobile terminal, and information is input from the viewer 60 to the image control unit (the image correction unit 141e and the setting unit 142) via the computer or the mobile terminal. Position information may be directly input to the image control unit.

  In step S527, the correction coefficient output unit 430e selects a correction coefficient from a plurality of correction coefficients based on the pupil position information and the projection unit position information. The selected correction coefficient is output to the scale update unit 420.

  Steps S530 and S540 are the same as in the first embodiment. As described above, the image correction unit 141e corrects the target image and generates a display image. The display unit 110 emits light based on the image information of the generated display image. Thereby, for example, an appropriate image can be displayed.

  For example, when the position of the projection unit 125 changes, the optical path length from the display unit 110 to the pupil 150 changes for each pixel. For this reason, depending on the position of the projection unit 125, optical distortion or color breakup may occur. On the other hand, in the present embodiment, the target image correction process is performed based on pupil position information and projection unit position information. That is, distortion correction is adaptively applied based on the position of the pupil 150 and the position of the projection unit 125. For this reason, the influence by the aberration which arises according to a pupil position and the position of a projection part can be suppressed. As a result, it is possible to display an easy-to-see virtual image with less image distortion, color breakup, and missing screen.

  The image display apparatus and the image processing apparatus according to the first to fifth embodiments have been described above. However, the embodiment is not limited to the above example, and for example, the first to fifth embodiments may be appropriately combined. For example, the embodiment may be as described with reference to FIGS. 26 (a) and 26 (b) below.

FIG. 26A and FIG. 26B are block diagrams illustrating an image display device according to the embodiment.
The image correction unit 141f of the image display device 106 illustrated in FIG. 26A includes a coordinate conversion unit 410, a scale update unit 426, and a correction coefficient output unit 430f. Other than this, the image display device 106 is the same as that of the first embodiment.

  The correction coefficient output unit 430f stores a plurality of correction coefficients for correcting the aforementioned color breakup and optical distortion. For example, the plurality of correction coefficients correspond to unique information indicating individual differences between image display apparatuses, and the correction coefficient output unit 430f selects a correction coefficient according to the unique information. Thereby, color breakup and optical distortion can be corrected. The structure and generation method of the correction coefficient are the same as those in the first embodiment.

  In this specific example, pupil position information is input to the scale update unit 426 by a sensor or a user. The scale update unit 426 updates the correction coefficient based on the pupil position information and the size information of the target image. At this time, for example, the scale update unit 426 generates information similar to the display area information described in the third embodiment based on the pupil position information, and, based on the generated information, the third embodiment and the third embodiment. Similarly, the correction coefficient is scaled. Then, the coordinate conversion unit 410 generates a display image from the target image using the scaled correction coefficient. Thereby, it is possible to correct image defects.

  The image correction unit 141g of the image display device 107 illustrated in FIG. 26B includes a coordinate conversion unit 410, a scale update unit 427, and a correction coefficient output unit 430g. Other than this, the image display device 107 is the same as that of the fifth embodiment.

  The correction coefficient output unit 430g stores a plurality of correction coefficients for correcting the above-described color breakup and optical distortion. For example, each correction coefficient corresponds to each projection unit position information. The structure and generation method of the correction coefficient are the same as those in the first embodiment. Projector position information is input to the correction coefficient output unit 430g by a sensor or a user. The correction coefficient output unit 430g selects a correction coefficient according to the projection unit position information. Thereby, for example, color breakup and optical distortion due to the position of the projection unit can be corrected.

  In this embodiment, pupil position information is input to the scale update unit 427 by a sensor or a user. Thereby, it is possible to correct the screen defect in the same manner as described with reference to FIG. For example, distortion caused by the position of the projection unit is automatically corrected based on information from a sensor or the like, and the size and position of the display image are corrected based on user input.

FIG. 27 is a schematic view illustrating the image display device according to the embodiment.
FIG. 27 illustrates an example of the system of the image display device according to the embodiment.
As illustrated in FIG. 27, the image processing unit 140 (circuit unit) includes, for example, an interface 42 (acquisition unit), a processing circuit (processor) 43, and a memory 44.

  For example, the image processing unit 140 is connected to an external storage medium or a network via the interface 42 and acquires an input image (target image). For connection to the outside, either a wired or wireless method may be used. The acquisition unit may be an input / output terminal, for example.

  In the memory 44, for example, a program 45 for processing the acquired target image is stored. For example, the target image is appropriately converted based on the program 45, and accordingly, an appropriate display is performed on the display unit 110. In the memory 44, image information may be held. The program 45 may be provided in a state stored in the memory 44 in advance, or may be provided via a storage medium such as a CD-ROM or a network, and may be installed as appropriate.

  The image processing unit 140 may include a sensor 46. For the sensor 46, for example, an arbitrary sensor such as a camera, a microphone, a position sensor, or an acceleration sensor can be used. For example, the image displayed on the display unit 110 is appropriately changed based on information obtained from the sensor 46. Thereby, the convenience and visibility of the image display device can be improved. For example, in the processing circuit 43, information obtained from the sensor 46, image information, and the like are processed based on the program 45. The image information obtained in this way is input from the image processing unit 140 to the display unit 110 and displayed on the image display device.

  For example, position information regarding the relative arrangement of the projection unit 125 and the reflection unit 130 is detected by the sensor 46. The correction coefficient used for correction in the image correction unit 141 is stored in the memory 44, for example. Further, for example, the processing in the coordinate conversion unit 410, the scale update unit 420, and the correction coefficient output unit 430 is performed in the processing circuit 43 based on the program 45.

  An integrated circuit such as LSI (Large Scale Integration) or an IC (Integrated Circuit) chip set can be used for a part or all of each block of the image processing unit 140 (image processing apparatus 200). An individual circuit may be used for each block, or a circuit in which part or all of the blocks are integrated may be used. Each block may be provided integrally, or a part of the blocks may be provided separately. In addition, a part of each block may be provided separately. The integration is not limited to LSI, and a dedicated circuit or a general-purpose processor may be used.

  The image display apparatus and the image processing apparatus have been described above as the embodiments. However, the embodiment may be in the form of a program that causes a computer to execute processing in the image processing apparatus, or in the form of a computer-readable recording medium that records this program. Specific examples of the recording medium include CD-ROM (-R / -RW), magneto-optical disk, HD (hard disk), DVD-ROM (-R / -RW / -RAM), and FD (flexible disk). ), A flash memory, a memory card, a memory stick, and other various ROMs and RAMs. An image processing program for causing a computer to execute the processing of the image processing apparatus according to the above-described embodiment may be recorded and distributed on these recording media. This facilitates the realization of image processing. Then, the information processing apparatus such as a computer is loaded with the recording medium as described above, and the image processing program is read by the information processing apparatus, or the program is stored in a storage medium included in the information processing apparatus, Read accordingly. Thereby, the image processing according to the embodiment can be executed.

  According to the embodiment, an image display device and an image processing device that can provide an easy-to-see image can be provided.

  In the present specification, “vertical” and “parallel” include not only strictly vertical and strictly parallel, but also include, for example, variations in the manufacturing process, and may be substantially vertical and substantially parallel. It ’s fine.

The embodiments of the present invention have been described above with reference to specific examples. However, embodiments of the present invention are not limited to these specific examples. For example, regarding the specific configuration of each element such as a projection unit, an optical unit, a display unit, a reflection unit, and an image processing unit, the person skilled in the art similarly implements the present invention by selecting appropriately from a well-known range. As long as the above effects can be obtained, the scope of the present invention is included.
Moreover, what combined any two or more elements of each specific example in the technically possible range is also included in the scope of the present invention as long as the gist of the present invention is included.

  In addition, all image display devices and image processing devices that can be implemented by those skilled in the art based on the image display device and image processing device described above as the embodiment of the present invention are also included in the gist of the present invention. As long as it is included, it belongs to the scope of the present invention.

  In addition, in the category of the idea of the present invention, those skilled in the art can conceive of various changes and modifications, and it is understood that these changes and modifications also belong to the scope of the present invention. .

  Although several embodiments of the present invention have been described, these embodiments are presented by way of example and are not intended to limit the scope of the invention. These novel embodiments can be implemented in various other forms, and various omissions, replacements, and changes can be made without departing from the scope of the invention. These embodiments and modifications thereof are included in the scope and gist of the invention, and are included in the invention described in the claims and the equivalents thereof.

  11p: first surface 31, 32, 33: long hole, 35: opening, 42: interface, 43 ... processing circuit, 44 ... memory, 45 ... program, 46 ... sensor, 51, 52, 53 ... movable shaft, 54 ... Rotating shaft part, 55 ... Mounting part, 60 ... Viewer, 80 ... Display object, 101-105, 109 ... Image display device, 110 ... Display part, 110e ... Pixel, 120 ... Optical part, 120a ... Optical axis 125, 125b ... projection unit, 126, 126a, 126b, 126c, 126d, 126e ... position control unit, 130, 130b ... reflection unit, 140 ... image processing unit, 141 ... image correction unit, 1410 ... display area, 1411 ... Non-display area, 141b, 141c, 141d, 141e ... image correction unit, 142 ... setting unit, 145 ... cable, 1 0, 150a, 150b ... pupil, 160 ... spectacle lens, 161 ... first surface, 162 ... second surface, 170, 170b, 171 ... visual image, 1710 ... use region, 1711 ... non-use region, 180 ... eye range 182 ... Detection unit 200 ... Image processing device 320 ... Holding unit 320e ... One end 321 ... Nose pad 322 ... Bridge 410 ... Coordinate conversion unit 420,421,422,426 ... Scale update unit 430 430b, 430c, 430d, 430e ... correction coefficient output unit, 431 ... correction coefficient selection unit, 432 ... correction coefficient calculation unit, 433 ... correction coefficient selection unit, 435 ... correction coefficient storage unit, 436, 437 ... correction coefficient, 920, 930: Rectangular area, 940: Area, A1: Target image, A2: Display image, A3: Viewing image A4 ... target image, A5, A6 ... display image, A7 ... target image, A8 ... display image, A9 ... target image, A10 ... display image, Ba ... image, Bb ... third color image, Bc ... inscribed rectangle, C1 ... first correction coefficient, C2 ... second correction coefficient, Ca, Cb, Cc, Cd ... correction coefficient, D1 ... first direction, D2 ... second direction, D5, D6 ... direction, DLa ... incident direction, DLb ... reflection Direction, Ga ... image, Gb ... second color image, Gc ... inscribed rectangle, L1, L2, L3 ... light, Ln1-Ln10 ... length, PA, PB ... position, PI ... image, Pa, Pb, Pr ... Region, Pw ... image, Q1 ... first image, R1 ... first region (target region), R2 ... second region, Ra ... image, Rb ... first color image, Rc ... inscribed rectangle, Rd ... region, S1 ... 1st image area, S2 ... 2nd image area, Sa ... overlapping region, Sb ... non-overlapping region, S510-S514, S520, S521, S525-S527, S530-S532, S540, S542 ... step

Claims (9)

An optical unit that emits light including image information of a display image;
A reflection unit that reflects at least a part of the light emitted from the optical unit toward a pupil of a viewer;
An image processing unit that generates the display image by changing at least one of a size and a shape of a target region of at least a part of the target image based on a positional relationship between the reflection unit and the pupil;
An image display device comprising:   The image according to claim 1, wherein the image processing unit generates the display image by changing the at least one of a size and a shape of the target region based on the positional relationship and the size information of the target image. Display device.   The image according to claim 1, wherein the image processing unit generates the display image so that image information of the target region is displayed in a first image region of the display image determined based on the positional relationship. Display device. The viewing image viewed by the viewer includes a first image formed based on the target area of the target image,
The image display device according to claim 1, wherein the image processing unit generates the display image so that an aspect ratio of the target region is equal to an aspect ratio of the first image.   The said image processing part converts the coordinate of the some pixel contained in the said target image using the correction coefficient corresponding to the said positional relationship, and produces | generates the said display image. Image display device. Each of the plurality of pixels has a plurality of color components,
The image display device according to claim 5, wherein the image processing unit performs coordinate conversion for each of the color components.   The image processing unit stores a plurality of correction coefficients corresponding to a plurality of positional relationships between the reflecting unit and the pupil, and selects the correction coefficient used for correction from the plurality of correction coefficients based on the positional relationship. The image display device according to claim 5 or 6. A position control unit for controlling a relative arrangement of the optical unit and the reflection unit;
The image display device according to claim 5, wherein a distance between the optical unit and the reflection unit is variable. An acquisition unit for acquiring a target image;
An image processing unit that generates a display image by changing at least one of a size and a shape of at least a part of a target region of the target image, wherein the display image is an image of the display image emitted from an optical unit The image processing unit that is generated based on a positional relationship between the pupil and a reflecting unit that reflects at least part of the light including information toward the pupil of the viewer;
An image processing apparatus.