JP2006308674A - Image display device - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To provide an image display device on which the video in the line-of-sight direction of an observer is properly superimposed on a virtual space video and presented. <P>SOLUTION: The image display device of this invention is provided with: a display section on which the video is displayed; a first mirror section which guides the video displayed on the display section to the eyeball of the observer; a first camera section by which the direction of the eyeball of the observer is photographed, to detect the line-of-sight direction of the observer; a second camera section whose optical axis coincides with that of the first camera section and by which the line-of-sight direction of the observer is photographed; and a computer terminal which is wiredly or wirelessly connected to the first camera section and the second camera section and combines the video in the line-of-sight direction of the observer photographed by the second camera section and virtual information. In addition, the video in the line-of-sight direction of the observer is superimposed on the virtual information and displayed on the display section by the computer terminal. <P>COPYRIGHT: (C)2007,JPO&INPIT

Description

  The present invention directly projects the transmitted or reflected light beam of the image display element onto the retina of the viewer, senses the viewer's gaze, and superimposes the virtual space video on the video in the gaze direction and presents it. The present invention relates to an image display device.

  In the field of mixed reality, which is a technology that fuses reality and virtual, a display device (image display device) that fuses (virtually displays) virtual reality and displays it to the user without a sense of incongruity is one of the important devices. It is.

  Problems with such an optical see-through display, which is a display device, include a viewing angle, a sharpness of a video (contrast), a difference in a distance where a real video and a virtual video are in focus, and the like. In the clearness of the image, the brightness of the image in the real space changes sequentially depending on the situation, but the brightness of the virtual image will cause the viewer to feel uncomfortable if it is not properly adjusted. There is a problem.

  Further, in the conventional display device, the imaging optical system and the display optical system are matched in order to eliminate the parallax between the observer's line of sight and the optical axis of the camera. However, in this case, a realistic image is only presented in the fixed front.

  In addition, there is a conventional display device in which a video presentation device and a remote camera are aligned in the line of sight of an observer in order to give a sense of reality of a remote video (for example, Patent Document 1). However, in the case of this invention, since the camera rotation axis cannot be physically located at the same position as the eyeball, the scene in the gaze direction of the observer in the real world and the mixed reality technology are matched to the effective range of eye movement. There is a problem that it is impossible to display the virtual product in a three-dimensional manner in accordance with the observer's own optical axis.

  Further, in the conventional image display device, when the device presents a virtual product at a specific position in the real world, even if the observer focuses on a position different from the virtual product, the virtual product Is presented to the observer as a clear image. Therefore, it is indispensable to blur the virtual product. The geometric position of the virtual product can be known precisely on the computer. Here, if the blur of the real world image and the blur of the virtual product can be matched from the viewpoint of the position, the observer can appropriately recognize the position of the virtual product in the real world image.

In a known image display device, there is a device that presents two different definition images to an observer using a line-of-sight detection device in order to give a sense of reality of the image in the line-of-sight direction. This device aims to add blur similar to blur generated when an observer observes the real world to a virtual object on a display device (for example, Patent Document 2). In some cases, the amount of blur with respect to a virtual object is calculated by an element that changes the depth of field for a camera installed in a three-dimensional virtual space (for example, Patent Document 3). However, the invention described in Patent Document 2 has a problem that the blur in the real world cannot match the blur of the virtual object in the virtual world. The invention described in Patent Document 3 has a problem that blur in the depth of field direction cannot be expressed.
JP-A-8-313843 JP-A-11-109279 JP-A-11-134074

  An object of the present invention is to provide an image display device that appropriately superimposes and presents a virtual space image on an image in the viewing direction of an observer.

The present invention has been made to achieve the above object. An image display device according to the present invention includes:
A display unit for displaying images;
A first mirror section for guiding the image displayed on the display section to an observer's eyeball;
A first camera unit that captures the direction of the eyeball of the observer and detects the line of sight of the observer;
A second camera unit whose optical axis coincides with the first camera unit and captures the line-of-sight direction of the observer;
A computer terminal that is wired or wirelessly connected to the first camera unit and the second camera unit, and synthesizes the video of the observer's line of sight imaged by the second camera unit and virtual information; Prepared,
A video image of the observer's line of sight and virtual information are superimposed and displayed on the display unit by a computer terminal.

  An image display apparatus according to the present invention includes means for detecting an observer's line of sight and means for detecting an image in the observer's line of sight, and presents the virtual space image superimposed on the image in the observer's line of sight. Thus, since the real space image and the virtual space image can be observed in accordance with the direction of the observer's line of sight, the observer can fuse the virtual reality without any sense of incongruity.

  Further, the image display device according to the present invention blurs and presents virtual information other than the vicinity of the distance of the observer's viewpoint according to the distance of the observer's viewpoint, so that the observer can see the virtual product in the real world. It has the effect of knowing exactly where it is located.

  Hereinafter, preferred embodiments according to the present invention will be described with reference to the drawings.

Embodiment 1 FIG.
FIG. 1 is a block diagram showing an image display apparatus according to Embodiment 1 of the present invention. The configuration diagram of FIG. 1 shows a portion for one eye (left eye) of the image display device. There is usually also a part for the other eye (right eye), which forms a mirror object with the part of FIG.

  The image display apparatus of FIG. 1 includes a display unit 10, a first rotary drive unit 7, an arm unit 6, a second rotary drive unit 8, and a slide drive unit 9. The display unit 10 has a generally hemispherical shape, and one end of the display unit 10 is fixed to the first rotary drive unit 7. And the direction is changed by the slide type drive unit 9). The display unit 10 includes a display unit 1, a first mirror unit 2, an eyeball camera unit 4, and a line-of-sight camera unit 5. Although not shown, a computer terminal that synthesizes video and displays it on the display unit 1 is used.

  In the image display device according to the first embodiment, an image displayed on the display unit 1 built in the display unit 10 is reflected by the first mirror unit 2 and projected onto the eyeball 3. An observer (which is the main eyeball) observes an image displayed on the display unit 1.

  The first mirror unit 2 is translucent, for example, and the eyeball camera unit 4 observes the direction in which the observer's eyeball 3 is facing, that is, the line of sight. In addition, the eye direction photographing camera unit 5 has the same optical axis as the eyeball photographing camera unit 4 and the photographing unit, and photographs a real image in the observer's eye direction. The line-of-sight imaging camera unit 5 transmits the captured image information to a computer terminal (not shown) via wireless or wired connection. Here, a virtual image generated by a computer terminal (not shown) and a real image photographed by the visual line direction photographing camera unit 5 are superimposed with the brightness adjusted, and the display unit 1 is displayed. In this way, the observer can observe a real and virtual fusion image without any sense of incongruity.

  In addition, the first rotary drive unit 7, the second rotary drive unit 8, and the slide type drive unit 9 connected by the arm unit 6, in the direction of the observer's line of sight measured by the eyeball camera unit 4. At the same time, the display unit 10 is moved. The image information of the eyeball photographed by the eyeball camera unit 4 is transmitted to a computer terminal (not shown) via wireless or wired communication, and the computer terminal is connected to the first rotary drive unit 7 and the second rotary type. The drive unit 8 and the slide type drive unit 9 may be controlled and driven according to the image information. At the same time, the visual direction photographing camera unit 5 incorporated in the display unit 10 photographs a real image in the visual line direction of the observer. Therefore, the observer can always observe the real image in the direction of the line of sight and can receive information provided by the virtual image.

Embodiment 2. FIG.
FIG. 2 is a diagram showing the configuration of the display unit 10 of the image display apparatus according to Embodiment 2 of the present invention. The image display device according to the second embodiment is substantially the same as the image display device according to the first embodiment. Therefore, the same parts are denoted by the same reference numerals and description thereof is omitted.

  In the image display device according to the second embodiment, the light beam including the video information displayed on the display unit 1 is reflected by the first mirror unit 2 and reaches the eyeball 3. An observer observes an image displayed on the display unit 1.

  Here, the first mirror unit 2 is translucent. On the same optical axis as the luminous flux from the display unit 1 that is reflected by the first mirror unit 2 and directed toward the eyeball 3, on the opposite side (to the eyeball 3) of the first mirror unit 2, the camera unit 4 for photographing the eyeball. Is placed. The eyeball camera unit 4 observes the observer's eyeball 3 via the semitransparent first mirror unit 2.

  Furthermore, the second mirror unit 11 provided behind the eyeball camera unit 4 causes the optical axis of the camera unit 5 for direction-of-sight shooting to be reflected from the first mirror unit 2 from the display unit 1 toward the eyeball 3. Match on the same optical axis as the luminous flux. As described above, the optical axis of the light beam from the display unit 1 reflected by the first mirror unit 2 toward the eyeball 3, the optical axis of the eyeball camera unit 4, and the optical axis of the visual direction camera unit 5 are determined. Match. Therefore, by aligning the observer's line of sight measured by the eyeball camera unit 4 with the coincident optical axis, the observer can always observe a real image in the line of sight and can receive information by a virtual image.

  Here, although the first mirror unit 2 is translucent, it may be a polarization separation mirror. Further, the first mirror section 2 may be a visible light reflecting and infrared light transmitting mirror.

Embodiment 3 FIG.
FIG. 3 is a diagram showing the configuration of the display unit 10 of the image display apparatus according to Embodiment 3 of the present invention. The image display device according to the third embodiment is substantially the same as the image display device according to the first embodiment. Therefore, the same parts are denoted by the same reference numerals and description thereof is omitted.

  In the image display device according to the third embodiment, the display unit 1 is a transmissive image display element 102 such as an LCD (Liquid Crystal Digital) panel. The image display element 102 is illuminated with a light beam from the light source unit 102. The light beam that has passed through the image display element 102 is reflected by the polarizing beam splitter 14 in the direction of the eyeball 3 of the observer, and is projected onto the retina in the eyeball 3 by the eyepiece lens 103. Therefore, since the observer can observe the virtual image of the video displayed on the image display element 102, the real video taken by the camera unit 5 for photographing the line of sight can be observed with an actual sense of distance.

Embodiment 4 FIG.
4 and 5 show calibration for matching the optical axis of the light emitted from the display unit 1 of the image display apparatus according to Embodiment 4 of the present invention with the optical axis of the camera unit 4 for eyeball photography. It is a figure which shows a method. As the image display device, for example, the one according to Embodiment 1, 2, or 3 may be used.

When calibrating the optical axis, the display unit 1 and the eyeball camera unit 4 are used. The calibration procedure is as follows.
(1) The display unit 1 emits a calibration signal (first calibration signal) 41. This signal is emitted to a point that becomes the center of the optical axis (one point that is perpendicular to and passes through the center of the eyeball 3) after the calibration is completed.
(2) The light emitted from the display unit 1 enters the eyeball 3 as the second calibration signal 42, and the light is further reflected by the eyeball 3 to become the third calibration signal 43.
(3) The reflected light (third calibration signal 43) enters the eyeball camera unit 4 through the first mirror unit 2. When the optical axis centers coincide with each other, the calibration signal irradiation position 44 becomes the center of the image photographed by the eyeball photographing camera unit 4 as shown in the left part of FIG. If the optical axis centers do not match, as shown in the right part of FIG. There is a possibility that the third calibration signal 43 does not enter the eyeball camera unit 4.

  The optical axis of the light emitted from the display unit 1 and the optical axis of the eyeball photographing camera unit 4 can be matched by using the information on the deviation as described above. That is, the image information of the eyeball photographed by the eyeball camera unit 4 is transmitted to a computer terminal (not shown) via wireless or wired communication, and the computer terminal is connected to the first rotary drive unit 7 and the second rotational drive unit 7. The rotary drive unit 8 and the slide type drive unit 9 may be controlled and driven according to the image information. By periodically performing such a calibration procedure in a time shorter than the time recognizable by the observer, the observer's line of sight can be measured.

Embodiment 5. FIG.
FIG. 6 is a diagram showing the configuration of the display unit 10 of the image display apparatus according to Embodiment 5 of the present invention. The image display device according to the fifth embodiment is substantially the same as the image display device according to the third embodiment. Therefore, the same parts are denoted by the same reference numerals and description thereof is omitted.

  In the image display device according to the fifth embodiment, the eyeball camera unit 4 is an infrared camera. That is, the infrared light source 21 provided in the eyeball camera unit 4 irradiates the eyeball 3 with the infrared irradiation 22 so that the observer's line of sight can be measured without being noticed by the observer. That is, the line-of-sight measurement does not interfere with the observer's video observation. Furthermore, the loss of light from the display unit 1 and light observed by the eyeball observation camera unit 4 can be reduced by using the first mirror unit 2 as the visible light reflecting infrared light transmission mirror 23.

Embodiment 6 FIG.
FIG. 7 is a diagram showing a configuration of the display unit 10 of the image display apparatus according to Embodiment 6 of the present invention. The image display device according to the sixth embodiment is substantially the same as the image display device according to the fifth embodiment. Therefore, the same parts are denoted by the same reference numerals and description thereof is omitted.

  In the image display device according to the sixth embodiment, a light beam received by the eyeball camera unit 4 is incident on the back surface of the imaging device 25 such as a CCD or CMOS that constitutes the camera unit 5 for capturing the eye direction. The eyeball camera unit 4 is integrally formed so as to be processed. That is, the imaging device 25 includes a circuit that performs an imaging (image receiving) process of the eyeball camera unit 4. In general, the image pickup device 25 such as a CCD or CMOS is formed on a silicon substrate. However, even if infrared light transmitted through the silicon substrate is incident from the back surface, the camera unit 5 for photographing in the line-of-sight direction can pick up an image.

  With such a configuration, the imaging device 25 and the electronic circuit necessary for imaging can be shared by the camera unit 5 for photographing the eye direction and the camera unit 4 for photographing the eyeball, so that the number of components can be reduced and the display unit 10 can be reduced. The whole can be reduced in size or weight.

  In addition, a shutter 24 is provided in the front part of the camera unit 5 for photographing the line-of-sight direction. The shutter 24 is opened and the infrared light source 21 is extinguished when the real space image is captured by the camera unit 5 for photographing the line of sight, and the infrared light source 21 is turned on when the eyeball is photographed by the camera unit 4 for photographing the eyeball. Therefore, the real space image and the eyeball image are not shot simultaneously. Further, when the eyeball is photographed by the eyeball photographing camera unit 4, the shutter 24 is closed. If an image captured immediately before as a real image is presented on the display unit 1 during this period, the observer feels uncomfortable. Not give.

Embodiment 7 FIG.
FIG. 8 is a diagram showing the configuration of the display unit 10 of the image display apparatus according to Embodiment 7 of the present invention. The image display device according to the seventh embodiment is substantially the same as the image display device according to the sixth embodiment. Therefore, the same parts are denoted by the same reference numerals and description thereof is omitted.

  In the image display device according to the seventh embodiment, the image display element 102 of the display unit 1 is a reflection type such as an LCD panel or DLP. The light beam from the light source 101 passes through the visible light polarization beam splitter 26, is reflected by the image display element 102, is reflected again by the visible light polarization beam splitter 26 in the direction of the observer's eyeball 3, and the eyeball 3 forms the eyeball 3. Projected on the inner retina. Here, the visible light polarization beam splitter 26 transmits infrared rays, and the eyeball camera unit 4 observes the eyeball with the eyeball observation camera 4 by the infrared irradiation 22.

  Since the reflection type device as the image display element 102 is generally easy to achieve high definition, an observer can observe a high definition image.

Embodiment 8 FIG.
FIG. 9 is a diagram showing the configuration of the display unit 10 of the image display apparatus according to Embodiment 8 of the present invention. The image display device according to the eighth embodiment is substantially the same as the image display device according to the sixth embodiment. Therefore, the same parts are denoted by the same reference numerals and description thereof is omitted.

  Infrared light emitted from the infrared light source 21 provided in the eyeball observation camera 4 is reflected by the visible light transmission / infrared reflection mirror 27 and the infrared reflection / visible light half mirror 28 and travels toward the eyeball 3. The infrared rays reflected by the eyeball 3 are reflected by the infrared reflection / visible light half mirror 28 and the visible light transmission / infrared reflection mirror 27 and are imaged by the eyeball observation camera 4. The line-of-sight imaging camera unit 5 captures an actual video image that is incident on the infrared reflection / visible light half mirror 28 via the correction lens 29.

  In the image display device according to the eighth embodiment, the observer can directly see the real space through the eyepiece lens 103 and the correction lens unit 29. The line-of-sight imaging camera 5 can grasp the brightness of the real space viewed by the observer by photographing the line-of-sight direction of the observer. Therefore, the display unit 1 presents only virtual information, but the superimposition position with the real space viewed by the observer and the brightness to be presented can be adjusted, so that it is possible to provide an image that does not feel strange to the observer.

Embodiment 9 FIG.
FIG. 10 is a diagram illustrating the operation of the display unit 10 of the image display device according to the ninth embodiment of the present invention.

  In FIG. 10, the display unit 10 moves along a moving surface 51 that is a spherical surface. That is, the moving surface 51 is a moving spherical surface that restricts the movable range of the display unit 10. The moving spherical surface 51 is arranged with the display unit 10 so that the optical axes of the visual direction photographing camera unit 5 and the eyeball photographing camera unit 4 always coincide with the visual line direction even when the human eyeball faces in various directions. Is a spherical surface formed by Here, when the optical axis of the display unit 10 is parallel to the line extending from the rotation center 52 of the eyeball 3 and the distance is constant, the optical axes always coincide. With such a configuration, after the initial calibration is performed, the optical coincidence state can be maintained only by tracking the movement of the eyeball 3 along the moving spherical surface 51.

  The movement of the display unit 10 along the moving spherical surface 51 is performed by, for example, the first rotary drive unit 7, the arm unit 6, the second rotary drive unit 8, and the slide drive unit 9 shown in FIG. .

  The mechanism for controlling the movement of the display unit 10 along the moving spherical surface 51 may be different. For example, a display unit moving guide for moving the display unit 10 along the moving spherical surface 51 is arranged, a plurality of first position control magnets are arranged on the display unit moving guide, and the display unit 10 has a second position. Arrange the control magnet. Here, the first position control magnet is composed of an electromagnet, and the second position control magnet is composed of a permanent magnet. The display unit 10 can be moved along the display unit movement guide by adjusting the magnetic force of the first position control magnet.

Embodiment 10 FIG.
FIG. 11 is a diagram showing a concept of virtual information presentation of the image display apparatus according to Embodiment 10 of the present invention.

  The observer views the information 63 at the observer's viewpoint with the position of the focal position 62 of the eye as the viewpoint. At this time, when the virtual information 61 is information at a position farther from the observer than the focal position 62 of the eye, the shape appears blurred in the real space, whereas the conventional image display device is based on the information. The shape looks clear without blurring. Therefore, there is a gap between the virtual display position and the actual appearance for the observer. This gap causes the observer to become unrecognizable.

  Therefore, in the image display apparatus according to Embodiment 10 of the present invention, virtual information other than the vicinity of the distance of the focal point 62 of the eye, which is the viewpoint of the observer, is presented in a blurred manner. Thus, by presenting the virtual information other than the vicinity of the observer's viewpoint according to the distance of the observer's viewpoint, the observer can see where the virtual product is placed in the real world. Can know exactly what to do.

Embodiment 11 FIG.
FIG. 12 is a diagram showing a viewpoint measurement method according to Embodiment 11 of the present invention.

  The line-of-sight intersection position 37 is a point where the lines of sight of the right eyeball 31 and the left eyeball 34 intersect. That is, it corresponds to the observer's viewpoint 62 (see FIG. 11). There are eyeball rotation centers 33 and 36 inside the left and right eyeballs 31 and 34, and the distance between the rotation centers of each eyeball is L. The angles θ1 and θ2 formed by the line segment L between the rotation centers and the line of sight of the eyeballs 31 and 34 can be calculated by obtaining a vector of the rotation center of the eyeball and measuring the line-of-sight direction of the eyeball. By determining these three parameters (L, θ1, θ2), the coordinates of the line-of-sight intersection position 37 are calculated. As a result, the distance L1 between the center of the left eye and the line-of-sight intersection L1, and the distance L2 between the center of the right eye and the line-of-sight intersection L2 are obtained. The observer's viewpoint 62 can be identified.

  FIG. 13 is a block diagram showing an image display apparatus according to Embodiment 11 of the present invention. It may be the same as that according to the first embodiment. The image displayed on the display unit 1 built in the display unit 10 is reflected by the first mirror unit 2 and projected onto the eyeball 3, and the observer observes the image displayed on the display unit 1. Further, the first mirror unit 2 is, for example, translucent, and the direction in which the observer's eyeball 3 is directed, that is, the line of sight, is observed by the eyeball camera unit 4. With such a configuration, the line-of-sight directions 32 and 35 of the left and right eyeballs 31 and 34 can be measured. If the eyeball camera unit 4 transmits the information on these line-of-sight directions to a computer terminal (not shown) via wireless or wired connection, the computer terminal (not shown) will be the observer's line-of-sight intersection position. 37 and the position of the observer's viewpoint can be calculated.

Embodiment 12 FIG.
First, FIG. 14 shows a conceptual diagram of the directivity of the light source 101 of the display unit 1 in the display unit 10 and how the observer sees. FIG. 14 (a) is a conceptual diagram of how it looks when a diffused light source is used. As shown in FIG. 14A, at the time of the diffuse light source, the distance from the eyeball on the virtual display surface 202 depends on the distance between the eyepiece lens 103 and the actual display position 201 with respect to the eyeball position 200. It is determined.

  FIG. 15 shows an example of the configuration of the display unit 1 at the time of a diffused light source. A light source 301 is disposed in the vicinity of an image display element (video plate) 302. Since the illumination light transmitted through the image display element (image plate) 302 is diffused in all directions, the observer adjusts the lens of the observer's eyes (that is, the crystalline lens) through the eyepiece lens 303. A virtual image of the image display element (video plate) 302 is observed. At this time, the position of the virtual image is a fixed position determined by the focal length of the eyepiece lens 303 and the distance between the eyepiece lens 303 and the image display element (video plate) 302. That is, since the observer always sees the virtual image at the same position, even if an image in the real space is displayed on the image display element (video plate) 302, a real sense of distance cannot be obtained. In addition, even if the virtual space image is superimposed on the video of the image display element (video board) 302, the observer can grasp the position (distance) where the virtual product should be placed in the real world from the sense of distance. I can't. Therefore, the viewer's visual image is uncomfortable.

  On the other hand, FIG.14 (b) is a conceptual diagram of the appearance at the time of a directional light source. When the directional light source is used, the distance from the eyeball of the virtual display surface 202 can be changed in the variable range 203 without depending on the distance between the position of the eyepiece 103 and the actual display position 201 with respect to the eyeball position 200. That is, the observer can see a virtual image at a desired position.

  FIG. 16 is an example of a configuration of the display unit 1 at the time of a directional light source, and is a diagram illustrating a configuration of the display unit 10 of the image display device according to the twelfth embodiment. In the display unit 10, the light beam emitted from the light source 301 is transmitted through the image display element 302, and the light beam is converged at the position of the pupil 308 of the eyeball 3 of the observer by the eyepiece lens 303. The video displayed on the image display element 302 is projected onto the screen.

  “A” in the figure is the width of the light beam on the pupil 308. When the directivity of the light beam emitted from the light source 301 is selected so that “a” is sufficiently smaller than the pupil diameter, a so-called Maxwellian state is obtained. At this time, the observer can see a clear image without depending on the lens effect of the crystalline lens 306. That is, with such a directional light source, even if the observer's viewpoint 62 changes, the video can be presented without blurring.

  The image display devices described in Embodiments 1 to 9 have a line-of-sight tracking configuration in which the optical axis of the display unit 10 is aligned with the line of sight of the observer. Therefore, by combining with the configuration as in the twelfth embodiment, a clear image can always be provided to the observer.

  When the display unit 10 using the directional light source according to the twelfth embodiment is used, the observer can arbitrarily set the position of the virtual image of the video displayed on the image display device corresponding to the position in the real space. Furthermore, as in the tenth or eleventh embodiment, the virtual information other than the vicinity of the observer's viewpoint distance is presented in a blurred manner according to the distance of the observer's viewpoint, so that the observer superimposes the real image and the virtual information. And correctly recognize where the virtual product is located in the real world.

  In the above description, it is assumed that the optical system constituting the display unit 10 is in a Maxwellian view. Actually, the light source 301 has a finite size, and the luminous flux emitted from the light source 301 also has a directivity distribution. Therefore, the light beam width on the pupil 308 is set to be 2 to 4 times the pupil diameter, that is, about 6 mm to 12 mm, and the size of the light source 301 and the light beam emitted from the light source 301 are set to have a directivity distribution. Then, it becomes a state close to Maxwell's view. At this time, the observer is less affected by the lens effect of the crystalline lens 106, and the observer can see a clear image even if the observer's line of sight and the optical axis of the display unit 10 are slightly shifted. By so doing, the accuracy of line-of-sight tracking can be somewhat relaxed, and the observer can easily attach the image display device.

Embodiment 13 FIG.
FIG. 17 is a conceptual diagram of the operation of the image display apparatus according to Embodiment 12 of the present invention. A concept of countermeasures to be taken when a failure occurs in the display unit 10 or a computer for displaying images will be described.

  The display unit 10 according to the present invention is always present on the line of sight of the person and at the center of the line of sight. Here, when a problem occurs in which the video cannot be presented due to a failure of the display unit 10 or a computer for displaying an image, the risk of human behavior increases. This is because the display unit 10 is in a state of blocking the observer's line-of-sight direction.

  Therefore, when the power of the display unit 10 is turned off, the display unit 10 is physically detached from the line of sight by, for example, a spring. Further, when no video is sent from a computer or the like for displaying an image, the display unit 10 automatically and immediately turns off the power of the display unit 10 so that the display unit 10 moves out of the line of sight by a spring. To do. Through these operations, the safety of the observer can be ensured.

It is a block diagram which shows the image display apparatus which concerns on Embodiment 1 of this invention. It is a figure which shows the structure of the display unit of the image display apparatus which concerns on Embodiment 2 of this invention. It is a figure which shows the structure of the display unit of the image display apparatus which concerns on Embodiment 3 of this invention. It is a figure which shows the method of the calibration for making the optical axis of the light radiated | emitted from the display part of the image display apparatus which concerns on Embodiment 4 of this invention correspond with the optical axis of the camera part for eyeball imaging | photography. It is a figure which shows the method of the calibration for making the optical axis of the light radiated | emitted from the display part of the image display apparatus which concerns on Embodiment 4 of this invention correspond with the optical axis of the camera part for eyeball imaging | photography. It is a figure which shows the structure of the display unit of the image display apparatus which concerns on Embodiment 5 of this invention. It is a figure which shows the structure of the display unit of the image display apparatus which concerns on Embodiment 6 of this invention. It is a figure which shows the structure of the display unit of the image display apparatus which concerns on Embodiment 7 of this invention. It is a figure which shows the structure of the display unit of the image display apparatus which concerns on Embodiment 8 of this invention. It is a figure which shows operation | movement of the display unit of the image display apparatus which concerns on Embodiment 9 of this invention. It is a figure which shows the concept of the virtual information presentation of the image display apparatus which concerns on Embodiment 10 of this invention. It is a figure which shows the method of the viewpoint measurement which concerns on Embodiment 11 of this invention. It is a block diagram which shows the image display apparatus which concerns on Embodiment 11 of this invention. (A) is a conceptual diagram of the appearance at the time of a diffused light source. (B) is a conceptual diagram of the appearance at the time of a directional light source. It is an example of a structure of the display part at the time of a diffused light source. It is an example of a structure of the display part at the time of a directional light source. It is a conceptual diagram of operation | movement of the image display apparatus which concerns on Embodiment 12 of this invention.

Explanation of symbols

DESCRIPTION OF SYMBOLS 1 Display part, 2 1st mirror part, 3 Eyeball, 4 Eyeball camera part, 5 Gaze direction camera part, 6 Arm part, 7 First rotary drive part, 8 Second rotary drive part 9 Slide-type drive unit, 10 Display unit, 11 Second mirror unit, 13 Eyepiece lens, 14 Polarizing beam splitter, 21 Infrared light source, 22 Infrared irradiation, 23 Visible light reflecting infrared transmission mirror, 24 Shutter, 25 Imaging device, 26 Visible Polarizing Beam Splitter, Infrared Light Transmitting Mirror, 27 Visible Light Transmission, Infrared Light Reflecting Mirror, 28 Infrared Light Reflecting, Visible Light Half Mirror, 29 Correction Lens Unit, 31 Right Eye Eyeball, 32 Right Viewing Line Direction, 33 Right Eye Eye rotation center, 34 Left eyeball, 35 Left eye direction, 36 Left eye eye rotation center, 37 Eye crossing position, 41 First caribbean Calibration signal, 42 second calibration signal, 43 third calibration signal, 44 calibration signal irradiation position, 51 moving spherical surface, 52 eyeball rotation center, 61 virtual information, 62 eye focus position, 63 observer's viewpoint Information, 200 eyeball position, 201 actual display position, 202 virtual display surface, 203 variable range, 301 light source, 302 image display element (video plate), 306 lens, 307 retina, 308 pupil.

Claims (11)

A display unit for displaying images;
A first mirror section for guiding the image displayed on the display section to an observer's eyeball;
A first camera unit that captures the direction of the eyeball of the observer and detects the line of sight of the observer;
A second camera unit whose optical axis coincides with the first camera unit and captures the line-of-sight direction of the observer;
A computer terminal that is wired or wirelessly connected to the first camera unit and the second camera unit, and synthesizes the video of the observer's line of sight imaged by the second camera unit and virtual information; Prepared,
An image display device, wherein a video image of the observer's line of sight and virtual information are superimposed and displayed on the display unit by a computer terminal. A display unit for displaying images;
A first mirror section for guiding the image displayed on the display section to an observer's eyeball;
A first camera unit that captures the direction of the eyeball of the observer and detects the line of sight of the observer;
A second camera unit whose optical axis coincides with the first camera unit and captures the line-of-sight direction of the observer;
An eyepiece and a correction lens unit that directly show the real space to the observer through the first mirror unit;
The second camera unit is connected to the second camera unit by wire or wirelessly, receives an image of an observer's line of sight imaged by the second camera unit, and displays virtual information on the display unit based on the information of the image An image display device comprising a computer terminal. The display unit, the first mirror unit, the first camera unit and the second camera unit are included in a display unit,
A drive unit that is connected to a computer terminal wirelessly or by wire to drive the display unit is further provided.
In order to make the optical axis of the first camera unit coincide with the observer's line of sight, a part of the video display range of the display unit is turned on, and the first camera unit captures information on the lighting position reflected in the eyeball. 3. The image display device according to claim 1, wherein the computer terminal controls the drive unit so that the lighting position reflected in the eyeball is at the center of the pupil of the eyeball. . The image display apparatus according to claim 1, wherein the first camera unit includes an infrared irradiation unit, and images an eyeball with infrared rays from the infrared irradiation unit. The image display device according to claim 4, wherein the image sensor of the first camera unit and the image sensor of the second camera unit are integrally formed. The image evaluation apparatus according to claim 1, wherein the image display unit provided in the display unit is a reflective element. The computer terminal controls a drive unit of the display unit so that the display unit moves on a spherical surface centering on the rotation center of the eyeball of the observer so as to follow the line of sight of the observer. The image display apparatus as described in any one of Claims 1-6. The image display apparatus according to claim 1, wherein, in the virtual information by the computer terminal, a video other than the vicinity of the observer's viewpoint is blurred. Information on the first line of sight detected from the first camera unit for one eye and information on the second line of sight detected from the first camera unit for the other eye are transmitted to the computer terminal. ,
The image display device according to claim 1, wherein the computer terminal determines an observer's viewpoint from the first line-of-sight information and the second line-of-sight information. The image display according to any one of claims 3 to 9, wherein a light beam emitted from the display unit has directivity and has a width not more than four times a normal pupil size. apparatus. The image display device according to any one of claims 3 to 10, wherein when the display of the display unit stops, the display unit moves from the front of the observer's eyeball.

Images