JP2005311754A - Head mounted video display device provided with image pickup camera and pupil position detection function - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To realize a head mounted video display device that is small and light even though the video display device incorporates a function for picking up an outside world and a function for detecting the pupil positions of an observer. <P>SOLUTION: This head mounted video display device having the image pickup camera and the function for detecting the pupil positions of an observer has a first optical system for guiding image light of the pupils of the observer to the image pickup camera and a second optical system for guiding image light of the outside world to the image pickup camera, wherein the image light of the pupils is light a having fixed wavelength that does not affect the observer such as infrared light, the first optical system is configured so as to guide only the light a and the second optical system is configured so as to interrupt components of the light a included in an outdoor daylight image. <P>COPYRIGHT: (C)2006,JPO&NCIPI

Description

  The present invention relates to a head-mounted image display apparatus having a camera that captures an external image and a function of detecting a pupil position of an observer.

  A head-mounted display device equipped with a display device and an imaging camera corresponding to the left and right eyes of the observer, respectively, so that an external image captured by the imaging camera is reflected on the display device is known as a video see-through head mounted display ing. In this device, by aligning the optical axis of the imaging optical system with the optical axis of the display optical system, it is possible to see the external image viewed through the imaging camera with the same line of sight as if it were viewed with one's own eyes. There is something that you can do. This device is useful for, for example, research on mixed reality in which a so-called real space and virtual space are merged by superimposing a virtual world image created by computer graphics on a real-world image captured by an imaging camera. Used.

  On the other hand, in such an apparatus, a line-of-sight input apparatus that detects a direction seen by an observer and uses it as an input means is known (Japanese Patent Laid-Open No. 08-050256). In order to realize this, it is necessary to detect the position of the observer's pupil.

  Originally, such a head-mounted video display device is preferably mounted on a human head, so it is desirable that the head-mounted image display device be small and lightweight. However, a device incorporating the above functions must be complicated and large. I do not get.

  Therefore, the present invention aims to realize a head-mounted image display device that is small and lightweight while incorporating the above-described function of imaging the outside world and the function of detecting the observer's pupil position. .

  The present invention is characterized in that an imaging camera for capturing an external image provided corresponding to the left and right eyes of the observer and a camera for imaging the observer's pupil are shared. For this purpose, it has a first optical system that guides the image light of the pupil of the observer to the imaging camera, and a second optical system that guides the image light of the outside world to the imaging camera. Furthermore, the image light of the pupil is light a having a constant wavelength that does not affect the observer, such as infrared light, the first optical system is configured to guide only the light a, and the second optical system is The light a component included in the external light image is blocked. In addition, a separation unit that separates the pupil image and the external image from the pupil image light and the external image light guided to the imaging camera is provided.

  Furthermore, it has a detection means which detects a pupil position from the isolate | separated pupil image. Further, based on the detected pupil position, an indicator is displayed on the display device so that the optical axis center of the display optical system, that is, the center of the display panel matches the eye width of the observer, and based on the indicator, The optical system is configured to be movable so that the centers of the display panels of the left and right optical systems can coincide with each other.

  With the above-described configuration, in the imaging camera that captures the external image and the head-mounted image display device that has the gaze detection function, the camera that captures the external image and the observer's pupil for detecting the pupil position are provided. By sharing the cameras for imaging, the number of imaging cameras to be used can be reduced, so that a small and lightweight device can be realized. Furthermore, it is possible to easily match the optical axis (display panel center) of the display optical system with the eye width of the observer based on the pupil position information detected for line-of-sight detection. This is because the observer's eye width is usually different for each person, and considering the variation, it is necessary to increase the diameter of the observation pupil in terms of optical design, and the optical system itself becomes larger. However, since the observation pupil diameter can be reduced by matching the optical system with the eye width of the observer, the optical system itself can be reduced. That is, it can contribute to reduction in size and weight of the apparatus.

  As described above, in a head-mounted display device having a camera for capturing an external image and a pupil position detecting function, the camera for capturing an external image and the camera for capturing an observer's pupil can be used in common. The size and weight can be reduced, and the cost can be reduced by reducing the number of imaging cameras. Further, by aligning the left and right optical units with the eye width of the observer based on the detected pupil position data of the observer, the optical system can be made smaller and further reduced in size and weight.

(First embodiment)
FIG. 1 and FIG. 2 are perspective views of a head-mounted image display device (Pideoseal HMD) according to an embodiment of the present invention. In the figure, reference numeral 1 denotes a display unit, and imaging camera windows 1a and 1b and observation windows 1c and 1d for viewing displayed images are provided at positions corresponding to the left and right eyes of the observer. A pad 2 is provided at the center of the display unit 1 via the arm 2a to press the wearer's forehead when worn. Leaf springs 3-R and 3-L are integrally attached to both sides of the arm portion 2a in the left-right direction in a circular arc shape. Further, frames 4-R and 4-L are integrally attached to the leaf springs 3-R and 3-L. Frames 4-R and 4-L are partially stored in the frames 4-R and 4-L, and extend rearward from the rear ends 4a-R and 4a-L of the frames 4-R and 4-L. The flexible bands 8 are connected to each other. A occipital region presser 5 is provided at the center of the band 8 to press the occipital region of the wearer when the band 8 is attached. Further, knobs 6-R and 6-L are integrally provided at the end of the band 8 on the display unit side. The knobs 6-R and 6-L are slidably mounted in the longitudinal direction of the frames 3-R and 3-L, and freely move in the direction of arrow A and in the direction of arrow B by a ratchet mechanism (not shown). Is supposed to be locked. This lock is released by pushing the release knobs 7-R and 7-L provided on the knobs 6-R and 6-L upward in FIG. When the lock is released, the knobs 6-R and 6-L can freely slide in the direction of arrow B.

  As described above, the knobs 6-R and 6-L are slid in the directions of arrows A and B in accordance with the size of the observer's head, and the front head pressing pad 2 and the back head pressing 5 respectively The display unit 1 is fixed to the observer's head by pressing the front and back heads.

  FIG. 3 shows a state where the apparatus is mounted. As shown in FIG. 3, the observer wears the apparatus on the head and observes the external image captured through the windows 1a and 1b provided on the display unit 1 through the observation windows 1c and 1d. At this time, the optical axis of the imaging optical system and the optical axis of the display optical system are optically adjusted so that the imaging direction coincides with the observation direction as indicated by L1. Thereby, the observer can see the external world image imaged with the imaging camera with the same feeling as having seen with his gaze.

  4 is a perspective view showing the inside of the display unit with the exterior cover removed, and FIG. 5 is a side view thereof.

  In the figure, 12-R is a prism lens constituting the display optical system, 13-R is a prism for guiding an external field image to an imaging camera, and 14-R is a CCD that captures the guided external field image, which will be described later. An imaging camera composed of an imaging lens to be imaged on the element and the imaging element, 15-R is a display unit having a display panel to be described later for displaying an image, and these are integrated with the bases 10-R and 11-R. Fixed. The optical unit 20-R corresponding to the right eye of the observer by the prism lens 12-R, the prism 13-R, the imaging camera 14-R, and the display unit 15-R fixed to the bases 10-R and 11-R. Is configured.

  The optical unit 20-R is rotatably attached to the shaft 17 rotatably supported by the main body cover at the protruding portions 10a-R of the base 10-R and the protruding portions 11a-R of the 11-R.

  Here, the protrusion 11a-R of the base 11-R and the 17a portion of the shaft 17 corresponding to the protrusion 11a are screwed to each other and are screw-fitted.

  Similarly, an optical unit 20-L corresponding to the left eye of the observer includes a prism lens 12-L, a prism 13-L, an imaging camera 14-L, a display unit 15-L, and a base 10-L for fixing them. 11-L. Then, the optical unit 20-L, like the optical unit 20-R, is mounted on the shaft 17 rotatably supported by the main body cover, on the projecting portions 1Oa-L and 11-L of the base 10-L. L is rotatably attached. Further, the protruding portion 11a-L of the base 11-L and the 17b portion of the shaft 17 corresponding thereto are screwed in the same manner as the optical unit 20-R. Here, the screw portions 17a and 17b are in a reverse screw relationship. Therefore, when the knob 18 provided at the center of the shaft 17 is rotated from the outside, the optical units 20-R and 20-L move toward and away from each other along the shaft 17 as indicated by arrows D and C. It is possible to move in the direction to do. Reference numeral 19 denotes a rotation stopper provided on the display cover side so that the optical units 20-R and 20-L do not relatively rotate when the knob 18 is rotated.

  6 and 7 are schematic views for explaining the optical functions of the optical unit. FIG. 6 is a plan view of the right optical unit 20-R, and FIG. 7 is a cross-section EE along the optical axis L1.

  Here, in order to distinguish the left and right optical units in the above description, the symbols R and L are given. However, since they are functionally the same, the symbols R and L are omitted in the following description.

  In FIG. 6, the external image light L3 is reflected by the half mirror surface 13b of the prism 13 as indicated by arrows L5 and L6, and further totally reflected by the surface 13a whose angle is set so as to satisfy the total reflection condition. 22 forms an image on the surface of an image sensor 23 such as a CCD. At this time, hot mirror coating is applied to the surface 13a of the prism 13, and the infrared component included in the external image light is reflected on the surface 13a. Therefore, only the visible light component of the external image light is formed on the surface of the image sensor 23.

  On the other hand, the image light L8 of the pupil 100 of the observer irradiated by the LEDs 24, 24 having only the infrared component so as not to affect the observer is transmitted through the surface 12a of the prism lens 12 coated with the cold mirror, The light passes through the correction lens 16 and the half mirror surface 13b of the prism 13, and is reflected by the surface 13a of the prism 13 coated with hot mirror. The pupil image light reflected by 13a is reflected by the half mirror surface 13b of the prism 13, similarly to the external image light, further totally reflected by 13a, and imaged on the image sensor 23 by the imaging lens 22. .

  On the other hand, the image light L9 displayed on the display unit 15 constituted by the liquid crystal display panel 20 and the backlight 21 is on the surfaces 12b and 12a having the free curved surface shape of the prism lens 12 as indicated by arrows L10 and L11. It is reflected and enlarged and projected onto the observer's pupil 100.

  Here, the external image light containing only the visible light component is partially transmitted through the half mirror surface 13b of the prism lens 13, but is blocked by the 12a surface of the cold mirror-coated prism lens 12 and affects the observer. There is nothing.

  FIG. 8 is a block diagram showing an image processing operation and corresponds to the right optical unit 20-R shown in FIG. Since the block diagram corresponding to the left optical unit 20-L is the same as that of the right optical unit, a description thereof will be omitted.

  As described above, the observer's pupil image and external image captured by the imaging camera 14 are sequentially captured into the image memories 1 and 2 for each screen via the input / output control unit 101 of the control device such as a personal computer. It is. At this time, the infrared LED 24 blinks in synchronization with the image for each screen captured by the imaging camera 14. Accordingly, the image of the imaging camera captured at the timing when the infrared LED 24 is turned on becomes an image in which the external image and the observer's pupil image overlap as shown in FIG. As shown in FIG. 9 (b), the image captured at step S is an image of only the external world image.

  The former image is captured in the image memory 1 and the latter image is captured in the image memory 2. Images captured in the memories 1 and 2 are calculated by the image processing unit 102, and only the pupil image is extracted as shown in FIG. The pupil position detection unit 103 obtains the center position of the pupil from the extracted pupil image. The pupil center position data is sent to the eye width adjustment instruction index creation unit 104.

  Since the positional relationship between the imaging center of the imaging camera and the center of the display panel is known as a device-specific value, the center position of the display panel is based on the center position data of the observer's pupil determined by the pupil position detection unit 103. Can be calculated.

  For example, when the center position of the display panel of the optical unit and the pupil center position of the observer are shifted as shown in FIG. 10A, as a result, the optical unit must be moved to the left in FIG. An arrow F shown in FIG. 9 serving as an index for moving the unit is created in the eye width adjustment instruction index creating unit 104. The created arrow F is sent to the image composition unit 105. At the same time, only the image of the external world image taken into the memory 2 and a virtual image created by computer graphics or the like from the image generation unit 106 are sent to the image synthesis unit 105, and are superposed on each other via the input / output control unit 101. It is sent to the display panel 20.

  Based on the arrow F displayed on the liquid crystal display panel 20, the observer rotates the knob 18 in the direction corresponding to the arrow to move the left and right optical units 20-R and 20-L. When the observer's pupil center and the center of the display panel coincide as shown in FIG. 10B, the arrow F disappears. As described above, the observer can easily and accurately confirm that the eye width of the observer matches the display panel centers of the left and right optical units.

  In adjusting the eye width, it is of course possible to automatically adjust the eye width by moving the optical unit using a motor or the like instead of rotating the knob 18 by hand.

The perspective view which shows the whole apparatus by this invention The perspective view which shows the whole apparatus by this invention The side view which shows the state with which the apparatus by this invention was mounted | worn The perspective view which shows the state which removed the exterior cover Side view showing the state with the exterior cover removed The top view which shows the outline of the optical system of this invention Sectional view of FIG. Block diagram showing image processing according to the present invention. Explanatory drawing showing image processing Explanation of eye width adjustment

Explanation of symbols

DESCRIPTION OF SYMBOLS 1 Display part 2 Front head pressing pad 3 Leaf spring 4 Frame 5 Back head pressing 6 Knob 7 Release knob 8 Band 10 Base 11 Base 12 Prism lens 13 Prism 14 Imaging camera 15 Display unit 16 Correction lens 18 Knob 19 Rotation stop 20 LCD panel 21 Backlight 24 Infrared LED

Claims (3)

  In a head-mounted image display device having a function of detecting an imaging camera and an observer's pupil position, a first optical system that guides the image light of the observer's pupil to the imaging camera, and the imaging of external image light A second optical system that leads to the camera, and the image light of the pupil is light a having a constant wavelength that does not affect the observer, such as infrared light, and the first optical system receives only the light a A head-mounted image display device configured to guide the second optical system so as to block a component of light a included in an external light image.   The head-mounted image display device according to claim 1, further comprising a separating unit that separates a pupil image and an external image captured by the imaging camera from each other.   The left and right pupil positions of the observer are detected from the separated pupil images, and based on the detected positions, the center of the display panel of the left and right optical units provided corresponding to the left and right eyes of the observer, and the eye width of the observer are determined. In order to match, it has an index presenting means for presenting an index on the display panel, and on the basis of the index, the right and left optical units can be moved so that the center of the display panel of the optical unit matches the eye width of the observer. The head-mounted image display device according to claim 1.

Images