JPH10142552A - Head mounted display - Google Patents

Abstract

PROBLEM TO BE SOLVED: To eliminate problems of such that a visual angle of an outside field picture was narrow and light quantity is insufficient, or videos are not observable superimposed. SOLUTION: A HMD(head mounted display) 1 comprises an image pickup part 1, an image processing part 3, a signal input part 4, an image synthesizing part 5, and an image display part 6, and in the image pickup part 2, an image signal outputted from CCD is inputted to the image synthesizing part 5 via the image processing part 3 and the image synthesizing part 5 superimposes the image signal on a video signal inputted from the signal input part 4 and the image is displayed by the image display part 6 positioned in optical axes of the pupils of a HMD mounter, therefore, two videos are electrically superimposed and observable with a more natural angle of visibility and a sufficient light quantity.

Description

DETAILED DESCRIPTION OF THE INVENTION

[0001]

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention relates to a head-mounted display, and more particularly, to a head-mounted display in which the outside world can be viewed.

[0002]

2. Description of the Related Art Head mounted displays (hereinafter referred to as HM)
D) is mounted on the user's head and displays a video signal input from a video signal input terminal or the like on a built-in image display device disposed in front of both eyes. This allows the wearer of the HMD to view the image.

On the other hand, in order to be able to recognize the external environment in a closed state, such as when an HMD is mounted,
In Japanese Patent Application Laid-Open No. 8-160349, a transmissive mode is realized by employing a half mirror and a mechanical shutter or an optical shutter, and has a switching structure in which the outside world can be seen in addition to the image display. That is, by opening the mechanical shutter or the optical shutter in this switching structure, the scene of the external environment can be observed through the half mirror, and the video signal input from the video signal input terminal or the like can be changed to the same image signal. By being reflected by a half mirror so that observation is possible, H
The structure allows the wearer of the MD to observe one or both of the superimposed images.

FIG. 6 is a schematic diagram showing a schematic configuration of the conventional HMD. The superposition of images by the half mirror 101 will be described with reference to FIG. Generally, the shutter mechanism 102 opens and closes according to an opening / closing signal, and passes / blocks the scenery of the external environment accordingly. In the case of a mechanical shutter, it is an opening / closing operation of a mechanism, and in the case of an optical shutter, a liquid crystal shutter. Is realized by electric control for transmission control. The transmitted light transmitted by these opening / closing signals passes through the half mirror 101 arranged at 45 ° to the optical axis, the light amount is attenuated, and is observed by the wearer of the HMD.

A video signal input from a video signal input terminal or the like is displayed on a display element LCD 103 arranged in parallel with an optical axis input to an eye during observation.
Half mirror 1 arranged at 45 ° to the optical axis 03
The light is reflected at 01 and the amount of light is attenuated and observed by the wearer of the HMD.

On the other hand, when observing only the scenery of the external environment, the shutter mechanism 102 is kept open by an open / close signal to suppress the video signal. On the other hand, when observing only an external video signal, the shutter mechanism 102 is kept closed by the open / close signal, and the video signal is displayed.
If the shutter mechanism 102 is kept open and a video signal is displayed on the display device LCD 103 at the same time, both the scenery and the image are superimposed and observed.

On the other hand, as a visual aid for the visually impaired, an LCD is arranged in front of the left or right eye to display an image picked up by a CCD.
49 is also known.

[0008]

The above-mentioned conventional head mounted display has the following problems. In the former case, the amount of transmitted light is attenuated due to the use of a half mirror, and the outside scene must be darkened, and the viewing angle is narrow due to the arrangement of a half mirror etc. in front of both eyes. I had to be.

[0009] On the other hand, when an image of the outside world is displayed unilaterally in front of one eye, its use is specified, and even if an image is displayed in front of the other eye, it is inevitably lacking in interest.

SUMMARY OF THE INVENTION The present invention has been made in view of the above problems, and has as its object to provide a head mounted display that allows an image of the outside world to be viewed in a sufficiently natural state and also to be visually recognized together with an image.

[0011]

According to one aspect of the present invention, there is provided a head mounted display which is mounted on a user's head and has image display means at a position visible to an observer. Image capturing means for capturing an external image, external video signal input means for inputting an external video signal, external image captured by the image capturing means, and external image input by the external video signal input means. And an image synthesizing unit for displaying the image from the image display unit on the image display unit. In the invention according to claim 1 configured as described above, the external image captured by the image capturing unit is input to the image combining unit together with the video signal input from the outside by the external video signal input unit, and the image combining unit The external image and the image from outside are superimposed by the means and displayed on the image display means arranged at a position visible to the observer.

The invention according to a second aspect is configured such that the image pickup means includes a CCD image pickup device and an objective lens unit and picks up an external image at a natural viewing angle.
According to this configuration, since the imaging unit includes the CCD imaging device and the objective lens unit, the viewing angle and the like can be set under desired conditions, and an external image is captured at a natural viewing angle. .

Further, the invention according to claim 3 is configured such that the external image is displayed as a real-size virtual image. According to this configuration, various types of optical systems and the like may be provided to display the external image as a virtual image at the actual size, and the objective lens unit described above may be used for the display size. However, it can also be realized by an optical system that forms the virtual image.

Further, according to a fourth aspect of the present invention, the image pickup means is arranged in a pair corresponding to the approximate binocular position, and the image display means is individually arranged in a pair corresponding to the both eyes. The image synthesizing unit is configured to individually display the external image captured by the imaging unit on the image display unit. According to this configuration, since the image pickup unit and the image display unit are arranged as a pair corresponding to both eyes, the image combining unit displays the external images picked up by the image pickup unit individually on the image display unit. By doing so, display can be performed with a more natural feeling.

Further, in the invention according to claim 5, the image synthesizing means superimposes an external image picked up by the image pick-up means and an external image input by the external video signal input means. The configuration is such that it can be selectively executed as appropriate. According to this configuration, for example, in a system in which an image is displayed in accordance with an external image, it is necessary to superimpose the external image so that the external image is mainly used. If there is, it is necessary to superimpose images mainly. In such a case, the image synthesizing means appropriately and selectively executes the superposition of the external image and the video.

Further, the invention according to claim 6 is the audio reproducing means arranged at the position of both ears of the user, an external audio signal inputting means for inputting an audio signal accompanying a video signal from the outside, It is provided with a microphone for converting sound into an electric signal, and a sound synthesizing means for synthesizing the sound inputted from the microphone and the sound accompanying the video signal and reproducing the sound by the sound reproducing means. According to this configuration,
In addition to displaying the video, when the external audio signal input means inputs the audio signal accompanying the video signal from the outside and reproduces it with the audio reproduction means arranged at the binaural position of the user, the microphone In addition, the external sound is converted into an electric signal, and the sound input from the microphone and the sound accompanying the video signal are synthesized by the sound synthesizing means and reproduced by the sound reproducing means.

[0017]

Embodiments of the present invention will be described below with reference to the drawings. FIG. 1 is a block diagram of an HMD showing a first embodiment of the present invention, in which an input image from a small CCD camera is subjected to electric image processing and displayed on an image display device arranged in front of both eyes. It has a configuration. First,
The problems in the conventional device will be summarized again. The first problem is that, in the transmission mode for recognizing the external environment in a closed state when the HMD is mounted, the field of view at the time of transmission is narrow. The reason is that the wearer of the HMD has a structure in which a video signal input from a video signal input terminal or the like can be displayed on the image display device and watched. This is because the transmission mode is realized by opening and closing the external field of view with a mechanical shutter or an optical shutter so that a plurality of switching structures can be used for viewing the external world in addition to the image display.

A second problem is that the amount of transmitted light during external observation is not small in a transmission mode for recognizing an external environment in a closed state when the HMD is mounted. The reason is that when the HMD is mounted, the video signal input from the video signal input terminal or the like is displayed on the image signal display device so that the external environment can be viewed in the closed state when the HMD is mounted. The transmission mode is realized by opening and closing the external field of view with a mechanical shutter or an optical shutter so that it can be recognized,
This is because a half mirror is employed as a switching mechanism for viewing the outside world in addition to the image display, and since the images are optically superimposed, the amount of transmitted light must be attenuated.

In the following embodiment, these problems can be solved and the wearer of the HMD can recognize the external environment without removing the HMD when observing an external video signal on the image display device. HMD that secures a transparent wide viewing angle in the transmission mode and a sufficient amount of external light
To provide a transparent function.

Referring to FIG. 1, the HMD 1 of the present invention comprises an image pickup unit 2, an image processing unit 3, a signal input unit 4, an image synthesizing unit 5, and an image display unit 6. The signal input unit 4 has an input terminal for a video signal 8 from the outside, and is connected to the electronic image generation device 7 to convert an electronic image into an HM signal.
It can be observed on the image display unit 6 of D1. In general,
The electronic image generator 7 is a TV game machine or a TV image receiver having a video signal output. At this time, the video signal 8 becomes a video signal and is input to the signal input unit 4. In this sense, the signal input unit 4 constitutes an external video signal input unit.

The image pickup section 2 is a small camera having an image pickup device CCD. In the first embodiment, one image pickup section is attached near the eye to be integrated with the HMD 1. The mounting position is above, below, left or right of the HMD 1, but is preferably on a horizontal line connecting both eyes in the vertical direction and an intermediate position between both eyes in the horizontal direction in order to eliminate a sense of discomfort during image display. The image photographed by the small camera is displayed on one image display unit 6 arranged in front of the eyes of the wearer of the HMD 1.

An image signal output from the CCD in the image pickup unit 2 is input to an image processing unit 3 and subjected to electrical processing, and is converted into a signal substantially equivalent to an output signal from the signal input unit 4. You. Therefore, in the present embodiment, the image pickup section is constituted by the image pickup section 2 having the image pickup device CCD and the image processing section 3. Of course, not limited to the CCD,
Another image sensor may be used. In the image processing, various types of effect processing may be performed.

The image synthesizing section 5 superimposes images on the basis of signals input from the image processing section 3 and the signal input section 4, and enables selective superposition as described later. I have. In this sense, the image synthesizing unit 5 constitutes an image synthesizing unit.

The image display unit 6 is arranged on the optical axis of the HMD wearer's eye, and displays an image and a video image superimposed by the image synthesizing unit 5. In this sense, the image display unit 6 constitutes an image display unit. Although the image display unit 6 is capable of color display, it may be monochrome display.

Next, the operation of the first embodiment of the present invention will be described in detail with reference to FIG. FIG. 2 is a detailed configuration diagram of the HMD showing the first embodiment of the present invention. FIG.
Referring to FIG. 1, the image pickup unit 2 in FIG.
An image of the external environment is sensed on the image sensor CCD 13 through the objective lens 12 and input to the image processor 3 as an electric signal to be processed as an image signal. Further, the combination of the image signal and the signal input from the signal input unit 4 in FIG. 1 which is omitted in FIG.

That is, a normal mode in which a signal input from the signal input unit 4 is selectively displayed, a see-through mode in which an image signal from the image processing unit 3 is displayed, and a display priority order based on luminance signal information of both signals. Operation mode processing, such as a superimpose mode or the like, in which a composite display is decided upon, is possible. In the normal mode or the see-through mode, the signal may be selectively switched by a switch. In the superimpose mode, images may be superimposed using a superimpose circuit.

The image display section 6 shown in FIG. 1 comprises the display element LCD 14, the eyepiece (R) 15, and the eyepiece (L) 16 shown in FIG. HMD1 is displayed on the device LCD14.
Is observed through an eyepiece (R) 15 and an eyepiece (L) 16 corresponding to both eyes at the time of wearing.

Objective lens 12 constituting image pickup section 2
Employs a slightly wide-angle lens in order to secure a wide field of view in a range where there is no discomfort during image observation, and employs a lens with a large depth of field to eliminate the need for focusing.

Further, the eyepiece (R) 15 and the eyepiece (L) 16 adjust the diopter to the image display surface of the display element LCD 14, and convert the image of the external environment taken by the image pickup element CCD 13 into the image on the display element LCD 14. Can be observed as a full-scale virtual image.

Next, a second embodiment of the present invention will be described in detail with reference to FIG. FIG. 3 is a block diagram of the HMD 11 showing the second embodiment of the present invention. The third embodiment has a configuration in which each block in FIG. 1 is mounted in two systems corresponding to the left and right eyes respectively, and two small cameras are provided as a simple stereoscopic vision system. Referring to FIG. 3, the HMD 11 of the present invention is a right system corresponding to the right eye, and includes an image capturing unit 2R and an image processing unit 3R.
, A signal input unit 4R, an image synthesizing unit 5R, and an image display unit 6R.

The signal input unit 4R has an input terminal for the video signal 8 from the outside. By connecting the signal input unit 4R to the electronic image generator 7, an electronic image can be observed on the image display unit 6R of the HMD 11. Generally, the electronic image generator 7 is a TV game machine or a TV image receiver having a video signal output. At this time, the video signal 8 becomes a video signal and is input to the signal input unit 4R. Here, the current TV game machine and T
Since the video signal from the V image receiver is a two-dimensional display image, the same signal is input to the signal input unit 4R and the signal input unit 4L.

The image pickup section 2R is a small camera having an image pickup device CCD. In the second embodiment, one is attached near the right eye to be integrated with the HMD 11. The mounting position corresponds to both eyes, respectively, above, below,
Or, on the right, it is desirable to position on the horizontal line connecting both eyes in the vertical direction or the width of both eyes in the horizontal direction in order to eliminate discomfort during image display.

The left system corresponding to the left eye of the HMD 11 of the present invention includes an image pickup unit 2L, an image processing unit 3L, a signal input unit 4L, an image synthesis unit 5L, and an image display unit 6L. The connection relation of each part is the same as that of the right system except for a part of the above-described differences, and therefore, the description is omitted here.

The image taken by the small camera is an HMD
The right image and the left image are displayed on the two image display units 6R and 6L disposed in front of the 11 wearer's eyes, respectively, corresponding to the right system and the left system. Next, the operation of the second embodiment of the present invention will be described in detail with reference to FIG. FIG. 4 is a detailed configuration diagram of the HMD showing the second embodiment of the present invention. Each configuration in FIG. 2 is configured with two systems corresponding to the left and right eyes respectively, and has a configuration including two small cameras as a simple stereoscopic vision system. Referring to FIG. 4, the image capturing unit 2R in FIG. 3 includes an objective lens 12R and an image sensor CCD 13R, and an image of an external environment is passed through the objective lens 12R to the image sensor CC.
D13R, which is sensed on the image processing unit 3R as an electric signal.
And is processed as an image signal. Further, the image signal is normally displayed in the image synthesizing unit 5R in combination with the signal input from the signal input unit 4R in FIG. 3, which is omitted in FIG. 4, to selectively display the signal input from the signal input unit 4R. Operation mode processing such as a mode, a see-through mode for displaying an image signal from the image processing unit 3R, and a superimpose mode for performing composite display by determining a wired order of display based on luminance signal information of both signals.

The image display section 6R in FIG. 3 comprises the display element LCD 14R and the eyepiece (R) 15 in FIG.
The video signal processed by the image synthesizing unit 5R is displayed on one display element LCD 14R or one display element LCD 14L corresponding to the left and right eyes, and an eyepiece (R) 15 corresponding to both eyes of the wearer of the HMD 11 is provided. Or through an eyepiece (L) 16.

The objective lens 12R or the objective lens 12L constituting the image pickup section 2R and the image pickup section 2L employs a slightly wide-angle lens in order to secure a wide field of view within a range where there is no discomfort during image observation. A lens with a large depth of field is used to eliminate the need for focusing.

Further, the eyepiece (R) 15 and the eyepiece (L) 16 are connected to the display element LCD 14R or the display element L
Adjust the diopter to the image display surface of CD14L, and
The image of the external environment taken by the D13R or the imaging device CCD13L is displayed on the display device LCD14R and the display device LC.
The image on D14L can be observed as a full-size virtual image.

As described above, a small camera is incorporated as an image pickup unit, an input image from the camera is subjected to electrical processing, and the image is displayed on an image display device disposed in front of both eyes. The configuration in which the user can observe the image captured by the camera as a natural image has been described. In the first and second embodiments, only the method is described from the image system, and the audio circuit is omitted, but the sound of the external environment and the sound of the external signal input are operated in the same operation mode as the image, You may comprise.

FIG. 5 shows this specific example. Speakers 24 are arranged at the positions of both ears of the user, and an external audio signal input circuit 2 for inputting an audio signal accompanying an external video signal.
Microphone 2 that converts external sound into electrical signals
1 is provided. The mixer 23 is connected to the microphone 21
And the audio accompanying the video signal input from the external audio signal input circuit 22 is synthesized and reproduced from the speaker 24.

As described above, in the present embodiment, a sufficient transmission field of view and a sufficient amount of light of the transmission image can be obtained by employing an electric image processing method irrespective of a mechanical or optical method. It is possible to secure. That is, a small CCD camera is incorporated in the HMD as an image pickup unit, an electric image of the input image from the camera is processed, and the image is displayed on an image display device arranged in front of both eyes. It can be observed by the wearer.

According to this, instead of the transmission function by opening and closing the external field of view by a mechanical shutter or an optical shutter, according to the electric image processing method, the switching structure for viewing the external world other than the image display is complicated. Since the switching mechanism does not require a half mirror, attenuation of the amount of transmitted light by the half mirror can be suppressed.

[0042]

As described above, according to the present invention, since the external image captured by the image capturing means is superimposed on the video and displayed, the external image can be viewed in a sufficiently natural state. It is also possible to provide a head-mounted display that can be visually recognized at the same time.

Here, the first effect is that, in the transmission mode for recognizing the external environment in the closed state when the HMD is mounted, an electrical image processing method is employed instead of the mechanical or optical transmission function. By doing so, a wide viewing angle at the time of transmission can be secured. The reason is that, in order to be able to recognize the external environment when the HMD is closed when wearing the HMD, the transmission mode is realized by opening and closing the external field of view with a mechanical shutter or an optical shutter. Therefore, the switching structure for viewing the outside world other than video display must be complicated.
This is because the field of view of the wearer of the MD was narrow.

The second effect is that, in the transmission mode for recognizing the external environment in the closed state when the HMD is mounted, the HM that secures a sufficient amount of light of the image of the external environment during external observation.
D display can be ensured. The reason is that, in order for the conventional one to be able to recognize the external environment in a closed state when the HMD is mounted, the transmission mode is realized by opening and closing the external field of view with a mechanical shutter or an optical shutter. This is because a half mirror is adopted as a switching mechanism for viewing the outside world in addition to the image display, and the images are optically superimposed, so that the amount of transmitted light must be attenuated.

In addition to the above effects, since the small camera is directly mounted on the HMD, it is possible to display an image of the heading direction of the HMD wearer, and to observe the transmission image when observing the transmission image. There is also an effect that a natural feeling of wearing can be given to the user.

According to the second aspect of the invention, a sufficient viewing angle can be obtained. Furthermore, according to the third aspect of the present invention, the external image is made to look more natural because it is made to look at the actual size.

Further, according to the invention of claim 4,
It becomes stereoscopically viewable. Further, if an image combining function of a captured image as a natural space and an electronic image can be easily realized, the captured image can display a stereoscopic image with a sense of depth. Furthermore, according to the fifth aspect of the present invention, the degree of freedom of the superposition is increased, and the optimum display according to the situation becomes possible. Furthermore, according to the invention according to claim 6, it is possible to reproduce the audio along with the video.

[Brief description of the drawings]

FIG. 1 is a block diagram of an HMD showing a first embodiment of the present invention.

FIG. 2 is a detailed configuration diagram of the HMD.

FIG. 3 is a block diagram of an HMD showing a second embodiment of the present invention.

FIG. 4 is a detailed configuration diagram of the HMD.

FIG. 5 is a configuration diagram of an audio circuit according to a modified example of the HMD in the present invention.

FIG. 6 is a configuration diagram of a conventional HMD.

[Explanation of symbols]

 DESCRIPTION OF SYMBOLS 1, 11 Head mounted display (HMD) 2 Image pick-up part 3 Image processing part 4 Signal input part 5 Image synthesizing part 6 Image display part 7 Image generator 8 Video signal 12, 12R, 12L Objective lens 13, 13R, 13L Video element (CCD) 14, 14R, 14L Display element (LCD) 15 Eyepiece (R) 16 Eyepiece (L)

Claims (6)

[Claims] 1. A head-mounted display mounted on a user's head and having image display means at a position visible to an observer, comprising: an image pickup means for picking up an external image; External video signal inputting means, and an image synthesizing means for superimposing an external image captured by the image capturing means and an external video input by the external video signal inputting means and displaying the image on the image display means. A head mounted display comprising: 2. The head mount display according to claim 1, wherein said image pickup means includes a CCD image pickup device and an objective lens unit, and picks up an external image at a natural viewing angle. display. 3. The head-mounted display according to claim 1, wherein the external image is displayed as a real-size virtual image. 4. The head-mounted display according to claim 1, 2 or 3, wherein said image pickup means is arranged in a pair corresponding to a position of both eyes, and said image display means is also provided on both eyes. A head-mounted display which is individually arranged in a pair, and wherein the image synthesizing means causes the image display means to individually display an external image captured by the imaging means. 5. The head-mounted display according to claim 1, wherein the image synthesizing unit receives the external image captured by the image capturing unit and the external image signal input unit. A head-mounted display characterized in that superposition with an external image can be selectively executed as appropriate. 6. The head mounted display according to claim 1, wherein said audio reproducing means arranged at a position of both ears of a user, and an audio signal accompanying an external video signal. External sound signal input means for inputting an audio signal, a microphone for converting external sound into an electric signal, and a sound to be reproduced by the sound reproducing means by synthesizing the sound inputted from the microphone and the sound accompanying the video signal. A head-mounted display comprising a synthesizing unit.