JP3368672B2 - Head mounted display - Google Patents

Description

Description: BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention relates to a head-mounted display used by a viewer wearing it on his or her head. 2. Description of the Related Art An image display unit is provided in close proximity to both eyes of a viewer, and an image display device mounted on the head for use, a so-called head mounted display (H)
Examples of MD) include those disclosed in JP-A-4-61495 and JP-A-5-183943. The devices disclosed in these publications have an eyeglass-type appearance, are configured to be worn on the ears, and are mounted on the head, and a pair of image display units facing the left and right eyes. There is. As disclosed in Nikkei Electronics, 1993, 14, No. 571, there is also a headband worn on the head. As the image display unit, a liquid crystal element, an EL element, or the like is used, and is provided at a position in front of both eyes. If a left-eye image and a right-eye image are respectively displayed on the left and right display units, an image with a three-dimensional effect can be enjoyed. In addition, there is a transmission-type HMD in which a lens for reflecting an image is placed inside a spectacle-shaped screen so that an image displayed on an upper portion of the screen is reflected and projected on the screen for viewing.
This type can be viewed through the screen as well as the outside, simultaneously with the image on the screen. [0004] However, the above background art has the following inconveniences. (1) The non-transmissive type is convenient for enjoying only the image, but since the outside world is blocked, the HMD must be removed once to see the outside world. I can't say. On the other hand, in the case of the transmission type, the above-described inconvenience does not necessarily occur because the scenery of the outside world and the viewing image appear to overlap. However, even if the head is moved, the scenery of the outside world moves, but the viewing image occupies a certain position in the field of view. For this reason, when it is necessary to see both the scenery and the appreciation image necessary for the outside world, it is necessary to move and fix the head at a position where both of them can be seen. (2) Regardless of the non-transmission type or the transmission type, the position of the display image within the visual field of both eyes is fixed. For this reason, even if the head is moved, the appreciation image is always located at a certain place in the visual field. On the other hand, when watching television in a room, turning the head and looking in another direction causes the television image to disappear from view.
As is apparent from a comparison between the HMD and the HMD, the two have greatly different environments for viewing images. In other words, it is considered that the viewing image cannot be avoided in the HMD, so that the degree of pressure and fatigue when viewed for a long time is increased. The present invention focuses on the above points,
It is an object of the present invention to provide an easy-to-use head-mounted display that allows a desired view of the outside world to be viewed as required. Another object is to provide a head-mounted display that allows the user to enjoy an image in an environment similar to that of a conventional television and that is less fatigued even when viewed for a long time. [0007] In order to achieve the above object, the present invention detects the movement of the apparatus main body with a sensor, and, based on the detection result, determines the relative position of the viewing image with respect to the background image. Since a reverse display is given to a proper display position, a display environment similar to that of a stationary television is realized despite being a head-mounted television. According to a main aspect of the present invention, a microphone for obtaining an external sound is provided. Then, the external sound and the sound of the viewing image are mixed by the sound mixing unit such that the sound image position of the viewing image matches the image position in accordance with the movement of the horizontal sensor.
The above and other objects, features and advantages of the present invention will become apparent from the following detailed description and the accompanying drawings. DESCRIPTION OF THE PREFERRED EMBODIMENTS While there are many possible embodiments of the head mounted display of the present invention, a suitable number of embodiments are shown and described in detail. <First Embodiment> FIG. 1 shows a plan view of mechanical components of a first embodiment. As shown in the figure,
The entire body is in the form of goggles, and the main body 1 on the front side of the head
The image display units 12R and 12L are provided at 0 so that the image display units 12R and 12L are located close to both eyes when worn. These image display units 12R, 12R are respectively constituted by liquid crystal displays 14R, 14L and backlights 16R, 16L. The image display unit 12
Eyepieces 18R and 18L are provided on the naked eye side of R and 12L, respectively. On both sides of the main body 10, mounting arms 20R and 20L corresponding to eyeglass vines are respectively extended. Speakers 22R and 22L are provided inside the ends of the mounting arms 20R and 20L, respectively. Further, in the first embodiment, a video camera 24R facing forward (leftward in FIG.
24L are provided, and vertical sensors 26R and 26L are provided inside the left and right ends, respectively. A horizontal sensor 28 is provided at the center of the main body 10. Of these, video cameras 24R, 24R
L is for imaging the visual field range of the right eye and the left eye, respectively. The vertical sensors 26R and 26L are sensors for detecting the relative position of the visual field in the vertical direction, and the horizontal sensor 28 is a sensor for detecting the relative position of the visual field in the horizontal direction. In addition, the vertical sensors 26R, 2
The rotation of the visual field can be detected from the 6L detection output. Next, the electrical components of the first embodiment are as shown in FIG. The circuit block shown in the figure is only one of the left and right circuits.
One similar circuit is provided. The basic operation, that is, the operation when the above-mentioned sensors 26R, 26L, 28 are not taken into account, will be described. The R, G, B image signals of the scenery imaged by the video cameras 24R, 24L are output from the synchronization signal generation circuit 30. Is output based on the synchronizing signal supplied from the synthesizing switch 32. On the other hand, an external input image signal of a movie or the like is subjected to color demodulation by a color demodulation circuit 34 and then an A / D converter
At step 6, it is converted into a digital signal. Then, the data is stored in the memory 38 in synchronization with the input image signal. At this time, by thinning out or interpolating the pixels of the image stored in the memory 38 by the adjustment by the writing control circuit 40, the enlargement / reduction of the image and the change of the aspect ratio can be performed as desired by the viewer. Done. The image signal stored in the memory 38 is read out at the same timing as the synchronization signal of the video camera 24, is converted into an analog signal by the D / A converter 42, and is supplied to the synthesizing switch 32. Synthetic switch 3
In 2, the scenery image supplied from the video camera 24 is combined with a viewing image such as a movie. FIG. 3 shows this state. The video cameras 24R and 24L capture, for example, landscape images PBR and PBL as shown in FIG.
The synthesizing switch 32 performs a synthesizing process in which the viewing image is prioritized over the background image, that is, a region where the viewing image is present in one screen, and a background image is output in the region where the viewing image is not present. Done. Therefore, as shown in FIG. 3B, the appreciation images PMR and PML are combined with the landscape images PBR and PBL, respectively. This is the picture-in-television for televisions and VTRs.
It can be compared to a picture function. The synthesized image signals for the left and right eyes are displayed on the liquid crystal displays 14R and 14L.
Respectively. On the other hand, liquid crystal displays 14R, 14L
The display scan is performed by a scan timing signal supplied from the horizontal / vertical timing circuit 44. The horizontal / vertical timing circuit 44 includes the synchronization signal generation circuit 3
Since the operation is performed based on the synchronization signal supplied from 0, the scanning of the liquid crystal displays 14R and 14L is also performed in synchronization with the image signal output of the video cameras 24R and 24L. In other words, the operations of the circuits from outputting image signals from the video cameras 24R and 24L and reading image signals from the memory 38 to displaying on the liquid crystal displays 14R and 14L are all performed in synchronization. The liquid crystal displays 14R and 14L receive the light of the backlights 16R and 16L driven by the backlight driving device 46 and display the combined images for the left and right eyes shown in FIG. 3B. . These combined images are observed by both eyes of the viewer through the eyepieces 18R and 18L. In the first embodiment, the detection output side of the sensors 26R, 26L, and 28 described above is connected to the read control circuit 48.
The read control circuit 48 controls reading of data from the memory 38. The detection outputs of the vertical sensors 26R and 26L are connected to a rotation angle detection circuit 50, and the output side of the rotation angle detection circuit 50 is connected to an image rotation circuit 52. The image rotation circuit 52 is for rotating and outputting the image supplied from the memory 38. To explain these operations, that is, if the HMD shown in FIG. 1 moves in the horizontal direction, this is detected by the horizontal sensor 28. The read control circuit 48 controls the timing of reading signals from the memory 38 in accordance with the horizontal movement. As a result, the video camera 24
The display timing of the viewing image with respect to the background image sent from R and 24L is shifted, and as a result, the relative display position is shifted. The reading control circuit 48 controls reading of data from the memory 38 so that the moving direction of the viewing image is opposite to the moving direction detected by the horizontal sensor 28. FIG. 4A shows this state. Assuming that the detection movement direction of the horizontal sensor 28 is the arrow FA, the liquid crystal displays 14R, 14R
The viewing images PMR and PML on L are indicated by the reverse arrow FB.
Will move in that direction. Next, if the HMD moves in the vertical direction, this is detected by the vertical sensors 26R and 26L. In the read control circuit 48, the memory 3
8 is controlled. Also in this case, the read control circuit 48 controls reading of data from the memory 38 such that the moving direction of the viewing image is opposite to the moving direction detected by the vertical sensors 26R and 26L. FIG. 4B shows this state.
The direction of detection movement of the vertical sensors 26R and 26L8 is indicated by an arrow F.
C, FD, the liquid crystal display 14R,
The appreciation images PMR and PML on the 14L move in the direction of the opposite arrow FE by an average movement amount thereof. Further, at this time, the vertical sensors 26R, 26R
From the detection result of L, the rotation angle detection circuit 50 shown in FIG.
HMD rotation angle θ is detected. For example, FIG.
As shown in (C), if the detection direction of the vertical sensor 26R is a movement amount Δa indicated by an arrow FF and the detection direction of the vertical sensor 26L is a movement amount Δb by an arrow FG, the distance between the vertical sensors is represented by S, and θ = The rotation angle θ is obtained by performing the calculation of tan ⁻¹ ((Δa + Δb) / S). The image rotation circuit 52 rotates the image corresponding to −θ, which is opposite to the rotation angle θ. As a result, the liquid crystal display 14
As shown in FIG. 4 (C), the viewing image P
MR and PML are rotated and displayed. As described above, the direction in which the viewing image is displayed is fixed when viewed from the viewer, and when the viewer looks in another direction, the viewing image is deviated from the field of view and the background image becomes visible. . The background image at this time is video camera 2
This is a captured image of the outside world by 4R and 24L, and is an image in the direction in which the face of the viewer faces. Therefore, the viewer is
Even during use, if necessary, the user can move the face in that direction to view the image in that direction. In other words, it is as if watching a stationary television. For example, suppose that the viewer gradually moves his face to the left as shown by the arrow FH in FIG. 3C in the display state as shown in FIG. 3B. Then, the viewing images PMR, PM on the liquid crystal displays 14R, 14L.
L moves in the opposite direction indicated by arrow FI in FIG. The same applies to vertical movement and rotation. An initial setting switch 54 is connected to the read control circuit 48 and the rotation angle detection circuit 50.
This makes it possible to perform initial setting of the relative relationship between the desired posture of the viewer when the HMD is worn on the head and the positions of the viewing images PMR and PML in the field of view. Next, the overall operation of the first embodiment will be described. The viewer wears the wearing arms 20R and 20L of the HMD shown in FIG. In this state, the video cameras 24 are displayed on the liquid crystal displays 14R and 14L.
R, 24L, landscape images PBR, PBL
Then, images PMR and PML that the user wants to watch, such as a movie or a television program, input from the outside are displayed in a superimposed manner (see FIG. 3B). When the viewer changes the direction of the face in this state, the change is detected by the sensors 26R, 26L and 28, and the display position of the viewing image changes as shown in FIG. Therefore, the viewer takes a desired posture, such as sitting on a sofa seat, and faces an appropriate direction so that the viewing image is at a desired position in the display field of view. Then, in this state, the initial setting switch 54 is operated. Then, the read control circuit 48 and the rotation angle detection circuit 50 perform an initial setting operation ignoring the sensor input, and the display position of the viewing image on the liquid crystal displays 14R and 14L is fixed even if the field of view is changed. In this state, the viewer releases the initial setting switch 54. Then, the readout control circuit 48 and the rotation angle detection circuit 50 perform the above-described readout control and rotation angle detection operation based on the sensor output at the time of release. That is, based on the sensor output, the display position of the viewing image on the liquid crystal displays 14R and 14L changes as shown in FIG. When the viewer turns his / her face in that direction when necessary, the background image in that direction is displayed on the liquid crystal displays 14R and 14L, and the viewing image moves. You can easily know the state of the outside world. Next, audio will be described. In FIG.
An audio signal processing circuit is shown. In the figure, the output side of a microphone (hereinafter simply referred to as "microphone") 70 for obtaining external sound is connected to the input sides of sound mixing circuits 72R and 72L each constituted by a variable resistor. The microphone 70 is located at an appropriate position, for example, FIG.
Is provided above the horizontal sensor 28 in the HMD shown in FIG. An R sound and a monaural sound are input to the sound mixing circuit 72R. In addition, L sound and monaural sound are input to the sound mixing circuit 72L. The sliding terminals of the sound mixing circuits 72R and 72L move according to the detection result of the horizontal sensor 28. The output sides of the sound mixing circuits 72R and 72L are connected to amplifiers 74R and 74L.
Are connected to the speakers 22R and 22L, respectively. Such an audio signal processing circuit is housed at an appropriate position of the HMD. Next, the operation of the audio processing will be described.
First, as shown in FIG. 7 (A), it is assumed that the viewer is facing the front to view the viewing image. At this time, the horizontal sensor 28 is in a state of not detecting horizontal movement. In the sound mixing circuits 72R and 72L, since the sliding terminal is located at the center, R sound and L sound are respectively output. For this reason, the R and L audio signals are reproduced and output from the speakers 22R and 22L, respectively. That is, the audio output corresponds to the appreciation image. Next, as shown in FIG. 7B, it is assumed that the viewer turns left to see the outside world. Then, the movement is detected by the horizontal sensor 28 and the sound mixing circuit 72
In R and 72L, the sliding terminal moves in the “left” direction.
Therefore, the sound mixing circuit 72R outputs the monaural sound R + L.
Is output, and the sound mixing circuit 72L outputs the external sound of the microphone 70. That is, the monaural sound is heard by the right ear that virtually faces the viewing image PM, and the external sound is heard by the opposite left ear. It is also assumed that the viewer turns right to see the outside world, as shown in FIG. Then, the movement is detected by the horizontal sensor 28 and the sound mixing circuit 72
In R and 72L, the sliding terminal moves in the “right” direction.
Therefore, the sound mixing circuit 72R outputs the external sound of the microphone 70, and the sound mixing circuit 72L outputs the monaural sound R + L. That is, a monaural sound can be heard at the left ear that virtually faces the viewing image PM, and an external sound can be heard at the opposite right ear. In this way, the sound and the external sound are mixed in accordance with the detected movement amount of the horizontal sensor 28, and the position of the viewing image PM and the position of the sound source are made to correspond. As described above, according to the first embodiment, the desired scenery of the outside world can be seen well as required, and the usability is very good. In addition, images can be enjoyed in an environment similar to a conventional stationary television, and there is little fatigue even when watching for a long time. Further, since the size of the viewing image can be changed, it is convenient for observing an external scenery. In addition, since the position of the viewing image matches the position of the sound source, the discomfort of wearing the HMD is reduced, and external information can be obtained by voice. You will be closer to John. Second Embodiment Next, a second embodiment will be described with reference to FIG. The first embodiment described above is an example of a non-transmission type HMD, but the second embodiment is a transmission type HMD.
It is an example of MD. In the case of the transmission type, since the viewer can see the outside world through goggles, there is no need for a video camera for capturing an image of the outside world. Therefore, FIG.
The components related to the video camera inside have been removed,
Only components for controlling the display position of the externally supplied viewing image may be used. Further, since the second embodiment is a simple type, the components of the display position control based on the rotational movement are omitted.
Therefore, only one vertical sensor is required. In FIG. 5, a viewing image signal input from the outside is color-demodulated by a color demodulation circuit 34 and then supplied to the liquid crystal displays 14R and 14L as it is. The viewing image signal is also input to the synchronization signal separation circuit 60, where the horizontal and vertical synchronization signals HD and VD are separated. The horizontal synchronization signal HD is supplied to the horizontal timing circuit 62.
Are supplied to the liquid crystal displays 14R and 14L. The vertical synchronization signal VD is supplied to the vertical timing circuit 6
4, and the vertical scanning timing signal obtained therefrom is similarly supplied to the liquid crystal displays 14R and 14L. On the liquid crystal displays 14R and 14L, an image input from the color demodulation circuit 34 is displayed based on these timing signals. In this case, the detection output of the horizontal sensor 28 is supplied to a horizontal timing circuit 62, which adjusts the horizontal timing so that the viewing image moves in the direction opposite to the direction in which the face moves. . On the other hand, the detection output of the vertical sensor 26R (or 26L) is supplied to a vertical timing circuit 64, which adjusts the vertical timing so that the viewing image moves in the direction opposite to the face moving direction. As described above, in the second embodiment, the display position of the viewing image changes in the opposite horizontal and vertical directions in accordance with the direction of the face. Therefore, the outside world can be seen from the goggles, and the same effect as in the first embodiment can be obtained. Example 2
Then, because of the simple type, the size of the viewing image does not change. <Other Embodiments> The present invention can be variously modified based on the above disclosure, and includes, for example, the following. (1) In the first embodiment, the video cameras are provided for the left and right eyes, but of course, only one video camera may be provided. (2) The initial setting operation performed in the first embodiment may be performed by the horizontal and vertical timing circuits 62 and 64 in the second embodiment. Since the second embodiment is of a transmission type, an appreciation image is displayed over a background image. For this reason, it is convenient if the initial setting is made so that the viewing image is located in a monotone place, for example, a portion such as a white wall, in the scenery of the outside world. (3) Although the liquid crystal display is directly viewed by both eyes in the above embodiment, the liquid crystal display may be viewed indirectly by using a mirror or the like. In addition, various design changes can be made to achieve the same effect, such as detecting the movement of the image taken by the video camera instead of the horizontal and vertical sensors and moving the viewing image in the direction opposite to the movement. It is. As described above, according to the present invention, the following effects can be obtained. (1) Since the movement of the HMD is detected and the movement opposite to the movement is added to the viewing image, an environment similar to a stationary television can be realized, and a desired external scenery can be obtained only by moving the face. Easy to see and easy to use. (2) It is possible to enjoy images in an environment similar to a conventional stationary television, and there is no feeling of oppression and little fatigue even when viewed for a long time.

BRIEF DESCRIPTION OF THE DRAWINGS FIG. 1 is a plan view showing a main part of a mechanical configuration according to a first embodiment of the present invention. FIG. 2 is a block diagram illustrating a main part of an electrical configuration according to the first embodiment. FIG. 3 is a diagram illustrating the operation of the first embodiment. FIG. 4 is a diagram illustrating the operation of the first embodiment. FIG. 5 is a block diagram illustrating a main part of an electrical configuration according to a second embodiment. FIG. 6 is a block diagram illustrating an audio processing system according to the first embodiment. FIG. 7 is a diagram illustrating the operation of the audio processing system according to the first embodiment. [Description of Signs] 10 HMD main body 12R, 12L video display unit (image display means, image projection means) 14R, 14L liquid crystal displays 16R, 16L backlights 18R, 18L eyepieces 20R, 20L mounting arm 22R , 22L ... speakers 24R, L ... video cameras 26R, 26L ... vertical sensors (sensor means) 28 ... horizontal sensors (sensor means) 30 ... synchronization signal generation circuit 32 ... synthesis switch 34 ... color demodulation circuit 36 ... A / D converter 38 memory (position control means) 40 write control circuit 42 D / A converter 44 horizontal / vertical timing circuit 46 backlight driving device 48 read control circuit (position control means) 50 rotation angle detection circuit (Position control means) 52 ... Image rotation circuit (Position control means) 54 ... Initial setting switch 60 ... Sync separation circuit 62 ... Water Timing circuit (position control means) 64 ... Vertical timing circuit (position control means) 70 ... Microphones 72R, 72L ... Sound mixing circuit (sound mixing means) 74R, 74L ... Amplifiers PBR, PBL ... Background images PM, PMR, PML ... Viewing image

Claims (1)

(57) [Claims] [Claim 1] HMD main body mounted on the head; Image display means for displaying an image at a position close to both eyes in the main body; For capturing the field of view of both eyes An image synthesizing unit that superimposes a signal of a viewing image supplied from the outside on a signal of a background image obtained thereby and supplies the signal to the image display unit; a sensor unit for detecting a movement of the display device body A head-mounted display comprising: a position control means for giving a reverse movement to the synthesized position of the viewing image by the image synthesizing means in accordance with the detection result of the sensor means.