JP3898816B2 - Head mounted display with position detection function - Google Patents

Description

[0001]
BACKGROUND OF THE INVENTION
The present invention relates to a head-mounted display (HMD) used for aircraft training training and educational equipment, virtual reality (virtual reality) experience machines, and the like.
[0002]
[Prior art]
The HMD includes a closed-type HMD that provides only a virtual image to the wearer, and a see-through that allows the wearer to see the external scenery and provides the virtual image superimposed on the external scenery image (real image). There is a type HMD. In the latter case, if an attempt is made to superimpose and display a positional relationship between a real image and a virtual image, the orientation of the HMD in the external environment must be accurately detected. Conventionally, a magnetic method or an ultrasonic method has been used as a method for this purpose. In both cases, the magnetic generator or ultrasonic generator is fixed to the HMD, and the position (orientation) of the HMD is detected by the interaction with the magnetic detector or ultrasonic detector fixed outside the HMD. It is. Note that the generator and the detector may be attached in reverse.
[0003]
[Problems to be solved by the invention]
Precision output置検position both in the above conventional method is not always sufficient. For example, in the magnetic method, the detection accuracy may be reduced if a metal object is present in the vicinity.
[0004]
The present invention has been made in order to solve such problems, and the object of the present invention is to detect the position of an object using infrared light , thereby enabling more accurate real images and virtual images. It is to provide an HMD that can perform superimposing display. Thereby, for example, even when a metal object exists in the vicinity, the detection accuracy is prevented from deteriorating.
[0005]
The present invention was made in order to solve the above described problems is the see-through type head mounted display which includes a semi-transparent screen and hot mirror disposed in front of the eyes of the wearer,
d) an external video photographing means for photographing infrared light reflected by a hot mirror arranged in front of the wearer's eyes;
detection means for identifying said object based on temporal patterns or spatial pattern of the infrared light and detects the position of the infrared light from the based-out object body e) photographed external image,
with f) detected position, a projection means for projecting the shadow information related to the identified object on the semi-transparent screen,
It is characterized by having.
[0006]
When the wearer wears the HMD, a translucent screen and a hot mirror are positioned in front of the eyes. The wearer can visually recognize the external image through the translucent screen. On the other hand, the external image capturing means of the HMD captures an external image in front of the wearer's eyes by infrared light reflected by the hot mirror, and sends the image information to the detecting means. The detection means detects the position of infrared light from the object using a known image processing method and identifies the object based on the temporal pattern or spatial pattern of the infrared light . Information of the detected position is transmitted to the projection means, the projection means with the detected position, on a semi-transparent screen, for projecting the shadow of the information about the identified object. By setting the projecting shadow position information about the object properly, outside the shadow image and the information about the object is superimposed plainly on the wearer, training, education, given a greater effect than conventional in virtual reality, etc. Will be able to.
[0007]
In HMD according to the present invention, in order to identify said object based on temporal patterns or spatial patterns of infrared light and detects the position of an object by image processing from the photographed image, accurate information about the object to the wearer And it can be given in an easy-to-understand manner. In addition, position detection can be performed more accurately than conventional detection using magnetism or ultrasonic waves, and superimposed display with little deviation can be performed.
[0008]
【Example】
A see-through type HMD which is an embodiment of the present invention will be described with reference to FIGS. As shown in FIGS. 1 and 2, the HMD 10 of this embodiment is obtained by adding an external video photographing means to a conventional see-through HMD. That is, as with the conventional HMD, the main body 11 that is worn on the head of the wearer, the transflective screen 12 that is placed in front of the wearer's eyes when the main body 11 is worn, and the video is output to the screen 12 In addition, a pair of infrared imaging units 14 are provided in front of the left and right eyes of the wearer as external video photographing means. Each infrared imaging unit 14 includes a hot mirror 15 disposed in front of the wearer's eyes, a camera lens 16 and a CCD camera 17 provided outside the hot mirror 15. The hot mirror 15 is an optical element that transmits visible light and reflects infrared light. The hot mirror 15 reflects only infrared light from the scenery in front of the wearer's eyes toward the wearer's eyes, and is a CCD camera. Send it to 17.
[0009]
FIG. 3 shows a functional configuration of the HMD 10 of this embodiment. The video output unit 13 includes a video generation unit 21 that generates a video to be displayed on the translucent screen 12 and a projection unit 22 that projects the generated video on the screen 12. The video generation unit 21 includes a computer and functions as the video generation unit 21 by executing a program prepared in advance. In the HMD 10 of the present embodiment, the computer has a function as the infrared video processing unit 23 in addition to the function as the video generation unit 21.
[0010]
Next, a method of using the HMD 10 of the present embodiment having the above configuration will be described. First, an infrared light source 26 is attached in advance to an external object 25 to be visually recognized. As the infrared light source 26, an infrared LED (light emitting diode) is simple and easy to use. Note that the number of external objects 25 to be visually recognized may be one or plural. In the case of plural external objects 25, each infrared light source 26 is identified. For example, it is possible to take measures such as changing the temporal pattern of light emission or placing the spatial pattern in a barcode format.
[0011]
Next, the main body 11 of the HMD 10 is mounted on the head of the wearer, and the main body 11 is fixed to the head with an airbag or the like. After the position of the main body 11 with respect to the head is fixed, an appropriate adjustment pattern is projected onto the transflective screen 12 so that the wearer can visually recognize the adjustment pattern at a predetermined position. Make adjustments. If necessary, the focus of the output video is also adjusted.
[0012]
After the above preparations are complete, use will begin. When the wearer faces the external object 25, the infrared light emitted from the infrared light source 26 attached to the external object 25 is reflected by the left and right hot mirrors 15, and the corresponding CCD camera via the camera lens 16. 17 is sent. The CCD camera 17 generates a video signal representing the shot video and sends it to the infrared video processing unit 23. In order to make object recognition in the infrared image processing unit 23 easier, an infrared light filter may be provided in front of the CCD camera 17 to remove unnecessary visible light from the captured image in advance.
[0013]
First, the infrared video processing unit 23 generates a binary video from the video signal with a predetermined infrared wavelength as a threshold so that an image of the infrared light source 26 can be extracted. Based on this binary image, the position of each infrared light source 26 is detected by a known image processing technique, and each infrared light source 26 is specified based on a temporal pattern or a spatial pattern. The infrared image processing unit 23 extracts information (object information) stored in advance corresponding to the identified infrared light source 26 from a memory (not shown), and together with information on the position of the infrared light source 26, an image generation unit Send to 21. The video generation unit 21 converts the object information into a video, and generates a projection video in which the object information video is arranged at or near the infrared light source 26. The projected video is projected on the screen 12 by the projection unit 22.
[0014]
Accordingly, the wearer visually recognizes the image of the external object 25 passing through the screen 12 and the image projected on the screen 12 in a superimposed manner. An example of this superimposed image is shown in FIG. In this example, information about the object 25 (for example, the name, attribute, etc. of the object 25) is displayed near the object 25 to which the infrared light source 26 is attached. FIG. 5 shows an example in which an internal structure of the object 27 or an image 28 of an object included in the object 27 is displayed superimposed on the same location as the object 27. With such superposition display, the wearer can recognize information on the objects 25 and 27 on the spot, and can perform more effective training and education in airplane and automobile operation training, safety education, and the like. In the virtual reality experience and the like, various kinds of virtual reality experiences can be given to the wearer by arranging various objects outside and preparing information related thereto in advance.
[0015]
In the above embodiment, the hot mirror 15 is provided in front of the wearer's eyes so as to capture the image closest to the external image visually recognized by the wearer. However, the CCD camera directly forwards from the left and right sides of the HMD. In this case, the parallax may be corrected at the stage of image processing in the infrared video processing unit 23.
[0016]
Moreover, in the said Example, although it was set as the structure which attaches an infrared light source to an external object, you may make it attach the light emission source of infrared light to HMD. That is, it is possible to identify an object by projecting infrared light onto the front surface of the HMD and photographing the reflected light with a CCD camera to photograph an infrared light image and subjecting it to image processing. This method is particularly effective when the outer shape of the object has characteristics.
[Brief description of the drawings]
FIG. 1 is a plan view of an HMD that is an embodiment of the present invention.
FIG. 2 is a side view of the HMD of the example.
FIG. 3 is a functional block diagram of the HMD of the embodiment.
FIG. 4 is an explanatory diagram illustrating an example of display of an HMD according to an embodiment.
FIG. 5 is an explanatory diagram showing another display example.
[Explanation of symbols]
10. Head mounted display (HMD)
DESCRIPTION OF SYMBOLS 11 ... Main-body part 12 ... Translucent screen 13 ... Video output part 14 ... Infrared imaging part 15 ... Hot mirror 16 ... Camera lens 17 ... CCD camera 21 ... Video generation part 22 ... Projection part 23 ... Infrared video processing part 25, 27 ... External object 26 ... Infrared light source

Claims (3)

In see-through type head mounted display which includes a semi-transparent screen and hot mirror disposed in front of the eyes of the wearer,
a) an external video photographing means for photographing infrared light reflected by a hot mirror disposed in front of the wearer's eyes;
b) detection means for identifying said object based on temporal patterns or spatial pattern of the infrared light and detects the position of the infrared light from the based-out object body captured external image,
with c) the detected position, a projection means for projecting the shadow information related to the identified object on the semi-transparent screen,
A head-mounted display comprising: 2. The head mounted display according to claim 1, wherein the transflective screen, the hot mirror, the external video photographing unit, and the projection unit are provided in each of the left and right eyes of the wearer. further,
d) An infrared light source that projects infrared light on the front surface of the head mounted display;
  2. The infrared light reflected by the hot mirror photographed by the external video photographing means is reflected light from the object with respect to infrared light projected from the infrared light source. The described head mounted display.

Images