JP2004205711A - Display device - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To provide a display device with which an observer easily recognizes a marker in the vicinity while observing far real space. <P>SOLUTION: The display device mounted on an observer's head and presenting an image to the observer is provided with 1st cameras 2 and 3 for photographing the video (real space) of external environment, 2nd cameras 4 and 5 for photographing the marker (virtual image) for alignment, and a display part synthesizing the videos imaged by the 1st and the 2nd cameras and displaying the video. Then, focal distances in imaging by the 1st and the 2nd cameras are different from each other. <P>COPYRIGHT: (C)2004,JPO&NCIPI

Description

[0001]
TECHNICAL FIELD OF THE INVENTION
The present invention relates to a display device having an image pickup optical unit for capturing an external image and forming the image pickup device on an image pickup device, and a display unit such as a liquid crystal display, and a technique using the display device.
[0002]
[Prior art]
2. Description of the Related Art In recent years, research on mixed reality (hereinafter referred to as “MR” (Mixed Reality)) for the purpose of seamless connection between a virtual space and a real space has been actively conducted. Conventionally, this MR aims at coexistence of a virtual reality (hereinafter abbreviated as VR) world and a real space, which can be experienced only in a state separated from the real space, and is attracting attention as a technology for enhancing the VR. . Applications of this MR include medical assistance, which presents a doctor with a perspective of the inside of a patient's body, and work assistance, which superimposes the product assembly procedure on a real product at a factory. Is expected to be a new field qualitatively completely different from VR.
[0003]
A common requirement for these applications is how to remove the "shift" between real and virtual space. This "shift" can be classified into a position shift, a time shift, and a qualitative shift, and among them, many efforts have been made to eliminate the position shift, which is the most basic requirement.
[0004]
In the case of a video-see-through type MR in which a virtual object is superimposed on a video image captured by an imaging camera and displayed on a head-mounted display (HMD), the problem of this alignment is that the HMD is mounted on the HMD. This results in the problem of accurately determining the three-dimensional position of the imaging camera in the real space.
[0005]
In addition, the problem of alignment in the case of an optical-see-through MR using a transflective HMD can be said to be a problem of obtaining a three-dimensional position of a user's viewpoint in the real space.
Conventionally, as a method of measuring such a three-dimensional position, it is common to use a three-dimensional position and orientation sensor such as a magnetic sensor, an ultrasonic sensor, and a gyro, but the accuracy of these is not necessarily sufficient. The error causes a displacement.
In the case of the video see-through method, since the image in the real space is captured using the imaging camera, it is necessary to directly perform registration on the image based on the captured image information without using such a sensor. Is also possible. For example, a color marker or a geometric pattern marker is presented in a space, the marker position is detected from an image, and a three-dimensional position in a real space is calculated from a previously measured marker spatial position. There is a method to do.
[0006]
[Problems to be solved by the invention]
In the method using such a marker, since the displacement can be directly handled, the positioning can be performed with high accuracy, but there is still a problem in reliability. For example, when the marker moves out of the field of view due to the movement of the line of sight, there is a problem that the measurement of the line of sight cannot be performed. In particular, when a virtual object is superimposed on a distant real space, the input image moves greatly even with a slight movement of the line of sight, and the marker placed on the distant object is likely to be out of the field of view. Therefore, the probability that the position information of the object will not be obtained increases. Such a case can be dealt with by increasing the number of markers arranged in the real space and arranging the markers in a space covering the range in which the line of sight moves.
[0007]
However, in such a method, it is necessary to arrange the marker in a wide area far away, so that it is not practical in terms of the installation location of the marker, the size, shape, and pattern of the marker itself. That is, in order to recognize a marker from an input video, a marker having a certain size is required, and the more distant the marker is, the larger the area of the marker needs to be.
Therefore, if the marker is arranged near the viewpoint position, the area where the marker is installed can be narrowed. However, the following problem arises in observing the real space far away through the marker.
(1) The view is obstructed and the sense of reality in the real space is impaired.
(2) When the focus of the imaging camera, which is the input means, is set to a distant place, the focus cannot be focused on a nearby marker and the marker cannot be recognized.
[0008]
The present invention has been made in view of the above conventional example, and has as its object to enable easy recognition of a nearby marker while observing a distant real space.
[0009]
[Means for Solving the Problems]
In order to achieve the above object, a display device of the present invention has the following configuration. That is,
A display device attached to the observer's head and presenting an image to the observer,
First image capturing means for capturing an image of the outside world;
Second imaging means for photographing a marker for alignment,
Display means for combining and displaying the images picked up by the first and second image pickup means,
It is characterized in that the focal lengths of the first and second imaging means are different from each other.
[0010]
BEST MODE FOR CARRYING OUT THE INVENTION
Hereinafter, preferred embodiments of the present invention will be described in detail with reference to the accompanying drawings.
[0011]
[Embodiment 1]
FIG. 1 is a schematic diagram showing a mounted state of a head mounted display (HMD) according to Embodiment 1 of the present invention. In FIG. 1, illustration of a power supply, an input signal source, cables, and the like is omitted.
[0012]
In the figure, reference numeral 1 denotes a housing holding a display optical system of an HMD described later. Reference numerals 2, 3, 4, and 5 are imaging cameras. The display optical system that can be employed in the HMD according to the present embodiment is not particularly limited. For example, a display optical system having a configuration as shown in FIGS. 2, 3, and 4 can be used. The configuration of the head mounted display (HMD) 1 shown in FIG. 1 is the same as the configuration shown in FIG. 22 described later, and the description thereof will be described later. Further, the HMD 2201 and the PC 2202 shown in FIG. 22 may be integrally formed.
[0013]
FIG. 2 is a diagram illustrating an example of the optical system of the HMD according to the present embodiment.
[0014]
An image displayed on a liquid crystal display (LCD) 11 is enlarged and displayed using a lens 12 (shown as a single lens here), and the enlarged image is displayed to a user (observer) as a virtual image. ing.
[0015]
Also, as shown in FIG. 3, an optical see-through type HMD can be configured by using a half mirror or the like for the reflecting mirror 13.
[0016]
FIG. 3 shows a display optical system of the HMD according to the present embodiment having the simplest configuration, and an image displayed on the LCD 11 is enlarged by using a lens 12 (here, schematically shown by one lens). Then, the light is reflected by the reflection mirror 13 so as to be seen by the user.
[0017]
FIG. 4 shows that a non-rotationally symmetric aspherical optical surface (free-form surface) called a free-form surface prism is applied to three surfaces of the prism 14, and has the same operation as the lens 12 of the optical system shown in FIGS. FIG. The display optical system having such a configuration is held in the housing 1, and presents a virtual reality space to a user wearing the HMD.
[0018]
In the HMD according to the present embodiment, imaging cameras 2 and 3 are cameras for inputting an image of a real space presented to an observer through a display optical system, and imaging cameras 4 and 5 are for inputting a marker as an image. Camera.
[0019]
Specifically, in FIG. 5, the imaging cameras 2 and 3 are for inputting an image of an observation object existing in the real space at the point L, and the cameras 2 and 3 are also adjusted so that the focal points are adjusted to the point L. I have. The imaging cameras 4 and 5 are for inputting a marker arranged in the marker area at the M point as an image, and the focus of the cameras 4 and 5 is adjusted so as to be at the M point. FIGS. 6 and 7 show examples of images input by the respective imaging cameras at this time. Here, the angle of view of the marker installation area 50 is set according to the imaging area in the real space.
[0020]
FIG. 6 shows images input by the imaging cameras 2 and 3 focused on the real space at the point L, and FIG. 7 shows examples of images input by the imaging cameras 4 and 5 focused on the marker area, respectively. Is shown.
[0021]
Here, what is presented to the observer is the image shown in FIG. 6, and the marker is displayed with a blurred focus because the focus is set in a distant real space.
[0022]
On the other hand, in the marker position detection, the marker position is recognized based on an image as shown in FIG. This marker recognition method is not particularly limited, and can be realized by using an existing technology.
[0023]
That is, the display device according to Embodiment 1 of the present application includes at least two first imaging devices that image a real space, and at least two second imaging devices that image markers in a space different from the real space. Having. Here, the first and second imaging devices have different focal lengths.
[0024]
As described above, according to the HMD according to the first embodiment, it is possible to realize an HMD that can easily perform position measurement by recognizing a marker.
[0025]
[Embodiment 2]
Normally, in an HMD as described in the present embodiment, when an external image is input, the angle of view of the camera is adjusted so that the size of the real space and the input image is 1: 1. General. In the first embodiment described above, it is assumed that the angle of view of the imaging cameras 2 and 3 (for inputting a real space) and the imaging cameras 4 and 5 (for inputting a marker image) are the same. However, the angle of view of the imaging cameras 4 and 5 (for marker video input) does not need to be the same as that of the imaging cameras 2 and 3. Therefore, by increasing the angle of view of the imaging cameras 4 and 5 so that as many markers as possible are included in the same image, marker detection errors are reduced, and position information can be detected with high accuracy.
[0026]
FIG. 8 shows the principle of the second embodiment, and FIGS. 9 and 10 show the input video in this case.
[0027]
FIG. 8 is a diagram for explaining a relationship between a physical space and a marker installation area according to Embodiment 2 of the present invention.
[0028]
Here, as apparent from a comparison between FIG. 5 and FIG. 8 according to the first embodiment, in FIG. 8, the angle of view 80 of the cameras 4 and 5 for imaging the marker installation area is the same as that in FIG. It can be seen that it is larger than the angle 50.
[0029]
FIG. 9 is a diagram illustrating an example of an image input by the imaging cameras 2 and 3 for real space, and FIG. 10 is a diagram illustrating an example of an image input by the imaging cameras 4 and 5 for the marker.
[0030]
That is, the display device according to Embodiment 2 of the present application includes at least two first imaging devices that image a real space, and at least two second imaging devices that image markers in a space different from the real space. And the angle of view of the second imaging device is larger than the angle of view of the first imaging device.
[0031]
As described above, by increasing the angle of view 80 for inputting a marker image, it is possible to further improve the accuracy of position measurement using the marker.
[0032]
[Embodiment 3]
In the first and second embodiments described above, the marker based on visible light is assumed. That is, it has been assumed that "colors" or "symbols" that can be photographed by a normal imaging camera are used as markers. In the case of a marker using visible light, there is a problem in that the presence of the marker is impaired because the marker is presented to the observer as a part of the image. Therefore, in the third embodiment, the imaging cameras 4 and 5 (for marker image input) in FIG. 1 are infrared cameras, and the markers are infrared light emitting elements such as infrared lamps or infrared LEDs.
[0033]
That is, the display device according to Embodiment 3 of the present application includes at least two first imaging devices that image a real space, and at least two second imaging devices that image markers in a space different from the real space. , The marker is an infrared marker, and the second imaging device is an infrared camera.
[0034]
As described above, since the markers are infrared markers and the imaging cameras 4 and 5 are infrared cameras, the markers are invisible to the observer, so that the viewer can enjoy the real space video without worrying about the presence of the markers. It becomes possible. Further, when recognizing a marker, only the marker of infrared light needs to be recognized, so that there is an effect that the recognition means can be simplified.
[0035]
[Embodiment 4]
In the above-described third embodiment, the imaging cameras 2 and 3 for inputting a real space and the infrared imaging cameras 4 and 5 for inputting a marker image are arranged independently of each other. It can also be realized by a camera.
[0036]
2. Description of the Related Art As an image pickup device of a conventional image pickup camera, a device using a CCD is generally used. In this CCD camera, three CCD elements corresponding to each color of RGB are arranged. Further, there is a device in which four CCD elements corresponding to one RB image and two G images are arranged. Therefore, in the fourth embodiment, as shown in FIG. 11, this imaging camera is configured by providing a CCD 124 corresponding to infrared light in addition to CCDs 121, 122, and 123 corresponding to RGB colors.
[0037]
Thus, the real space can be photographed by the CCDs 121, 122, and 123, and the infrared marker can be detected by the CCD.
[0038]
FIG. 12 is a schematic diagram showing the configuration of the display device according to the fourth embodiment.
[0039]
Each of the imaging cameras 131 and 132 in FIG. 12 has a configuration as shown in FIG. 11, and can obtain images for RGB and infrared light.
[0040]
That is, the display device according to the third embodiment of the present application includes the image pickup device for RGB and the image pickup device for infrared light in the image pickup device, so that the real space and the marker can be photographed by one image pickup device. Can be.
[0041]
In this way, by enabling the input of the real space and the marker image with one camera, the device configuration can be simplified as shown in FIG. 12, and the influence of parallax due to the camera position can be reduced. Becomes possible.
[0042]
[Embodiment 5]
In the fourth embodiment, since there is an offset between the positions of the imaging cameras 131 and 132 and the actual eyeball position of the observer, a parallax shift occurs. Therefore, it is possible to eliminate this parallax shift by modifying the optical system of the HMD.
[0043]
13 and 14 are schematic diagrams of an optical system of an optical sheath HMD having no parallax displacement.
[0044]
FIG. 14 is a block diagram showing a modified configuration of the optical system of the HMD in FIG. 3 described above, and portions common to FIG. 3 are indicated by the same symbols.
[0045]
Here, an image on the LCD 11 is enlarged and displayed using an optical system (shown here with one lens) 12 to display a virtual image to an observer. At this time, by using the reflecting mirror 13 as a half mirror, the observer can simultaneously observe the outside world that can be seen through the half mirror 13 and the virtual space image display of the LCD 11 reflected on the half mirror 13, thereby realizing the virtual reality. A so-called optical see-through HMD for space observation can be formed.
[0046]
On the other hand, facing the LCD 11 with the half mirror 13 interposed therebetween, an imaging camera 27 for photographing the outside world (in FIG. 13, an optical system 16 (represented by one lens for simplification) and an imaging device such as a CCD) 17 is shown). Here, the entrance pupil 16P of the imaging camera 27 is configured to be at a position optically equivalent to the pupil P of the eyeball of the observer. Thereby, the pupil position of the camera 27 for the external world image matches the pupil position of the observer, so that the parallax displacement correction processing for the image of the camera 27 becomes unnecessary.
[0047]
In addition, in the configuration of FIG. 13, in order to prevent not only the outside world but also the image from the LCD 11 facing the image camera 27 from being reflected, the polarization axis of the image light of the LCD 11 is located between the image sensor 17 and the half mirror 13. By providing a polarizing plate (not shown) having a polarizing axis orthogonal to the above, reflection can be prevented. As a method for preventing the display image from being reflected on the LCD 11, various methods other than using a polarizing plate, such as a method using a polarizing beam splitter instead of the half mirror 13 as an optical path separating unit, can be used. Can be adopted arbitrarily.
[0048]
FIG. 14 is based on the HMD optical system of FIG. 4 described above, and has a lens function by applying a non-rotationally symmetric aspherical optical surface (free-form surface) called a free-form surface prism 14 to three surfaces of the prism. Based optical system.
[0049]
Image light from the LCD 11 is incident on a free-form surface prism 14 having a lens function, and is reflected by a surface 14a, which is an optical path separating means provided with a half mirror, to display a virtual image to an observer. The free-form surface prism 14 is joined with a correction free-form surface prism 18 for canceling the optical power, and an image of the outside world can be observed through the joined prisms 14 and 18.
[0050]
On the other hand, below the prism 18 where the external image reflected by the surface 14a is observed, an imaging camera 28 for capturing an external image (an optical system shown by a lens 19 as in FIG. Are shown). In this configuration, the correction free-form surface prism 18 also forms a part of an optical system (imaging optical system) for photographing an external image.
[0051]
Further, similarly to the configuration shown in FIG. 13, the entrance pupil 19P of the imaging optical system 28 is configured to be at a position optically equivalent to the pupil P of the eyeball of the observer. This eliminates the need to perform parallax displacement correction processing on the image obtained by the imaging optical system 28.
[0052]
Also in the configuration according to the fifth embodiment, it is possible to prevent the image light from the LCD 11 from being incident on the image pickup device 20 such as a CCD, and the polarization of the image light of the LCD 11 between the image pickup device 20 and the prism 18. A polarizing plate (not shown) having a polarization axis orthogonal to the axis can be provided.
[0053]
That is, the display device according to the fifth embodiment of the present invention is provided with a real space input (external image camera 27) near the viewpoint of the observer or at a position where the viewpoint position and the photographing position are substantially equivalent optically. Two types of input elements for marker image input (virtual space) (LCD 11, lens 12, etc.) can be arranged.
[0054]
As described above, according to the present embodiment, in the HMD mounted on the observer's head, a real space input position is set near the observer's viewpoint or at a position where the viewpoint position and the photographing position are substantially equivalent optically. By arranging two types of input elements for inputting a marker image and a marker image, it is possible to realize an HMD capable of easily performing alignment.
[0055]
Embodiment 6
In the sixth embodiment, an observer equipped with a translucent (see-through) head-mounted display (HMD) equipped with an imaging camera is displayed outside the enclosure at an observation place surrounded by a marker-equipped enclosure. The case of observing the world will be explained.
[0056]
FIG. 21 is a diagram illustrating a flow of the display system according to Embodiment 6 of the present invention.
[0057]
The HMD 2101 includes an image display device 2103 that displays an input image and an imaging camera 2104. The HMD 2101 is connected to the PC 2102. With this configuration, an image (real space image) captured by the imaging camera 2104 is captured by the capture device 2105 of the PC 2102. The image captured by the capture device 2105 is sent to a marker recognition unit 2106 and an image synthesis unit 2107. The marker recognizing unit 2106 detects the position of the marker, and calculates the position and orientation of the HMD 2101 from the detected position of the marker. The CG creation unit 2108 creates an object in the virtual space, and sends the object image (virtual space image) to the image synthesis unit 2107. The image combining unit 2107 combines the image of the real space captured by the capture device 2105 with the virtual image created by the CG creating unit 8, and creates an MR space in which the real space and the virtual space are merged. . The image combined and created by the image combining unit 2107 in this manner is sent to the image display device 2103 of the HMD 2101 and displayed.
[0058]
FIG. 15 is a diagram showing an example of a marker-enclosed box according to the sixth embodiment.
[0059]
In this enclosure, a marker 1500 is arranged on a transparent cylinder cut in half vertically. An image captured by the imaging camera 2104 mounted on the HMD 2101 is captured by the computer 2102. The marker attached to the enclosure only needs to be in the observation direction of the observer, and FIG. 15 shows a case where only the front is observed.
[0060]
FIG. 16 is a front view of the enclosure of FIG. 15 arranged in front of the observer 1600.
[0061]
The observer 1600 will see a virtual object several meters to several hundred meters away from an enclosure several tens of centimeters to several meters away. Since the marker 1500 is small and has a distance between the object and the enclosure, even if the marker 1500 is located in front of the object, the observer can observe without much concern.
[0062]
This enclosure has another function. It is to limit the observer's observation position. This function is also effective in an MR system in which the observation position is limited.
[0063]
FIG. 17 is a side view of FIG.
[0064]
The observer 1600 can see the object 1700 beyond the enclosure as shown in this figure.
[0065]
When looking around the entire circumference, the marker 1801 is arranged around the entire circumference of the cylinder as shown in FIG. When the observation area extends to the upper part of the observer, a dome-shaped enclosure as shown in FIG. 19 is used.
[0066]
FIG. 20 is a view showing an example of another enclosure, in which a marker is arranged on a thin wire.
[0067]
That is, in the display system according to Embodiment 6 of the present application, a plurality of markers are arranged on a transparent member, and the observer looks at the real space through the transparent member so that the marker for alignment can be obtained. There is an effect that the arrangement can be facilitated.
[0068]
Embodiment 7
FIG. 22 shows a system in which a marker camera 2211 and a capture device 2212 for capturing an image from the camera 2211 are added to the system according to Embodiment 7 of the present invention.
[0069]
The marker camera 2211 is incorporated in the HMD 2201 and operates as a marker read-only. The capture device 2212 is incorporated in the PC 2202 and captures an image captured by the marker camera 2211. Since the image of the marker is sent from the capture device 2212 to the marker recognition unit 2206, the image is not sent from the capture device 2212 to the marker recognition unit 2206.
[0070]
By separately providing the imaging camera 2204 for photographing the observation space and the marker camera 221 for photographing the marker in this manner, the imaging camera 2204 for photographing the observation space is focused at a long distance, and the marker camera for photographing the marker is used. The camera 2211 will focus on a short distance. As a result, the marker is not clearly seen in the video of the observation space, and the observer does not have to care much about the marker in front.
[0071]
That is, according to the display system of Embodiment 7 of the present invention, a marker used for positioning is arranged in an enclosure set up around the observer, and an imaging camera that captures real space and a marker that captures virtual space are provided. By separately photographing with the camera, positioning with the marker and photographing of the real space can be easily performed.
[0072]
As described above, according to the seventh embodiment, the marker used for the alignment can be arranged in the enclosure set up around the observer, and can be installed even in an event venue or an outdoor where it is difficult to install the marker. It will be easier.
[0073]
[Other embodiments]
The present invention can be applied to a system including a plurality of devices (for example, a host computer, an interface device, a reader, a printer, etc.), but also to a device including one device (for example, a copying machine, a facsimile machine, etc.). May be applied.
[0074]
Further, an object of the present invention is to provide a storage medium storing a program code of software for realizing the functions of the above-described embodiments to a system or an apparatus, and a computer (or CPU or MPU) of the system or apparatus to store the storage medium. Needless to say, this can also be achieved by reading out and executing the program code stored in the.
[0075]
In this case, the program code itself read from the storage medium realizes the function of the above-described embodiment, and the storage medium storing the program code constitutes the present invention.
[0076]
As a storage medium for supplying the program code, for example, a floppy disk, hard disk, optical disk, magneto-optical disk, CD-ROM, CD-R, magnetic tape, nonvolatile memory card, ROM, etc. can be used. When the computer executes the readout program code, not only the functions of the above-described embodiments are realized, but also an OS (Operating System) running on the computer based on the instruction of the program code. This includes a case where a part of the actual processing is performed and the function of the above-described embodiment is realized by the processing.
[0077]
Further, after the program code read from the storage medium is written into a memory provided in a function expansion board inserted into the computer or a function expansion unit connected to the computer, the function expansion is performed based on the instruction of the program code. This includes the case where the CPU or the like provided in the board or the function expansion unit performs part or all of the actual processing, and the processing realizes the functions of the above-described embodiments.
[0078]
The embodiment described above can be realized by the following embodiments.
[0079]
[Embodiment 1] A display device that is mounted on a head of an observer and presents an image to the observer,
First image capturing means for capturing an image of the outside world;
Second imaging means for photographing a marker for alignment,
Display means for combining and displaying the images picked up by the first and second image pickup means,
A display device wherein the focal lengths of the first and second imaging means are different from each other.
[0080]
Second Embodiment The display device according to the first embodiment, wherein an angle of view of the second imaging unit is larger than an angle of view of the first imaging unit.
[0081]
[Embodiment 3] The display device according to Embodiment 1 or 2, wherein the second imaging means captures infrared light.
[0082]
[Embodiment 4] The apparatus according to any one of Embodiments 1 to 3, wherein the first imaging unit performs imaging with visible light, and the second imaging unit is capable of imaging with an invisible wavelength. Display device.
[0083]
[Embodiment 5] Each of the first and second imaging means has at least two cameras, and each camera is located at a position optically substantially equivalent to the viewpoint positions of the right and left eyes of the observer. The display device according to embodiment 4, wherein the display device is arranged.
[0084]
[Embodiment 6] A plurality of the markers are arranged on a transparent member arranged around the observer, and the observer views a virtual image through the transparent member. The display device according to any one of the above.
[0085]
[Embodiment 7] An imaging method in a display device mounted on a head of an observer and presenting an image to the observer,
A display step of combining and displaying a first camera that captures an image of the outside world and a video captured by a second camera that captures a marker for alignment, the first and second cameras Have different focal lengths.
[0086]
[Eighth Embodiment] The display method according to the seventh embodiment, wherein the angle of view taken by the second camera is larger than the angle of view of the first camera.
[0087]
Embodiment 9 The display method according to embodiment 7 or 8, wherein the second camera captures infrared light.
[0088]
[Embodiment 10] The display method according to any one of Embodiments 7 to 9, wherein the first camera captures an image using visible light, and the second camera captures an image using an invisible wavelength.
[0089]
[Embodiment 11] Each of the first and second cameras has at least two cameras, and each camera is located at a position optically substantially equivalent to the viewpoint position of the right eye and the left eye of the observer. The display method according to embodiment 10, wherein the display method is arranged.
[0090]
[Embodiment 12] The marker according to any one of Embodiments 7 to 11, wherein a plurality of the markers are arranged on a transparent member arranged around the observer, and the observer views a virtual image through the transparent member. Display method described in any of them.
[0091]
【The invention's effect】
As described above, according to the present invention, there is an effect that a nearby marker can be easily recognized while observing a distant real space.
[Brief description of the drawings]
FIG. 1 is an external perspective view of an HMD according to Embodiment 1 of the present invention.
FIG. 2 is a side view showing a schematic configuration of an optical system that can be employed in the HMD according to the first embodiment.
FIG. 3 is a side view showing a schematic configuration of another optical system that can be employed in the HMD according to the first embodiment.
FIG. 4 is a side view showing a schematic configuration of still another optical system that can be employed in the HMD according to the first embodiment.
FIG. 5 is a diagram for explaining a relationship between a real space video and a marker video according to the first embodiment.
FIG. 6 is a diagram illustrating an example of a video in a real space according to the first embodiment.
FIG. 7 is a diagram illustrating an example of a marker image according to the first embodiment.
FIG. 8 is a diagram illustrating a relationship between an image of a real space and an image of a marker according to Embodiment 2 of the present invention.
FIG. 9 is a diagram illustrating an example of an image in a real space according to the second embodiment.
FIG. 10 is a diagram illustrating an example of a marker image according to the second embodiment.
FIG. 11 is a diagram illustrating a configuration example of an imaging device according to a fourth embodiment of the present invention.
FIG. 12 is a schematic external view of an HMD according to a fourth embodiment.
FIG. 13 is a side view showing a schematic configuration of an optical system that can be employed in the HMD according to Embodiment 5 of the present invention.
FIG. 14 is a side view showing a schematic configuration of another optical system that can be employed in the HMD according to the fifth embodiment.
FIG. 15 is a diagram showing an example of a transparent enclosure provided with a semi-cylindrical marker according to Embodiment 6 of the present invention.
FIG. 16 is a diagram showing a state where an observer stands in the enclosure of FIG. 15;
FIG. 17 is a diagram showing a virtual object added to FIG. 16 and a diagram viewed from the side.
FIG. 18 is a diagram illustrating an example of a transparent enclosure including a cylindrical marker according to the sixth embodiment.
FIG. 19 is a diagram showing an example of an enclosure formed by a thin wire provided with a dome-shaped marker according to the sixth embodiment.
FIG. 20 is a diagram illustrating an example of an enclosure formed of a thin wire having a semi-cylindrical marker according to the sixth embodiment.
FIG. 21 is a block diagram showing a configuration of an MR system according to a sixth embodiment.
FIG. 22 is a block diagram showing a configuration of an MR system according to Embodiment 7 of the present invention.

Claims (1)

A display device attached to the observer's head and presenting an image to the observer,
First image capturing means for capturing an image of the outside world;
Second imaging means for photographing a marker for alignment,
Display means for combining and displaying the images picked up by the first and second image pickup means,
A display device wherein the focal lengths of the first and second imaging means are different from each other.

Images