KR101868405B1 - Augmented reality/virual reality convertible display device - Google Patents

Abstract

The present invention relates to an augmented reality (AR)/virtual reality (VR) convertible display device, comprising: a fixing unit formed in a shape to be mounted on a head; an integrated image implementing unit fixed to the fixing unit, and including a display element, first and second spatial light modulators, a lens array, and a beam splitter; and an external view opening/closing unit coupled to the fixing unit, and shielding or penetrating external view. According to the present invention, a default image reproduced in the display element and the spatial light modulator is implemented in an integrated image type by using the lens array. Light inputted from the external view is transmitted when a user wants to view AR, and is shielded when a user wants to view VR, thereby allowing the user to selectively switch and view an AR or VR mode in one device.

Description

TECHNICAL FIELD [0001] The present invention relates to augmented reality and virtual reality display device,

The present invention relates to a display device capable of switching between an augmented reality mode and a virtual reality mode. More specifically, the present invention implements a three-dimensional (3D) stereoscopic image using an integrated image technology, And more particularly, to an apparatus for viewing a display of a desired one of augmented reality and virtual reality by adjusting opening and closing.

Along with the rapid development of information and communication technology, there is an increasing interest in technologies for realizing a new space similar to the real world. Among them, augmented reality and virtual reality are typical forms of new space creation. Augmented Reality is realized by combining virtual images to the real world, and can provide users with a new type of visual effect by simultaneously providing the normal real world and the virtual image information that is not existed at the same time. For example, in augmented reality, concrete information about objects in the real world can be provided together as a virtual image, and a virtual object can be placed in the real world. Recently, it has been utilized variously in automobile technology field using HUD (head up display) device and entertainment field using mobile device such as smart phone.

Virtual Reality blocks the user's view of the real world and implements a new virtual space. The virtual reality realizes a space that allows the user to experience a variety of sensory phenomena, including the visual sense, in a virtual situation, so that the virtual reality can provide the user with a feel as if it is a real space. Virtual reality can provide a virtual space beyond visual and physical constraints.

In a conventional HMD (Head Mounted Display) device, a device for realizing an augmented reality implements an image in a user's front view through a method of refracting an optical path using a semi-transparent mirror. On the other hand, a device for realizing a virtual reality uses stereoscopic display technology, which is one of the three-dimensional stereoscopic image display technologies, to display two 2 We implemented virtual reality by providing 2D images. Therefore, there is a difficulty in realizing an augmented reality through an integrated optical system in a single device due to a difference in the manner of implementing the augmented reality and the virtual reality.

Conventionally, augmented reality and virtual reality have been applied to different fields according to their inherent characteristics, but both technologies have been applied in some fields such as simulation, education, and games, because they all share a virtual image. However, as described above, it is difficult to apply the two techniques to one device due to the difference between the methods of implementing the augmented reality and the virtual reality, and since it is implemented using a separate device, It was difficult to expect the effect.

 Wetzstein, Gordon, Douglas Lanman, Matthew Hirsch, and Ramesh Raskar. "Tensor Displays." ACM Transactions on Graphics 31 (4) pp. 1-11 (2012)

Accordingly, it is an object of the present invention to provide a display device which can be switched so that a user can view a display of a desired one of augmented reality and virtual reality by adjusting the opening and closing of an external view.

Further, it is an object of the present invention to provide an apparatus which can be easily worn on the face by the user by using a small optical system in the display device and minimizing the optical path, and can be carried easily.

According to an embodiment, a display device capable of switching between augmented reality and virtual reality includes a fixed portion configured to be worn on a head, a display element coupled to the fixed portion and reproducing a first basic image, A first spatial light modulator for reproducing a basic image, a second spatial light modulator for reproducing a third fundamental image, a lens array for realizing the basic images as a three-dimensional stereoscopic image, and a beam splitter And an external visual field opening / closing part coupled to the fixing part and for blocking or transmitting an external visual field.

In one embodiment, the outer viewing and closing section may include a polarization control device having a first polarizing film and a second polarizing film attached on both sides thereof.

In one embodiment, the display device further includes a data board connected to the display device, the first spatial light modulator, and the second spatial light modulator to transmit image information, and a controller connected to the polarization controller, And a driving board for driving the motor.

In one embodiment, the beam splitter is disposed at a predetermined angle on the optical path between the external view and the binocular, and may have a reflectance of 50% and a transmittance of 50%.

In one embodiment, the display device may further comprise a secondary fixing part formed to catch on the user's head or both ears.

A method for switching between augmented reality and virtual reality according to an exemplary embodiment includes playing a basic image on a display device configured to be worn on a head of a user, Outputting the integrated image, and selectively blocking the external view of the user wearing the display device for either the augmented reality mode or the virtual reality mode.

In one embodiment, selectively blocking the outer field of view may include transmitting or blocking light incident from the exterior to the display device.

In one embodiment, the step of transmitting or blocking light incident on the display device may be performed through polarization control.

In one embodiment, the step of outputting the integrated image may include reflecting the integrated image using a beam splitter disposed between the outer and the outer sides of a user wearing the display device.

According to one aspect of the present invention, a display device capable of switching between augmented reality and virtual reality realizes a three-dimensional stereoscopic image using a display device, a spatial light modulator, and a lens array, When a user wants to view a virtual reality, he or she intercepts light from an external view, thereby selectively switching an augmented reality mode or a virtual reality mode from one device . Also, according to an embodiment of the present invention, it is possible to automatically open / close an external view according to the application of a voltage using a polarization controller.

Furthermore, the display device uses a small optical system and minimizes the optical path, so that the user can easily wear his / her face and has an advantage of being portable.

1 is a side view of a display device capable of switching between augmented reality and virtual reality according to an embodiment of the present invention.
FIG. 2 is a block diagram illustrating a display device and an integrated image implementing unit according to an exemplary embodiment of the present invention. Referring to FIG.
3 is a view illustrating the inside and outside view opening / closing portions of a display device according to one embodiment.
4A is a view showing that a polarization controller according to an embodiment transmits external incident light.
4B is a view illustrating that the polarization controller according to an embodiment blocks external incident light.
5A is a view illustrating viewing in augmented reality mode of a display apparatus according to an exemplary embodiment.
5B is a view illustrating viewing in a virtual reality mode of a display apparatus according to an embodiment.
6 is a flowchart illustrating a method of switching between augmented reality and virtual reality according to an exemplary embodiment.

Hereinafter, preferred embodiments of the present invention will be described in detail with reference to the accompanying drawings. Prior to this, terms and words used in the present specification and claims should not be construed as limited to ordinary or dictionary terms, and the inventor should appropriately interpret the concepts of the terms appropriately The present invention should be construed in accordance with the meaning and concept consistent with the technical idea of the present invention.

Therefore, the embodiments described in this specification and the configurations shown in the drawings are merely the most preferred embodiments of the present invention and do not represent all the technical ideas of the present invention. Therefore, It is to be understood that equivalents and modifications are possible.

1 is a view illustrating a display device capable of switching between an augmented reality and a virtual reality according to an embodiment of the present invention. The display apparatus 1 includes a fixing unit 10 configured to be worn on the head, two-sided fixing units 21 and 22 configured to be able to view the inside of the apparatus corresponding to both eyes of the user, An integrated image implementing unit 40 for selectively interrupting an external field of view, and an external field of view switching unit 50 for selectively interrupting an external field of view.

The fixing portion 10 is configured in the form of a housing including a first opening and a second opening formed at positions corresponding to both eyes of the user and a third opening formed at a position opposed to the first opening and the second opening . The fixation portion 10 can be worn on the user's head in such a way that the first opening and the second opening correspond to both sides of the user and the third opening faces the front sight of the user.

In one embodiment, the display device 1 may further comprise a secondary fixing part formed to catch on the user's head or both ears. The auxiliary securing part may be in the form of a strap capable of adjusting the length or a resilient rubber band.

Referring to FIG. 2, the binocular fixing parts 21 and 22 are coupled to the first opening and the second opening of the fixing part 10, respectively, and fix the two sides of the fixing part 10 so as not to move. The user can view the three-dimensional stereoscopic image implemented in the integrated image implementing unit 40 through the two-dimensional fixing units 21 and 22. In one embodiment, the two-sided fixing portions 21 and 22 may each include a lens portion, and the lens portion may enlarge the three-dimensional stereoscopic image so that the image is formed on the user's eyes.

The integrated image implementing section 40 is a component for displaying a three-dimensional stereoscopic image using integrated image technology. In an embodiment of the present invention, an integral imaging technique is used to implement a basic image as a three-dimensional image. After acquiring an elemental image by photographing an object to be viewed using a lens array, Dimensional stereoscopic image by integrating the light of the basic image into a lens array.

2, the integrated image implementing unit 40 includes a display device 41 for reproducing the first basic image, a second basic image 42 for reproducing the first basic image, A second spatial light modulator 42b for reproducing a third fundamental image, a lens array 43, and a beam splitter 44. The first spatial light modulator 42a reproduces the third fundamental image,

2, the display device 41, the first spatial light modulator 42a, the second spatial light modulator 42b, and the lens array 43 are mounted on the housing side surface of the fixing portion 10, And the beam splitter 44 can be disposed at a predetermined angle on the optical path between the external view and the binocular.

The display device 41 is used to reproduce a first basic image including at least a part of a basic image (augmented reality or virtual reality-related content image) to be viewed. The display device includes a backlight unit (BLU). In an embodiment, the display device 41 may be a liquid crystal display (LCD), a plasma display panel (PDP), or a projector display. In addition, a light emitting diode (LED), an organic light emitting diode (OLED), a light emitting polymer (LEP), an electroluminescence element (EL element) A field emission display (FED), a polymer light emitting display (PLED), or the like may be applied.

The first spatial light modulator 42a and the second spatial light modulator 42b are used to reproduce the second fundamental image and the third fundamental image, respectively. The second and third basic images include at least a part of basic images (augmented reality or virtual reality-related content images) to be viewed. Unlike the display device 41, the spatial light modulators 42a and 42b do not include the backlight unit BLU.

The display device 41 and the spatial light modulators 42a and 42b reproduce different basic images, respectively, so that a richer depth can be expressed in the three-dimensional stereoscopic image. The range of the depth region in which the three-dimensional stereoscopic image can be expressed in the integrated imaging technology can be expressed by a display unit (for example, a display device 41, first and second spatial light modulators 42a, 42b).

Therefore, in the embodiment, since the distances between the display device 41, the first spatial light modulator 42a, and the second spatial light modulator 42b and the lens array 43 are different, Area can be expressed. For example, the display device 41 may reproduce a first fundamental image including a background of a farthest distance, such as a cloud, and the first spatial light modulator 42a may reproduce a second basic image including a background of a relatively short distance, And the second spatial light modulator 42b can reproduce a third fundamental image including an object located at a closest distance from a car, a person, or the like.

In one embodiment, the integrated image implementing unit 40 may further include a plurality of spatial light modulators capable of further reproducing a plurality of basic images, such as a third spatial light modulator and a fourth spatial light modulator.

In one embodiment, the display device 1 further comprises a data board for transmitting image information in connection with the display device 41, the first spatial light modulator 42a and the second spatial light modulator 42b . The data board transmits the first basic image information to the first display device 41 and transmits the second basic image information to the first spatial light modulator 42a and the third basic image information to the third spatial light modulator 42b, And basic video information, respectively. The driving board may be attached to the outer surface of the fixing portion 10 or may be attached to the inside of the housing.

The lens array 43 is an optical element in which a plurality of lenses of the same type are arranged in a two-dimensional form. The shape of each lens may be rectangular or hexagonal, but is not limited thereto. As described above, the basic image reproduced through the display device 41 and the first and second spatial light modulators 42a and 42b can be realized in the form of a three-dimensional stereoscopic image through the lens array 43. [

The beam splitter 44 reflects / transmits an integrated image passed through the lens array 43, thereby realizing a three-dimensional stereoscopic image in front of the user's field of view. In one embodiment, the beam splitter 44 has a predetermined angle on the optical path between the external view and the binocular (an angle at which the three-dimensional image is arranged in the optical path between the external view and the user's bin, for example, An angle of 45 degrees or 135 degrees with the array). In one embodiment, the beam splitter 44 may be a mirror with a reflectance of 50% and a transmittance of 50%.

The beam splitter 44 has a constant reflectance so that the user can view the integrated image reproduced from the side and has a constant transmittance so that the user can simultaneously view the external foreground through the external visual field switching unit 50. Accordingly, the user can simultaneously view the external view and the three-dimensional stereoscopic image in the augmented reality mode.

The external view opening / closing unit 50 is a component for selectively blocking the external view of the user wearing the display device for either the augmented reality mode or the virtual reality mode. Referring to FIG. 1, the external view opening / closing portion 50 is coupled to the third opening of the fixing portion 10. The user can simultaneously view the external foreground and the three-dimensional stereoscopic image by allowing the external visual field opening / closing unit 50 to transmit the light incident into the display device 1 through the third opening for viewing in the augmented reality mode. On the contrary, the external visual field opening / closing unit 50 may block the light for viewing in the virtual reality mode, thereby viewing only the three-dimensional stereoscopic image.

3, the outer viewing angle switching unit 50 may include a first polarizing film 52a and a second polarizing film 52b on both sides of the polarization adjusting device 51, And the polarizing film 52b may be respectively attached. In an embodiment, the polarization controller 51 may be a system that rotates the polarization direction of incident light by 90 degrees when no voltage is applied and does not change the polarization direction of incident light when a voltage is applied, May be an element having a configuration that is oriented. The two polarizing films 52a and 52b may be attached to both sides of the polarization adjusting device 51 and the polarization directions thereof may be perpendicular to each other.

In this case, the polarization controller 51 can automatically determine whether the incident light is transmitted / blocked according to the application of the voltage. 4A, when no voltage is applied to the polarization adjusting device 51, the polarization direction of the incident light is determined by the polarizing film 52a and the polarization direction is rotated by the polarization adjusting device 51 And then can pass through the polarizing film 52b. 4B, when a voltage is applied to the polarization controller 51, the polarization direction of the incident light is determined by the polarizing film 52a, and after passing through the polarization controller 51, the polarizing film 52b ). ≪ / RTI >

In one embodiment, the external field of view switch 50 may be in the form of a lid that the user can open and close, in which case the user may manually open and close the lid to determine whether to block the external field of view.

In one embodiment, the display device 1 may further include a driving board connected to the polarization adjusting device 51 to transmit a driving signal. The driving board may be attached to the outer surface of the fixing portion 10 or may be attached to the inside of the housing.

5A is a view illustrating viewing in an augmented reality mode of a display apparatus according to an embodiment. Referring to FIG. 5A, the display device, the first spatial light modulator, and the second spatial light modulator have first to third bases, respectively, each of which includes different parts of a basic image (for example, augmented reality related content image) Play each video independently. Then, the first through third basic images are integrated while being passed through the lens array to be realized as a three-dimensional stereoscopic image, and the stereoscopic image is displayed in front of the user's view while being reflected / transmitted by the beam splitter. At this time, since the external visual field opening / closing part is open, the stereoscopic image and the real world view can be seen simultaneously in the user's field of view.

5B is a view illustrating viewing in a virtual reality mode of a display apparatus according to an exemplary embodiment. 5A, a basic image (for example, a virtual reality-related content image) to be viewed is integrated through a lens array and is implemented as a three-dimensional stereoscopic image. At this time, since the external visual field opening / closing unit is closed, Only the stereoscopic image is viewed.

Conventional head mounted display devices are not suitable for realizing an augmented reality because a display providing binocular disparity is generally disposed in front and can not be viewed together with an external view. On the other hand, the display device for realizing an augmented reality is not suitable for realizing a virtual reality because it is generally in the form of a goggle with an open outside view. The display device according to the embodiment of the present invention can easily view the external foreground by adjusting the external view opening and closing so that the user does not need any additional equipment for viewing the external view such as the front camera, Dimensional stereoscopic image, it is possible to view the augmented reality contents image together with the actual view, not the external scene captured by the camera. In addition, the apparatus according to the above-described embodiment uses a small optical system and minimizes the optical path, so that the user can easily wear it on the face and is portable easily.

6 is a flowchart illustrating a method of switching between augmented reality and virtual reality according to an exemplary embodiment.

First, a basic image is reproduced on a display device configured to be worn on a user's head (S100). In one embodiment, the step S100 may be performed by a display device. As described above, the elemental is an image obtained by acquiring an object to be viewed through a lens array in order to implement a three-dimensional stereoscopic image using an integral imaging technique.

Then, the display device modulates the intensity and phase of light of the basic image to output an integrated image, which is a three-dimensional stereoscopic image, at step S200. In one embodiment, the intensity and phase of the light in the base image can be modulated by the first spatial light modulator and the second spatial light modulator arranged in parallel, and the modulated base image is integrated into the lens array, The image can be output.

The display device, the first spatial light modulator, and the second spatial light modulator may respectively reproduce first through third basic images including different portions of the basic image to represent different depth regions of the realized three-dimensional image .

In one embodiment, outputting the integrated image (S200) may include reflecting the integrated image using a beam splitter disposed between the outer and the outer sides of the user wearing the display device. As described above, the beam splitter is used to realize a stereoscopic image in front of the user's view by reflecting / transmitting the integrated image passed through the lens array. In one embodiment, the beam splitter may be disposed at a predetermined angle (e.g., an angle of 45 degrees or 135 degrees with the lens array) on the optical path between the outer field of view and the binocular, and has a reflectance of 50%, a transmittance of 50 % Can be a mirror.

Then, in step S300, an external view of a user who wears the display device is selectively blocked to implement either the augmented reality mode or the virtual reality mode. When the user desires to view the augmented reality mode, he or she can open an external field of view and simultaneously view an external foreground and a three-dimensional stereoscopic image (for example, augmented reality-related content image) , It is possible to view only a three-dimensional stereoscopic image (for example, a virtual reality-related content image) by blocking the external view.

In one embodiment, the step of selectively intercepting the external field of view (S300) may include transmitting or blocking light incident from the outside to the display device. In one embodiment, the external field of view coupled to the front of the display device may be manually opened and closed to determine whether the incident light is blocked.

In another embodiment, the step of transmitting or blocking light incident on the display device may be performed through polarization control. An external field-of-view opening / closing unit including a polarization controller for controlling polarization can be coupled to the front of the display device, and a first polarizing film and a second polarizing film can be attached to both sides of the polarization controller. In this case, as described above, it is possible to automatically determine whether the incident light is transmitted / blocked through the polarization control according to the application of the voltage, and detailed description has been given above with reference to FIGS. 4A and 4B.

While the invention has been shown and described with reference to certain embodiments thereof, it will be understood by those skilled in the art that various changes and modifications may be made therein without departing from the spirit and scope of the invention as defined by the appended claims. However, it should be understood that such modifications are within the technical scope of the present invention. Accordingly, the true scope of the present invention should be determined by the technical idea of the appended claims.

1: Display device
10:
21, 22:
40: direct image implementation unit
41: Display element
42a: a first spatial light modulator
42b: a second spatial light modulator
43: lens array
44: beam splitter
50: External view opening /
51: polarization controller
52a: first polarizing film
52b: second polarizing film

Claims (9)

A fixture configured to be worn on the head;
A fixing unit coupled to the fixing unit,
A display element for reproducing the first fundamental image;
A first spatial light modulator for reproducing a second fundamental image;
A second spatial light modulator for reproducing a third fundamental image;
An integrated image implementation unit including a lens array for realizing the basic images as a three-dimensional stereoscopic image, and a beam splitter; And
And an external visual field opening / closing part coupled to the fixing part for blocking or transmitting an external visual field,
A display device capable of switching between the augmented reality mode and the virtual reality mode, wherein the augmented reality mode and the virtual reality mode are operated only when the external view is transmitted through the augmented reality mode and the external view is blocked.
The method according to claim 1,
The external visual field opening /
A display device capable of switching between an augmented reality and a virtual reality, characterized by comprising a polarization control device having a first polarizing film and a second polarizing film attached on both sides thereof.
3. The method of claim 2,
A data board connected to the display device, the first spatial light modulator, and the second spatial light modulator to transmit image information; And
Further comprising a driving board connected to the polarization controller to transmit a driving signal to the augmented reality and virtual reality.
The method according to claim 1,
Wherein the beam splitter is disposed at a predetermined angle on an optical path between an external visual field and a binocular,
Wherein the display device has a reflectance of 50% and a transmittance of 50%.
The method according to claim 1,
Further comprising a secondary fixing part formed to catch on the user's head or both ears of the user.
Reproducing a basic image on a display device configured to be worn on a user's head;
Outputting an integrated image as a three-dimensional stereoscopic image using the basic image in the display device; And
Selectively blocking an external view of a user wearing the display device for either an augmented reality mode or a virtual reality mode,
A method for switching between an augmented reality mode and a virtual reality mode, wherein the augmented reality mode and the virtual reality mode are operated only when the external view is transmitted and the external view is blocked.
The method according to claim 6,
Selectively blocking the external field of view,
And transmitting or blocking light incident from the outside to the display device, wherein the augmented reality and the virtual reality are switched.
8. The method of claim 7,
Wherein the step of transmitting or blocking the light incident on the display device is performed through polarization control, wherein the augmented reality and the virtual reality are switched.
The method according to claim 6,
Wherein the step of outputting the integrated image includes the step of reflecting the integrated image using a beam splitter disposed between the binocular and the external visual field of a user wearing the display device, How to.

Images