KR101838501B1 - HMD device displaying both a virtual contents and a real contents and the method displaying the contents - Google Patents

Abstract

[0001] The present invention relates to a stereoscopic image HI-DEMD device, and more particularly, to a stereoscopic image HI-DEMD device configured to display a virtual screen and sometimes a virtual screen and a real screen according to a situation.
The stereoscopic image H / M device for simultaneously displaying a virtual screen and a real screen according to an embodiment of the present invention includes a left eye camera for capturing a left eye image and a right eye camera for capturing a right eye image, A stereoscopic image generating device for generating a right eye image and a left eye image for a virtual screen, a display device for displaying the virtual screen and the real screen, a display device mounted on a predetermined portion of the case, And a screen selector for displaying only the virtual screen on the display device or simultaneously displaying the virtual screen and the real screen according to a tilt value detected by the motion detection sensor, Wherein the screen selection unit comprises: Dividing a screen of the display device into a virtual screen area and a real screen area when the sensed slope value exceeds a preset threshold value and displays the real screen in the real screen area, And is displayed as a stereoscopic image.

Description

TECHNICAL FIELD [0001] The present invention relates to a stereoscopic image display device and a display method using the same, and more particularly,

[0001] The present invention relates to a stereoscopic image HI-DEMD device, and more particularly, to a stereoscopic image HI-DEMD device configured to display a virtual screen and sometimes a virtual screen and a real screen according to a situation.

The HMD (Head Mount Display) device is a sight goggle or a helmet type video device for viewing the screen in front of the user. It was developed mainly to realize virtual reality. A small display such as a liquid crystal is provided at a position close to both eyes to project a stereoscopic image using time difference, and a direction facing the user's head is detected by a gyro sensor or the like, and an image corresponding to the motion is emphasized, It has also been developed to enable the experience to be like that.

Typical HMS products include Oculus Lift. Conventional HMD devices typically use very small LCDs or OLED panels to create a screen, but in order to achieve a high-quality virtual reality as small as a panel, a sophisticated lens is needed, which often leads to a price increase. However, Oculus Lift chooses a way to split the panel one way, and combining the left and right screens is not a lens, but a computer software that can use a larger lens than a conventional HMD. Relatively inexpensive. In addition, the Oculus Lift reflects the movement of the user on the screen through the infrared (IR) sensor, and the image changes according to the direction and position of the user, thus realizing a virtual reality experience.

The recently released Oculular Lift consists of a headset, a tracking sensor for motion detection, a remote control and a controller. The headset has built-in headphones and a microphone for delivering 3D audio effects, and a gyro tracking function. The tilting according to the movement of the head is reflected. The tracking sensor tracks all the user's movements.

However, this conventional HDM device is a structure that shows only a virtual reality screen (virtual screen) on the screen.

When a user wears a HMI device, a situation in which a real screen is displayed instead of a virtual screen when necessary is required. In the conventional HMI device, only a virtual screen is provided without displaying a real screen Therefore, there is an inconvenience that the user must take off the HDM device in order to view the real screen.

SUMMARY OF THE INVENTION The present invention has been made to solve the above problems, and it is an object of the present invention to provide a method and system for simultaneously displaying a virtual screen and a real screen, HMI devices are provided.

Also, according to the present invention, when the user moves or tilts the HDM device at a predetermined slope, the user can easily see whether the user views the virtual screen or the virtual screen and the real screen at the same time by simultaneously displaying the real screen with the virtual screen Thereby enabling the user to quickly view the real screen when the user desires.

A stereoscopic image H / D device for simultaneously displaying a virtual screen and a real screen according to an embodiment of the present invention includes a case; A camera device mounted on a predetermined portion of the case and configured to capture a left eye image for capturing a left eye image and a right eye camera for capturing a right eye image, A stereoscopic image generation device for generating a right eye image and a left eye image for a virtual screen; A display device for displaying the virtual screen and the real screen; A motion detection sensor mounted on a predetermined portion of the case to sense movement and inclination of the case; And a screen selecting unit displaying only the virtual screen on the display device or displaying the virtual screen and the real screen at the same time according to a tilt value sensed by the motion detecting sensor.

Wherein the screen selecting unit divides the screen of the display device into a virtual screen area and a real screen area when the tilt value sensed by the motion detection sensor exceeds a preset threshold value, and displays the real screen in the real screen area do.

delete

delete

The virtual screen and the real screen are displayed as stereoscopic images.

According to another aspect of the present invention, there is provided a method of displaying an image using a stereoscopic image HI-DEM device for simultaneously displaying a virtual screen and a real screen, comprising the steps of: displaying a virtual screen input from the outside on a display device of the HI- A step of photographing a real screen using a camera device, which is mounted on a predetermined portion of the case and includes a sin camera for capturing a left eye image and a right eye camera for capturing a right eye image; Sensing movement and tilt of the case using a motion detection sensor mounted on a predetermined portion of the case; And a screen selection step of displaying only the virtual screen on the display device or simultaneously displaying the virtual screen and the real screen according to a tilt value sensed by the motion detection sensor.

Wherein the screen selection step divides the screen of the display device into a virtual screen area and a real screen area when the tilt value sensed by the motion detection sensor exceeds a preset threshold value, Display. Here, the virtual screen and the real screen are displayed as stereoscopic images.

delete

delete

According to the present invention, a function of viewing a virtual screen and a real screen at the same time is provided in the HDM device, so that the user can view the real screen simultaneously when the user wants to view the virtual screen using the HDM device.

In addition, according to the present invention, when a user moves or tilts the HDM device at a predetermined slope, a real screen is displayed at the same time as a virtual screen, so that the user can conveniently and quickly view the real screen when he / she desires.

FIG. 1 is a diagram illustrating a configuration of a stereoscopic image HI-DEM device for simultaneously displaying a virtual screen and a real screen according to an embodiment of the present invention, and FIG.
FIG. 2 is a flowchart for explaining an image display method by a stereoscopic image HI-DEM device for simultaneously displaying a virtual screen and a real screen according to an embodiment of the present invention,
3 is a view showing that a virtual screen is provided to a user,
4 is a view showing a virtual screen and a real screen at the same time to a user,
5 is a diagram showing an example in which the virtual screen and the real screen are displayed so as to gradually overlap with each other.

DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS Hereinafter, embodiments of the present invention will be described with reference to the accompanying drawings.

It is to be noted that the same components of the drawings are denoted by the same reference numerals and symbols as possible even if they are shown in different drawings. In the following description of the present invention, a detailed description of known functions and configurations incorporated herein will be omitted when it may make the subject matter of the present invention rather unclear. Also, when a part is referred to as "including " an element, it does not exclude other elements unless specifically stated otherwise.

It should also be understood that the terms or words used in the present specification and claims should not be construed in a conventional and dictionary sense and that the inventors may properly define the concepts of the terms to best describe their invention And should be construed in accordance with the principles and meanings and concepts consistent with the technical idea of the present invention. Therefore, the embodiments described in the present specification and the configurations shown in the drawings are merely preferred embodiments of the present invention, and are not intended to represent all of the technical ideas of the present invention, so that various equivalents And the scope of the present invention is not limited to the following embodiments.

FIG. 1 is a diagram illustrating a configuration of a stereoscopic image HI-DEMD device for displaying a virtual screen and a real screen simultaneously according to an embodiment of the present invention.

Referring to FIG. 1, a stereoscopic image HADM device 100 for simultaneously displaying a virtual screen and a real screen according to an embodiment of the present invention includes a case 190, a camera device 110, a display device (not shown) An image generation device (not shown), a motion detection sensor 190, and a screen selection unit.

The case 190 constitutes an external appearance of the HMI device 100 and the camera device 110 and the motion detection sensor 120 are mounted on the case 190.

The camera device 110 is mounted on a predetermined portion of the case 190 and is used for photographing a real screen. The camera device 110 is used for photographing the surroundings of a user wearing the H- will be.

The display device is for displaying a virtual screen and the real screen. The stereoscopic image generating device generates a right eye image and a left eye image of a virtual screen and displays a stereoscopic image on the display device. The display device may be implemented in various ways to display stereoscopic images. That is, the display device is implemented so that the right eye image is input only to the right eye of the user and the left eye image is input to the left eye of the user, so that the user can enjoy the stereoscopic image.

The motion detection sensor 120 is mounted on a predetermined portion of the case, and detects movement and tilt of the case to detect movement of the user. In other words, since the user wears the hermetic device when the case is fixed to the face of the user, if the movement and tilt of the case are sensed, the user can know the movement of the user or how much the user has tilted the head.

The screen selection unit displays only a virtual screen on the display device or displays a virtual screen and a real screen simultaneously according to a tilt value sensed by the motion sensor.

In the embodiment of the present invention, a virtual screen is usually displayed through the HDM device, and when the user leans his / her head at a predetermined slope, the user senses the virtual screen and displays the virtual screen and the real screen together. It is possible to detect the movement of the user wearing the H-MDM device and the inclination of the head of the user, thereby detecting a slope value exceeding a preset predetermined slope or detecting a preset predetermined limit Value is detected, a virtual screen and a real screen are simultaneously displayed through the HDM device.

In the embodiment of the present invention, when the tilt angle value of the case detected by the motion detection sensor exceeds a predetermined threshold value, the screen selecting unit divides the screen of the display device into a virtual screen area and a real screen area, The real screen is displayed on the real screen area.

When the virtual screen and the real screen are simultaneously displayed, the screen selection unit displays the virtual screen on the upper portion and displays the real screen on the lower portion.

The virtual screen and the real screen may be divided and displayed in various manners. When the virtual screen and the real screen are simultaneously displayed, the virtual screen is displayed on the upper part, the real screen is displayed on the lower part , It is possible to display the virtual screen and the real screen in such a manner that the virtual screen and the real screen progressively overlap at a midpoint between the virtual screen and the real screen.

In order to display the real screen as a stereoscopic image, the right eye image and the left eye image for the real screen must be input. Therefore, in this case, The apparatus 110 is implemented by a left eye camera 110a for taking a left eye image and a right eye camera 110b for taking a right eye image.

Hereinafter, a method of displaying an image by a stereoscopic image HI-DEM device for simultaneously displaying a virtual screen and a real screen according to an embodiment of the present invention will be described.

FIG. 2 is a flowchart for explaining a method of displaying an image by a stereoscopic image HADM device that simultaneously displays a virtual screen and a real screen according to an embodiment of the present invention.

A method of displaying an image by a stereoscopic image HADM device that simultaneously displays a virtual screen and a real screen according to an embodiment of the present invention includes displaying a virtual screen input from the outside on a display device of the HADM device (S110) (S130) of detecting a movement and a tilt of the case using a motion detection sensor mounted on a predetermined portion of the case (S130), and a step And a screen selection step (S140) of displaying only the virtual screen on the display device or simultaneously displaying the virtual screen and the real screen according to the tilt value sensed by the sensing sensor.

First, in step S110 of displaying a virtual screen input from the outside on a display device of the HDM device, a virtual screen, which is a virtual reality image selected by the user so that the user can view the virtual reality image, is displayed on the display device of the HDM device Of-life.

Next, a real screen is photographed using a camera device mounted on a case of the HDM device (S120). The camera device is for photographing a surrounding environment of a location where a user wearing the HDM device is mounted on the front or top of the case of the HDM device, A real-life image of a person is photographed.

In the real-scene photographing step S120, a real camera may be used to photograph a real image, or a stereoscopic image may be photographed using two cameras.

In case of implementing stereoscopic image capturing, two cameras including a left eye camera and a right eye camera are used to capture a left eye image with a left eye camera and a right eye image with a right eye camera.

There may occur a situation in which the user tries to check the actual surroundings environment of the user while viewing the virtual screen. For example, a driver enjoying a road driving game may experience a situation where he or she wants to check the position of a steering wheel, a brake, an accelerator, etc.,

In this case, the user has to give a certain command to provide an image of the surrounding environment in the HMI device. In the embodiment of the present invention, by using the moving detection sensor mounted on the HMI device, The user recognizes that the user desires to receive the image of the actual surroundings by detecting the inclination of the HDM device at a certain angle.

That is, the movement and inclination of the case are sensed using a motion detection sensor mounted on a predetermined portion of the case (S130), and it is recognized that the user desires to receive the image of the actual surroundings, Only the screen is displayed or the virtual screen and the real screen are simultaneously displayed (S140).

3 is a view showing a virtual screen provided to a user, and FIG. 4 is a view showing a virtual screen and a real screen simultaneously to a user.

In the screen selection step S140, a threshold value for a slope is preset and stored in a control unit (not shown) for controlling various operations such as the display operation of the HDM device, and then, And displays the real screen on the real screen area by dividing the screen of the display device into a virtual screen area and a real screen area when the slope value exceeds a preset threshold value.

4, in the embodiment of the present invention, when the virtual screen D100 and the real screen D200 are simultaneously displayed in the screen selection step S140, the virtual screen D100 is displayed on the upper side , And displays the real screen D200 at the bottom.

When the virtual screen and the real screen are displayed at the same time by displaying the virtual screen at the upper part and displaying the virtual screen at the lower part, the virtual screen and the real screen are progressively In FIG. 5, an area in which the virtual screen and the real screen are progressively overlapped in an intermediate area where the virtual screen D110 and the real screen D210 meet are displayed while simultaneously displaying the virtual screen and the real screen, (D300) is displayed.

100: HDM device 110: Camera device
120: Motion detection sensor 190: Case

Claims (9)

In a stereoscopic image DMB device,
case;
A camera device mounted on a predetermined portion of the case and including a left eye camera for capturing a left eye image of a real screen and a right eye camera for capturing a right eye image of the real screen;
A stereoscopic image generation device for generating a right eye image and a left eye image for a virtual screen;
A display device for displaying the virtual screen and the real screen;
A motion detection sensor mounted on a predetermined portion of the case to sense movement and inclination of the case; And
And a screen selector for displaying only the virtual screen on the display device or displaying the virtual screen and the real screen simultaneously according to a tilt value sensed by the motion detection sensor,
Wherein the screen selecting unit divides the screen of the display device into a virtual screen area and a real screen area when a slope value detected by the motion detection sensor exceeds a preset threshold value,
Displaying the virtual screen on the virtual screen area on the upper part of the screen of the display device and simultaneously displaying the real screen on the real screen area on the lower part of the screen of the display device,
Wherein the real screen is displayed as a stereoscopic image together with the virtual screen based on a left eye image of the real screen shot by the left eye camera and a right eye image of the real screen shot by the right eye camera. And a real-time screen simultaneously.
delete The method according to claim 1,
Wherein the screen selection unit displays the virtual screen and the real screen so that the virtual screen and the real screen progressively overlap at a midpoint of the screen of the display device where the virtual screen and the real screen meet, HI-MD device.
delete delete Displaying a virtual screen input from the outside on a display device of the HDM device;
Capturing the real screen using a camera device mounted on a predetermined portion of the case and including a left eye camera for capturing a left eye image of a real screen and a right eye camera for capturing a right eye image of the real screen;
Sensing movement and tilt of the case using a motion detection sensor mounted on a predetermined portion of the case; And
And a screen selection step of displaying only the virtual screen on the display device or simultaneously displaying the virtual screen and the real screen according to a tilt value sensed by the motion detection sensor,
Wherein the screen selection step comprises:
Wherein when the tilt value detected by the motion detection sensor exceeds a preset threshold value, the screen of the display device is divided into a virtual screen area and a real screen area,
The virtual screen is displayed on the virtual screen area on the upper side of the screen of the display device and the real screen is displayed on the real screen area on the lower part of the screen of the display device, Wherein the stereoscopic image is displayed as a stereoscopic image together with the virtual screen based on the left eye image of the real screen and the right eye image of the real image captured by the right eye camera. A method of displaying an image using a device.
delete The method according to claim 6,
Wherein the screen selection step displays the virtual screen and the real screen such that the virtual screen and the real screen progressively overlap at a midpoint of the screen of the display device where the virtual screen and the real screen meet. A method of displaying an image using a stereoscopic image HMI device.


delete

Images