KR101619468B1 - Display device for displaying 3D virtual reality media and control method thereof - Google Patents

Abstract

The present invention relates to a three-dimensional virtual reality image display device and a control method thereof which realizes a configuration that can be attached to and detached from spectacles, or held on the face of a user, and a mode that plays back various virtual reality images or three-dimensional images via wireless communication with an electronic device such as mobile devices. The present invention includes: an image receiving unit which wirelessly receives image data from the external image device; an object adjusting unit which extracts an object from the image data input into the image receiving unit, obtains depth information of the object and then adjusts the size of the object by using the depth information; an image adjusting unit which divides the adjusted image data which have been received from the object adjusting unit into left eye image data and right eye image data, respectively, and adjusts the resolution thereof; an image output unit which outputs the image data, which have been divided in the image adjusting unit, alternately as a left eye image and a right eye image to a left display and a right display, respectively; and an image control unit which controls the alternating timing of the left eye image and the right eye image which are output from the image output unit with reference to a preestablished value. Meanwhile, the left display and the right display have the configuration that can be attached to and detached from the spectacles, or held on the face of the user.

Description

Technical Field [0001] The present invention relates to a 3D virtual reality image display device and a control method thereof,

The present invention relates to a 3D virtual reality image display apparatus and a control method thereof, and more particularly, to a 3D virtual reality image display apparatus and a 3D virtual reality image display apparatus that interoperate with a video device that reproduces an image through wireless communication with a video device such as a smart phone, A display device and a control method thereof.

Virtual reality is the process of creating a stereoscopic 3D content of a specific environment or situation through a computer and creating a human-computer interaction that makes the person using the 3D content appear to be interacting with the actual surroundings and environment Interface and so on.

Generally, the three-dimensional sensation perceived by a person depends on the degree of thickness change of the lens according to the position of the observed object, the angle difference between the eyes and the object, the position and shape difference of the object on the left and right eyes, , And various other psychological and memory effects.

Among them, binocular disparity is the most important factor for a person to feel the stereoscopic effect, as the eyes of a person are about 6.5 cm apart in the horizontal direction. In other words, the binocular disparity causes an angle difference to the object. This difference causes the images coming into each eye to have different phases. When the binocular image is transmitted to the brain through the retina, It is possible to feel the original 3D stereoscopic image by fusing them exactly.

These stereoscopic 3D contents have already been widely used in various media fields and have been well received by consumers. For example, 3D movies, 3D games, and experience displays are examples.

In this situation, the recent development of mobile devices such as smart phones, high smartphone penetration rate, and the infinite contents network of smart phones enable users to easily experience virtual reality using a smartphone or easily view 3D images. A number of techniques have been steadily being sought and tried.

Conventionally, 3D contents, technologies, and inventions using smart phones have been developed. In general, goggles are mounted on the user's head, and a smartphone is inserted in front of the goggles. A display of a smartphone is displayed on a display device, and a stereoscopic screen reproduced on a smartphone is displayed in a form of a control system in which a smartphone detects movement of a head and applies it to a display. It is commonly used in various names such as face mounted display, VR goggles and the like, but the name HMD (Head mounted display) is generally widely used.

However, there are various problems in the case of a goggle-type HMD in which the smartphone is inserted in front of the user. For example, a user feels a considerable heat generated when a user plays an image using a smartphone, , A problem that the user wearing glasses may cause dizziness, the size of the goggles, or the space in which the smartphone is inserted, and thus, all the mobile devices can not be accommodated. There are several small and large technical problems, such as the lack of consistent display depending on the mobile device family, the battery exhaustion of mobile devices, and the large size and thick shape of the mobile device, making it unsafe for public use.

SUMMARY OF THE INVENTION The present invention has been made to solve the above-mentioned problems occurring in the prior art, and it is an object of the present invention to provide an electronic device, such as a video device, It is an object of the present invention to provide a virtual reality display device and method that are interlocked with an image device implementing a method of reproducing an image or a 3D image.

A 3D virtual reality image display apparatus according to an exemplary embodiment of the present invention includes: a video image receiving unit that wirelessly receives image data from an external video device; An object control unit for extracting an object from the image data input from the image receiving unit, acquiring depth information of the object, and adjusting the size of the object through the depth information; An image controller for dividing the adjusted image data received through the object controller into left and right eye image data and adjusting the resolution; An image output unit for outputting image data divided by the image adjusting unit to left and right displays alternately with a left eye image and a right eye image; And an image control unit for controlling the alternate timing of the left eye and right eye images output from the image output unit with reference to a predetermined value; Wherein the left and right displays have a form that can be attached to or detached from the glasses or a form that can be mounted on the face of the user.

Preferably, the image data may be side by side image data in which the left eye image and the right eye image are connected to each other to form a single frame.

Preferably, the image receiving unit includes at least one wireless communication unit selected from among a beacon, a Bluetooth, a Wi-Fi infrared communication, and a cellular network communication, It can transmit / receive video device and display signal.

Preferably, the object controller may convert the RGB data, which is input image data, into luminance data, and convert the luminance data into luminance data obtained by normalizing an object group having a luminance of a certain range into one object A luminance standardization unit generating a luminance standard; A depth generator for generating a depth image by dividing the normalized luminance data into n * m blocks and setting pixels having the same luminance value to have the same depth value in each block; And scanning the depth image to scan the target pixel and assuming that the depth of eight pixels surrounding the target pixel is different from the depth of the target pixel, And a size changing unit for changing the size of the image.

Preferably, the image adjusting unit comprises: an image data dividing unit dividing the image data adjusted by the object adjusting unit into left eye image and right eye image data; An area detecting unit for detecting a free area of each of the left eye image and the right eye image of the image data divided by the image data dividing unit; And restoring the empty area by obtaining image data of an area corresponding to each empty area of the left eye image and right eye image data detected by the area detecting part from an image adjacent to one direction of each of the left eye image and the right eye image data An image restoration unit; And a resolution controller for adjusting a resolution of the image data reconstructed by the image reconstructor.

A control method of a 3D virtual reality image display apparatus according to an embodiment of the present invention includes: receiving image data from an external image device; Extracting an object of the received left eye image and right eye image data; Acquiring depth information of an object of the extracted image data and adjusting an object size; Separating the image data in which the size of the object is adjusted into a left eye image and a right eye image, and adjusting respective resolutions; And alternately displaying the resolution-adjusted image.

Preferably, the step of acquiring the depth information and adjusting the size of the object comprises: converting RGB data, which is input image data, into luminance data; Generating luminance data by converting the converted luminance data into standardized luminance data of a group of objects having a luminance of a certain range; Dividing the normalized luminance data into n * m blocks and generating a depth image by setting pixels having the same luminance value to have the same depth value in each block; And scanning the depth image to scan the target pixel and assuming that the depth of eight pixels surrounding the target pixel is different from the depth of the target pixel, ; . ≪ / RTI >

Preferably, the step of separating the left eye image and the right eye image into the left eye image and the right eye image and adjusting the respective resolutions includes: dividing the image data from the stereoscopic image conversion unit into left eye image and right eye image data; Detecting an empty region of each of the left eye image and the right eye image of the divided image data; Acquiring image data of an area corresponding to each of the empty regions of the detected left eye image and right eye image data from an image adjacent to one direction of each of the left eye image and the right eye image data to restore the empty area; And adjusting a resolution of the restored image data.

According to the 3D virtual reality image display apparatus and the control method thereof of the present invention, it is possible to easily attach and detach the user to the face by providing a new glasses type display instead of using a smart phone display.

Also, the problem that the heat of the smartphone directly affects the user is solved, and the user can watch the consistent virtual reality image without limitation in the kind of the mobile device.

In addition, in the case of the wearer wearing glasses, the method of easily attaching and detaching the wearer's eyeglasses in front of the eyeglasses is effective to solve the dizziness which is appealed when the wearer wears glasses and wears the HMD goggles for a conventional smartphone.

In addition, there is an effect of significantly increasing the portability and convenience by wirelessly processing the data transmission / reception method between the glasses-type display and the mobile device.

1 is a block diagram briefly showing a configuration of a 3D virtual reality image display apparatus according to an embodiment of the present invention.
2 is a diagram illustrating a 3D virtual reality image display apparatus according to an exemplary embodiment of the present invention.
3 is a view showing another embodiment of a 3D virtual reality image display apparatus according to an embodiment of the present invention.
4 is a block diagram briefly showing a configuration of an object control unit in a 3D virtual reality image display apparatus according to an embodiment of the present invention.
5 is a block diagram briefly showing a configuration of an image control unit in a 3D virtual reality image display apparatus according to an embodiment of the present invention.
FIG. 6 is a flowchart illustrating a process of processing a 3D image received from an external video device according to an exemplary embodiment of the present invention. Referring to FIG.

Hereinafter, preferred embodiments of the present invention will be described in detail with reference to the accompanying drawings.

While the invention is susceptible to various modifications and alternative forms, specific embodiments thereof are shown by way of example in the drawings and will herein be described in detail. Prior to that, terms and words used in the present specification and claims are not intended to be limiting, nor should they be construed in a conventional or dictionary sense, and that the inventor shall, in order to best explain his invention in the best possible way It should be construed in the meaning and concept consistent with the technical idea of the present invention based on the principle that the concept of the term can be properly defined. Therefore, the embodiments described in the present specification and the configurations shown in the drawings are only the most preferred embodiments of the present invention, and they do not represent all the technical ideas of the present invention. Therefore, It should be understood that various changes and modifications may be made. In addition, the following embodiments are provided to help those skilled in the art to more easily understand the present invention. The shapes and sizes of elements in the drawings may be exaggerated for clarity.

1 is a block diagram briefly showing a configuration of a 3D virtual reality image display apparatus according to an embodiment of the present invention.

Referring to FIG. 1, a 3D virtual reality image display apparatus 100 according to an exemplary embodiment of the present invention includes an image receiving unit 110 for receiving image data wirelessly from an external image device 200; An object control unit for extracting an object from image data input from the image receiving unit 110 and acquiring depth information of the object and adjusting the size of the object through the depth information, (120); An image controller 130 for dividing the adjusted image data received through the object controller 120 into left and right eye image data and adjusting the resolution; An image output unit 140 that alternately outputs image data divided by the image adjusting unit 130 to a left eye image and a right eye image and outputs the same to the left and right display unit 160; And an image controller 150 for controlling the alternate timing of the left and right eye images outputted from the image output unit 140 with reference to predetermined values. And the left and right display units 160 can be attached to or detached from the glasses or can be mounted on the user's face.

Referring to FIG. 2, the left and right display units can be detachably attached to the spectacles. A typical clip 10 (see FIG. 2), in which ordinary spectacle wearers can easily attach and detach the left and right display units 160 to / Are provided at the center of the left and right displays.

Referring to FIG. 3, an embodiment of the left and right display units 160, which can be mounted on the user's face, includes general eye-glasses, glasses, an EMD (Eye Mounted Display), an HMD Head mounted display (HWD), head-up display (HUD), and goggles, and may include various shapes that can be easily attached and detached by the user, but it is preferably in the form of glasses.

The present invention is not particularly limited in terms of the above-mentioned terms with respect to display related terms or implementation methods.

The external image device 200 of the present invention means a digital television, a personal computer, a notebook, a smart phone, a tablet PC, a game machine, and various other portable devices having a display panel, and the present invention is not limited thereto. And various types of devices capable of outputting video and audio contents through a 3D virtual reality image display device.

The image content may include a general 2D image, a 3D image, a virtual reality, and the like, and may include all contents that can be implemented through a display.

The 3D virtual reality image display apparatus 100 of the present invention may include a power input unit (not shown) for receiving external AC power, and the power input unit may be configured to display the 3D virtual reality image display apparatus 100 Various 5-pin, 6-pin, 8-pin, and 30-pin ports used for various mobile devices can be used for the application to the product, and since the power input unit is a general matter, a detailed description will be omitted.

The present invention may further include a charge unit (not shown) capable of charging a power source input through the power input unit.

The battery used for the battery unit is not particularly limited and may be provided in various types. For example, a nickel-cadmium battery, a nickel-hydrogen battery, a lithium-ion battery, and a circular lithium-ion battery using various polymers can be used. Preferably, a small circular lithium- .

The image data of the present invention means data including 3D image information. The 3D image data includes left eye image data and right eye image data in one data frame region. The type of the 3D image data is classified according to the form including the left eye image data and the right eye image data.

Generally, 3D image data of a side-by-side method, a tom-bottom method, and a 2D + Depth method are divided into a left eye image data and a right eye image data by a split method, 3D image data including a horizontal interleaved method, a vertical interleaved method, and a checker board method are input to the left eye image data and the right eye image data by an interleave method, There is 3D image data including image data. Preferably, the image data of the present invention is a side-by-side image data composed of one frame, the left eye image and the right eye image being connected to each other have.

The image receiving unit 110 of the present invention may receive an image signal from a broadcasting station or satellite, but may be connected to an external image device to receive image data.

The S-Video component, the composite, the D-Sub, the DVI, the HDMI, and the HDMI may be connected to the external video device 200 through the video receiving unit 110. However, And various wireless interfaces such as a beacon, a Bluetooth, a Wi-Fi infrared communication, and a cellular network communication may be provided. In addition, And wireless communication means, but it is preferable that the image receiving unit 110 transmits / receives a display signal (image data) to / from the external image device 200 through the wireless communication means.

When the external video device 200 and the 3D virtual reality image display device 100 of the present invention are connected to the 3D virtual reality image display device 100 through the wireless communication means, the video device converts various compatible video content data into the 3D virtual reality image display device (100).

At this time, the received image may be a 2D image or a 3D image.

When the 2D image is received, the 3D virtual reality image display apparatus 100 of the present invention converts the 2D image into a 3D image and provides the 3D image to the user. When the 3D image is received, the 3D image data is processed You may.

4 is a block diagram briefly showing a configuration of an object control unit in a 3D virtual reality image display apparatus according to an embodiment of the present invention.

The object modifier 120 of the present invention extracts an object from the image data input from the image receiver 110 and obtains depth information of the object and then outputs the depth information Referring to FIG. 2, the object adjusting unit 120 includes a luminance normalizing unit 121, a depth generating unit 122, and a size adjusting unit 123.

The luminance normalization unit 121 converts the RGB data, which is input data, into luminance data, and then generates the luminance data that is the standardized object group of the object group having luminance within a certain range as the luminance data .

The brightness normalization unit 121 obtains brightness values of the respective pixels of the image data, and converts the RGB data of the input image into brightness data. Here, the luminance value Y of each pixel can be obtained by applying the R, G, and B values of the corresponding pixel to Equation (1).

The brightness normalization unit 121 normalizes and simplifies the brightness values of each pixel obtained through Equation (1) to several levels. Preferably, five steps are suitable, and the luminance in the image has a gradation value of 0 to 255 as a gradation value of the image. If this is standardized in five steps, it can be simplified to an image having the same luminance value in units of about 50 levels.

By simplifying the above-described luminance values, one identical object having a luminance value within a certain range in the image is treated as one luminance value, so that the image is divided into areas and identified and separated.

The depth generation unit 122 divides the standardized brightness data into n * m blocks and sets the pixels having the same luminance value in each block to have the same depth value to generate a depth image can do.

Here, the depth is basic information for providing a perspective of a video signal, which is distance information indicating a distance between the object and the camera in the image data and how close the camera is.

To be more specific, the boundary pixels of each block and adjacent blocks are compared, and the pixels having the same luminance value are set to have the same depth value. Thus, the higher the luminance, the closer it is expressed, and the lower the luminance, the greater the distance.

When the depth value of the target pixel and the surrounding eight pixels are different from each other while scanning the depth image, the size adjusting unit 123 may regard the target pixel as a noise, To a depth value of " 0 "

More specifically, the size of the object included in the input image is controlled by the extracted depth information. When it is determined that the distance between the camera and the object is close to the object based on the extracted depth information, The control unit 123 controls the size of the object determined to be close to the distance to be largely adjusted.

Also, when it is determined that the distance between the camera (not shown) and the object is long, the size adjusting unit 123 controls the size of the object determined to be long to be small.

As described above, by adjusting the size of the object using the depth information of the input image data through the function of the object adjusting unit 120, the user can view the 3D image with more perspective and three-dimensional feeling.

FIG. 5 is a block diagram illustrating a configuration of an image control unit in a 3D virtual reality image display apparatus according to an exemplary embodiment of the present invention. Referring to FIG.

5, the image controller 130 divides the adjusted image data received through the object controller 120 into left and right eye image data and adjusts the resolution, The image processing unit 130 includes an image data dividing unit 131, an area detecting unit 132, an image restoring unit 133, and a resolution adjusting unit 134.

The image data dividing unit 131 divides the image data adjusted by the object adjusting unit 120 into left eye image, right eye image data, and ancillary data.

Here, the additional data may be, for example, a subtitle or OSD (On Screen Display) data.

More specifically, the image data dividing unit 131 generates a left eye image and a right eye image corresponding to the input image under the control of the image controller 150, which will be described later. Specifically, when a 2D image is input, the image data dividing unit 131 moves the position of the object included in the 2D image to generate a left eye image and a right eye image. At this time, the image data division unit 131 generates a left eye image and a right eye image of which the size of the object is adjusted in accordance with the depth information, and provides the user with a more perspective and three-dimensional image.

In addition, when a 3D image is input, the image data dividing unit 131 can generate a left eye image and a right eye image interpolated in one screen size by using the 3D processed image data. At this time, the image data dividing unit 131 generates the left eye image and the right eye image whose size is adjusted according to the depth information.

The area detecting unit 132 detects a free area of each of the left eye image and the right eye image of the image data divided by the image data dividing unit 131. [ Specifically, the region detecting unit 132 may detect the associated image data from the left eye image and the right eye image of the 3D image data and separate the detected image data from the left eye image and the right eye image. When additional data is detected from the left eye image and the right eye image, the region detection unit 132 detects a free region of the left eye image and the right eye image according to detection of additional data.

The image reconstructing unit 133 reconstructs the image data of the region corresponding to each of the empty regions of the left eye image and the right eye image data detected by the region detecting unit 132, And reconstructs the blank area. To be more specific, the image reconstruction unit 133 may use the right eye image included in the 3D image data together with the left eye image to reconstruct the empty region of the left eye image. For example, if it is detected that the empty region of the left eye image is located at the lower right through the region detection unit 132, the image reconstruction unit 133 tracks the right eye image and outputs the lower right region of the left eye image to the lower right region of the right eye image And restores the corresponding image.

The resolution adjusting unit 134 adjusts the resolution of the image data restored by the image restoring unit 133.

The resolution converter may convert the resolutions of the reconstructed left and right eye images to generate a 3D image.

Here, the resolutions of the left eye image and the right eye image are converted through the resolution converting unit 220 to generate a 3D image.

The image output unit 140 of the present invention outputs the left and right images to the left and right display unit 160 by alternating the image data divided by the image adjusting unit 130 with the left eye image and the right eye image.

The image control unit 150 of the present invention controls the alternate timing of the left eye and the right eye images output from the image output unit 140 by referring to preset values.

The operation of the 3D virtual reality image display apparatus according to an embodiment of the present invention will now be described in detail with reference to the accompanying drawings.

6 is a flowchart illustrating a process of processing a 3D image received from an external video device according to an exemplary embodiment of the present invention.

Referring to FIG. 6, first, the image data is received from the external image device 200 (S101), and objects of the left eye image and the right eye image data are extracted from the image data received in the step S101 (S102).

Next, the depth information of the extracted image data object is acquired (S103), and the acquired depth information is obtained as follows.

That is, the RGB data, which is the input image data, is converted into luminance data, and the converted luminance data is generated as luminance data obtained by normalizing an object group having luminance of a certain range into one object, Data is divided into n * m blocks, and pixels having the same luminance value in each block are set to have the same depth value to generate and acquire a depth image.

Thereafter, the size of the object is adjusted (S104). When the depth value of the target pixel and the surrounding eight pixels are different while scanning the acquired depth image, it is assumed that the noise is noise And changes the target pixel to a depth value of the surrounding pixels.

In step S104, the image data whose size is adjusted is separated into a left eye image and a right eye image (S105).

That is, the image data from the stereoscopic image conversion unit is divided into a left eye image and a right eye image data, a blank region of each of the left eye image and the right eye image of the divided image data is detected, Acquires image data of a region corresponding to each free region of the left eye image and the right eye image data from an image adjacent to one direction of each of the left eye image and the right eye image data,

After the resolution of the restored image data is adjusted (S106), the resolution-adjusted image is alternately displayed on the left and right display unit 160 (S107).

While the present invention has been particularly shown and described with reference to exemplary embodiments thereof, it is clearly understood that the same is by way of illustration and example only and is not to be construed as limiting the scope of the invention as defined by the appended claims. It will be understood by those skilled in the art that various changes in form and details may be made therein without departing from the spirit and scope of the present invention.

10: Clip
100: 3D virtual reality image display device
110:
120:
130:
140: Video output unit
150:
160: right and left display unit
200: External image device

Claims (8)

An image receiving unit for receiving image data from an external image device;
An object control unit for extracting an object from image data received by the image receiving unit, acquiring depth information of the object, and adjusting the size of the object through the depth information;
An image controller for dividing the adjusted image data received through the object controller into left and right eye image data and adjusting the resolution;
An image output unit for outputting image data divided by the image adjusting unit to left and right displays alternately with a left eye image and a right eye image; And
And an image control unit for controlling the alternate timing of the left eye and right eye images output from the image output unit with reference to a preset value
The left and right displays have a form capable of being attached to or detached from the glasses or a form capable of being mounted on the face of the user,
Wherein the object control unit comprises:
A luminance standardization unit for converting RGB data, which is the received image data, into luminance data, and generating the converted luminance data into luminance data that is standardized into one object group having a luminance of a certain range;
A depth generator for generating a depth image by dividing the normalized luminance data into n * m blocks and setting pixels having the same luminance value to have the same depth value in each block; And
If the depth value of the target pixel and the surrounding eight pixels are different while scanning the depth image, the target pixel is changed to the depth value of the neighboring pixels, And a size adjusting unit for adjusting the size of the 3D virtual reality image. The method according to claim 1,
Wherein the image data is side by side image data in which the left eye image and the right eye image are connected side by side to form one frame. The method according to claim 1,
Wherein the image receiving unit includes at least one wireless communication unit selected from a beacon, a Bluetooth, a Wi-Fi infrared communication, and a cellular network communication,
And transmits / receives a display signal to / from an external video device through the wireless communication means. delete The method according to claim 1,
The image controller may include:
An image data dividing unit for dividing the image data adjusted by the object adjusting unit into left eye image and right eye image data;
An area detecting unit for detecting a free area of each of the left eye image and the right eye image of the image data divided by the image data dividing unit; And
Eye image data and a right eye image data obtained by obtaining image data of an area corresponding to each empty area of the left eye image and right eye image data detected by the area detecting unit from an image adjacent to one direction of each of the left eye image and the right eye image data, Restoring unit;
And a resolution adjusting unit for adjusting a resolution of the image data restored by the image restoring unit. Receiving image data from an external video device;
Extracting objects of the received left eye image and right eye image data;
Acquiring depth information of an object of the extracted image data and adjusting an object size;
Separating the image data in which the size of the object is adjusted into a left eye image and a right eye image, and adjusting respective resolutions; And
And displaying the image with the resolution adjusted in an alternating manner,
The step of acquiring the depth information and adjusting the size of the object includes:
Converting RGB data, which is the received image data, into luminance data; Generating luminance data by converting the converted luminance data into standardized luminance data of a group of objects having a luminance of a certain range;
Dividing the normalized luminance data into n * m blocks and generating a depth image by setting pixels having the same luminance value to have the same depth value in each block; And
If the depth value of the target pixel and the surrounding eight pixels are different while scanning the depth image, the target pixel is changed to the depth value of the neighboring pixels, And displaying the 3D virtual reality image on the display unit. delete The method according to claim 6,
Separating the left eye image and the right eye image into the left eye image and the right eye image,
Dividing the image data from the object adjustment unit into left eye image and right eye image data;
Detecting an empty region of each of the left eye image and the right eye image of the divided image data;
Acquiring image data of an area corresponding to each of the empty regions of the detected left eye image and right eye image data from an image adjacent to one direction of each of the left eye image and the right eye image data to restore the empty area;
And adjusting a resolution of the restored image data based on the restored image data.

Images