KR101373294B1 - Display apparatus and method displaying three-dimensional image using depth map - Google Patents

Abstract

The 3D depth map is acquired and synthesized using the image, and the image is synthesized in consideration of the viewer's position. The 3D image display apparatus of the present invention includes: a depth map extractor extracting a 3D depth map for screen synthesis; An object position determiner that determines a position of an object in consideration of the 3D depth map; An object synthesizing unit synthesizing an object to be shown to the viewer according to the determined object position; And a display unit which displays the synthesized object to the viewer. The video display device may further include a viewer location determiner that checks the location of the viewer, wherein the object location determiner may further determine the location of the object in consideration of the location of the viewer.

Description

Display apparatus and method displaying three-dimensional image using depth map

The present invention relates to an image display method and apparatus, and more particularly, to a method and apparatus for displaying a 3D image using a depth map.

Recent developments of telecommunication and information technology have led to the development of TVs, which are representative household appliances, as smart TVs. Unlike past TVs, which transmit information through the screen and speakers, various types of sensors are embedded in the smart TV, which enables various new interfaces.

Accordingly, various functions are added in addition to information transmission, which is a basic function of a TV, and various services using interactive properties have been developed.

In addition, with the development of multimedia technology, three-dimensional content has been developed and spread, and a three-dimensional TV capable of watching three-dimensional images at home has been spread.

In addition to displaying 3D broadcast images to viewers through 3D TV, augmented reality technology for synthesizing and providing virtual information and real information in real time has also been developed.

When a graphic object is synthesized and displayed on a 3D broadcast image like augmented reality, the distance of a background image to be synthesized is estimated and displayed on the screen.

Related arts include patent applications 10-2010-0043386 and patent applications 10-2009-0070254.

Patent application No. 10-2010-0043386 relates to the IP system and its content providing method, the set-top box drives the camera to shoot an image within the shooting range, and determines whether the captured image includes the predetermined recognition information The unique code corresponding to the recognition information is transmitted to the content server, the content associated with the unique code is received from the content server, synthesized with the image photographed by the camera, and the synthesized image is displayed on the display device. Through this, the user discloses a device and a method that can be applied to augmented reality automatically according to the learning content to receive the content.

Patent application No. 10-2009-0070254 relates to a method for providing augmented reality service and a server and a portable terminal therefor, the display unit for displaying the execution screen of the running application; A terminal communication unit interlocked with the augmented reality providing server; A camera module for acquiring image information corresponding to a still image or a moving image; Extracting an image feature point from the acquired image information, transmitting the extracted image feature point to the augmented reality providing server, receiving local information data corresponding to the image feature point from the augmented reality providing server, and receiving the image information and the Disclosed is a portable terminal comprising a terminal controller for synthesizing regional information data and outputting the synthesized local information data through the display unit.

However, such a conventional image synthesis method is difficult to show a natural image because it does not consider the limitation of the synthesis and the position of the viewer.

An object of the present invention is to provide a three-dimensional image display method and apparatus capable of showing a natural image when synthesizing an image on an augmented reality or TV screen, as the object of the present invention.

In order to solve such a problem, in the present invention, a 3D depth map may be obtained and synthesized by using the synthesized image, and the synthesized image may be synthesized in consideration of the position of the viewer during image synthesis.

That is, the three-dimensional image display device according to an aspect of the present invention, the depth map extraction unit for extracting a three-dimensional depth map (depth map) for screen synthesis; An object position determiner that determines a position of an object in consideration of the 3D depth map; An object synthesizing unit synthesizing an object to be shown to the viewer according to the determined object position; And a display unit for displaying the synthesized object to the viewer.

The depth map extractor may extract a depth map from a stereo image received by the image display apparatus, or extract the depth image as a depth map when the image display apparatus receives one image and a depth image. Can be.

The video display device may further include a viewer location determiner that checks the location of the viewer, wherein the object location determiner determines the location of the object by further considering the location of the viewer.

According to another aspect of the present invention, there is provided a method of displaying a 3D image, comprising: extracting a 3D depth map for screen synthesis; Determining a position of an object in consideration of the 3D depth map; Synthesizing an object to be shown to the viewer according to the determined object position; And displaying the synthesized object to the viewer.

The method may further include determining a location of the viewer, and in the synthesizing of the object, the location of the object may be determined by further considering the location of the viewer.

According to the present invention, a natural 3D image may be provided in a service for synthesizing and displaying a synthesized object and a 3D broadcast image such as augmented reality.

In addition, the 3D image display method using the depth map of the present invention can be used not only for 3D broadcasting, but also for synthesis and display technology of general 2D / 3D image and graphic objects, and application using various real-world and virtual object synthesis, It can be used for games, advertisements, and the like.

FIG. 1 is a diagram schematically illustrating an overall configuration of a smart TV system displaying a 3D image using a depth map according to an exemplary embodiment of the present invention.
2 is a block diagram showing in more detail the configuration of a smart TV system for displaying a three-dimensional image using a depth map according to an embodiment of the present invention.
3 is a flowchart illustrating an image display method of a smart TV system displaying a 3D image using a depth map according to an exemplary embodiment of the present invention.

Advantages and features of the present invention and methods for achieving them will be apparent with reference to the embodiments described below in detail with the accompanying drawings. The present invention may, however, be embodied in many different forms and should not be construed as being limited to the embodiments set forth herein. Rather, these embodiments are provided so that this disclosure will be thorough and complete, and will fully convey the scope of the invention to those skilled in the art. Is provided to fully convey the scope of the invention to those skilled in the art, and the invention is only defined by the scope of the claims. It is to be understood that the terminology used herein is for the purpose of describing particular embodiments only and is not intended to be limiting of the invention. In the present specification, the singular form includes plural forms unless otherwise specified in the specification. It is noted that the terms "comprises" and / or "comprising" used in the specification are intended to be inclusive in a manner similar to the components, steps, operations, and / Or additions.

Hereinafter, with reference to the accompanying drawings will be described in detail with respect to a smart TV and a display method for displaying a three-dimensional image using a depth map according to an embodiment of the present invention.

FIG. 1 is a diagram schematically illustrating an overall configuration of a smart TV system displaying a 3D image using a depth map according to an exemplary embodiment of the present invention.

As shown in FIG. 1, a smart TV system displaying a personalized 3D image according to a viewer's position according to an exemplary embodiment of the present invention may display a 3D image on a TV 100 and a TV 100. It consists of a three-dimensional glasses 200 that the user wears to watch the three-dimensional image.

2 is a block diagram showing in more detail the configuration of a smart TV system for displaying a three-dimensional image using a depth map according to an embodiment of the present invention.

As shown in FIG. 2, the smart TV 100 of the smart TV system displaying a 3D image using a depth map according to an exemplary embodiment of the present invention may include a depth map extracting unit extracting a depth map of the 3D image. 120, a three-dimensional object position determination unit 130 for determining the position of the three-dimensional object in consideration of the background position of the screen using the depth map, the object synthesis unit 140 for synthesizing the object to be viewed by the viewer and the synthesized object It includes a display unit 150 for displaying to the viewer.

On the other hand, considering the position of the viewer watching TV can synthesize a more natural three-dimensional image. To this end, the smart TV 100 of the smart TV system displaying a 3D image using a depth map according to an embodiment of the present invention is a viewer position checking unit 110 for checking the position of a viewer watching the TV 100. It may further include. The viewer location determiner 110 may be implemented using various sensors.

As an embodiment, the smart TV 100 may be equipped with a camera to recognize the user's face by using the camera to determine the viewer's location. That is, the position of the face is confirmed by processing image data captured by the camera through a face recognition algorithm. There is no particular limitation on the face recognition algorithm, and various algorithms can be used.

In another embodiment, a sensor (not shown) such as a light emitting sensor is mounted on the three-dimensional glasses 200, and a light emitting recognition sensor is mounted on the smart TV 100 by the viewer positioning unit 110. The light emitting sensor 110 of the smart TV 100 may recognize the light emitted from the light emitting sensor of the glasses 200 to check the glasses and the user position.

The depth map extractor 120 extracts a depth map from image information transmitted for displaying a 3D broadcast image.

Current 3D broadcasting technologies include a technique of transmitting a stereo image and a technique of transmitting a single image and a depth image.

In a smart TV system displaying a 3D image using a depth map according to an exemplary embodiment of the present invention, the depth map extractor 120 included in the smart TV 100 extracts a depth image from a stereo image (stereo image). Is used to synthesize a 3D object using the received depth image (when receiving one image and the depth image).

As described above, when the position of the 3D object in consideration of the background position of the screen is determined using the depth map, a natural 3D image may be provided even though the image is synthesized.

In particular, when a user's location is determined and the images are combined together, a more natural image can be obtained.

The 3D object position determiner 130 determines a 3D position of the 3D object to be displayed based on the depth map extracted by the depth map extractor 120. For example, in order to show a cola can for a cola advertisement, a natural 3D image is determined by judging based on a depth map extracted by the depth map extractor 120 in determining a 3D position of the cola can. Will be provided.

The object synthesizing unit 140 synthesizes the 3D object with the background screen such that the 3D object is located at the 3D position determined by the 3D object position determining unit 130. That is, an advertisement object (for example, cola can) is synthesized on a screen photographed in 2D or 3D (for example, a general street screen), and then synthesized into an image for showing to the viewer. Accordingly, the object synthesizing unit 140 performs a function of synthesizing the three-dimensional object of which the three-dimensional position is determined by the three-dimensional object position determining unit 130 with the received or stored background screen.

3 is a flowchart illustrating an image display method of a smart TV system displaying a 3D image using a depth map according to an exemplary embodiment of the present invention.

As shown in FIG. 3, the smart TV 100 extracts a depth map from the received 3D broadcast image (S310). The extraction method of the depth map is as described above.

Next, the position of the 3D object in consideration of the extracted depth map is determined (S320).

Although not shown in the drawings, if the position of the viewer is to be considered by considering the position of the viewer further, in determining the position of the 3D object in step S320 in consideration of the position of the viewer identified with the extracted depth map 3 Determine the location of the dimension object.

Next, the 3D object is synthesized (S330), and the synthesized object is displayed (S340).

Accordingly, when the viewer of the smart TV uses a service that synthesizes and displays a synthesized object such as augmented reality and a 3D broadcast image, the viewer may enjoy a natural screen.

In addition, the 3D image display method using the depth map according to an embodiment of the present invention can be used not only for 3D broadcasting but also for synthesis and display technology of general 2D / 3D image and graphic objects, and synthesis of various real-world and virtual objects. It can be used for applications, games, advertising, and the like.

While the invention has been described in terms of the preferred embodiments, it is to be understood that the invention is not limited to the disclosed embodiments, but, on the contrary, is intended to cover various modifications and equivalent arrangements included within the spirit and scope of the invention. Accordingly, the appended claims are intended to embrace all such modifications and variations as fall within the true spirit of the invention.

Claims (9)

An image display device for displaying a personalized three-dimensional image according to the viewer position,
A depth map extractor extracting a 3D depth map for screen synthesis;
A viewer location checking unit for checking a viewer location;
An object position determiner that determines a position of an object in consideration of the viewer's position and a background screen through the 3D depth map;
An object synthesizing unit synthesizing the object to be shown to the viewer with the background screen according to the determined object position; And
Display unit for displaying the synthesized object to the viewer
And a display device. The method of claim 1, wherein the depth map extractor,
And a depth map extracted from the stereo image received by the image display device. The method of claim 1, wherein the depth map extractor,
And extracting the depth image as a depth map when the image display apparatus receives one image and a depth image. delete delete A method of displaying a personalized 3D image according to a viewer's position in a 3D image display device,
Extracting a 3D depth map for screen synthesis;
Identifying a viewer's location;
Determining a location of an object in consideration of the viewer's location and a background screen through the 3D depth map;
Synthesizing an object to be shown to the viewer with the background screen according to the determined object position; And
And displaying the synthesized object to the viewer.
delete An image display device for displaying a personalized three-dimensional image according to the viewer position,
A depth map extractor extracting a 3D depth map for screen synthesis;
A viewer location checking unit for checking a viewer location;
An object position determiner that determines a position of an object in consideration of the viewer's position and a background screen through the 3D depth map;
An object synthesizing unit synthesizing the object to be shown to the viewer with the background screen according to the determined object position; And
Including a display unit for displaying a synthesized object to the viewer,
Wherein the viewer positioning unit identifies the viewer's face by recognizing the viewer's face through a mounted camera.
In image display device.
An image display system for displaying a personalized three-dimensional image according to the viewer's position in conjunction with the three-dimensional glasses,
Three-dimensional glasses equipped with a light sensor;
A depth map extractor extracting a 3D depth map for screen synthesis;
A viewer location checking unit for checking a viewer location;
An object position determiner that determines a position of an object in consideration of the viewer's position and a background screen through the 3D depth map;
An object synthesizing unit synthesizing the object to be shown to the viewer with the background screen according to the determined object position; And
Including a display unit for displaying a synthesized object to the viewer,
Wherein the viewer positioning unit is to identify the position of the three-dimensional glasses and the viewer by recognizing the light emitted from the light emitting sensor of the three-dimensional border through the light-emitting recognition sensor mounted
Video display system.

Images