KR101788471B1 - Apparatus and method for displaying augmented reality based on information of lighting - Google Patents

Abstract

An object of the present invention is to provide an object augmented reality display apparatus and method based on an illumination information that provides a realistic image through natural combination of an object and a background by enhancing an object based on the position of traceable illumination and the light distribution information . To this end, the present invention provides a display device comprising: a board for displaying surrounding illumination environment information; A camera for photographing the background of the board and a peripheral background of the board; A control unit for detecting an illumination environment by analyzing a peripheral background image of the board and the board photographed by the camera and generating a virtual structure augmented reality image reflecting the detected illumination environment information; And a display unit for displaying the augmented reality image generated by the control unit. Therefore, the present invention has the advantage of providing a realistic image through natural combination of object and background by enhancing the object based on the position of trackable illumination and the light distribution information.

Description

FIELD OF THE INVENTION [0001] The present invention relates to an object augmented reality display apparatus and method based on illumination information,

The present invention relates to an apparatus and method for displaying augmented reality based on illumination information, and more particularly, to an apparatus and method for displaying an augmented reality based on illumination information, and more particularly, And more particularly, to an apparatus and method for displaying object augmented reality based on illumination information.

Advances in computers and information and communication technologies have made it possible to analyze and process large amounts of data that have been difficult to try in the past, and especially in the field of video image data processing.

Augmented Reality (AR) is a field of Virtual Reality (VR). It is a field of virtual reality (VR). It combines a virtual object or a selected or designated multimedia into a multimedia image of a real environment, It is one of the areas of computer graphics.

Virtual reality (VR) is a high-tech computer graphics that excludes interaction with the real world and processes objects or objects to be exposed in a state where they interact with each other in a pre-built virtual space.

On the other hand, Augmented Reality (AR) is a computer graphic of a concept that complements real world information (reinforcement information) with virtual world information by exposing real environment and virtual objects in a synthesized (fusion) state in real time. But the protagonist is the environment of the real world.

In other words, the Augmented Reality (AR) uses a special interface called computer graphic for the actual or real image of the real world observed and registers the 3D virtual image by registering it, The user can apply the present invention to a situation where the observer can quickly, quickly and easily judge the situation of the information such as the specific area or the actual product by making the actual environment appear more improved and improved image.

Generally, in order to render a realistic composite image in which a virtual object appears to exist in a real image, illumination information present in the surrounding environment is required.

However, a technique of realistically synthesizing a virtual object in consideration of the influence of the illumination information in an actual surrounding environment (e.g., a global illumination model and the like) may require a large amount of calculation.

In addition, most of the conventional enhancement technologies have different illumination environments of objects and backgrounds, and result in unnatural image synthesis. In particular, since outdoor illumination changes in real time, there is a problem that objects can not be rendered by transmitting illumination environment information .

Korean Patent Registration No. 10-0897006 (entitled " Image synthesizer capable of image synthesis in real time and method thereof "

In order to solve these problems, the present invention provides an object augmented reality display device and method based on illumination information, which provides a realistic image through natural combination of object and background by enhancing an object based on the position of trackable illumination and light distribution information And to provide the above objects.

According to an aspect of the present invention, there is provided a display device including a board for displaying surrounding illumination environment information; A camera for photographing the background of the board and a peripheral background of the board; A control unit for detecting an illumination environment by analyzing a peripheral background image of the board and the board photographed by the camera and generating a virtual structure augmented reality image reflecting the detected illumination environment information; And a display unit for displaying the augmented reality image generated by the control unit.

The present invention further includes an input unit for inputting the direction and the number of the lights installed around the board by the user.

The present invention further includes a sensor unit installed on the board for detecting a position of an object disposed on the board.

The board according to the present invention is characterized in that it includes an illumination information detector for forming a shadow according to the direction of the illumination so that the direction information of the illumination is estimated.

Also, the board according to the present invention may further include one or more frame markers indicating at least one of size, shape, coordinates, and orientation information of the board.

Also, the illumination information detecting unit according to the present invention is characterized in that it is plated, painted in the same hue or a certain pattern color.

Further, the illumination information detecting unit according to the present invention may be integrally installed on the board or may be detachably mounted on the board.

Further, the illumination information detecting unit according to the present invention is characterized by being a three-dimensional model having a predetermined shape.

Further, the illumination information detecting unit according to the present invention is characterized by comprising an optical sensor for detecting the brightness and direction of the illumination.

In addition, the illumination information detecting unit according to the present invention is characterized in that at least one or more of the illumination information detecting units are installed at the center of the board or at any position of the board.

In addition, the control unit according to the present invention generates a board image in which the board reconstructs an arbitrary shape and texture by reflecting the detected illumination environment information and preset board setting information, and outputs the generated board image to the display unit.

In addition, the controller estimates the position of an object located on the board using at least one of size, shape, coordinates, and orientation information of the board.

In addition, the controller according to the present invention generates an object image reconstructing the shape and texture of the object located on the board, and outputs the object image to the display unit.

Also, the controller according to the present invention is characterized in that a dynamic image is generated by combining an alpha channel with a board and an object, and the dynamic image is output to the display unit.

According to another aspect of the present invention, there is provided a method of controlling a display device, the method comprising the steps of: a) recognizing an image of a board displaying illumination environment information of a surrounding area photographed by a controller; b) the control unit detects illumination environment information around the board; And c) generating a board image in which the shape and the texture of the board are reconstructed according to the detected illumination environment information, and outputting the generated board image through the display unit.

According to another aspect of the present invention, there is provided a method for displaying an object augmented reality, comprising the steps of: d) recognizing the object on a board and the position of the object; e) generating an object image by reconstructing the shape and texture of the object using the illumination environment information detected in the step b); And f) layering the board image and the object image generated by the controller in steps c) and e), and outputting the result through the display unit.

In addition, the illumination environment information in the step b) according to the present invention is at least one of shading information formed by the illumination information detection unit installed on the board according to the direction of the illumination, or illumination direction information of the user inputted by the user through the input unit .

The present invention has the advantage of providing a realistic image through natural synthesis of objects and background by enhancing objects based on the position of trackable illumination and the light distribution information.

Further, the present invention is advantageous in that an improved augmented reality image can be provided because a user can easily and accurately select illumination information.

In addition, the present invention has an advantage that it can be utilized for developing various games and educational applications by using the advanced augmented reality technology.

1 is a view illustrating an object augmented reality display apparatus based on an illumination information according to the present invention;
2 is a block diagram illustrating an object augmented reality display apparatus based on illumination information according to the present invention.
3 is a cross-sectional view illustrating a board structure of an object augmented reality display apparatus based on illumination information according to the present invention.
4 is a flowchart illustrating an object augmented reality display process based on illumination information according to the present invention.

DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS Hereinafter, a preferred embodiment of an apparatus and method for displaying augmented reality based on illumination information according to the present invention will be described in detail with reference to the accompanying drawings.

FIG. 1 is a diagram illustrating an object augmented reality display apparatus based on an illumination information according to the present invention. FIG. 2 is a block diagram illustrating an object augmented reality display apparatus based on an illumination information according to the present invention. Sectional view illustrating a board structure of an object augmented reality display device based on illumination information according to an embodiment of the present invention.

1 to 3, the illumination information-based object augmented reality display apparatus according to the present invention includes a board 100, a camera 200, a control unit 300, and a display unit 400 And estimates the illumination environment around the board 100 using the camera 200 and the controller 300, and generates a realistic augmented reality image using the estimated illumination environment.

The board 100 is disposed at a predetermined position of the interior space 10 and is arranged at a position corresponding to the illumination direction of the light emitted from the illumination 20 according to the position of the illumination 20 installed in the interior space 10. [ (100) so that shadow information can be generated on the board (100) according to the position of the illumination (20), wherein the illumination information detecting unit 110 are installed.

The illumination information detecting unit 110 may be installed at a center of the board 100 or may be installed at a predetermined position on the periphery of the board 100 such as the edge of the board 100, A three-dimensional model having a predetermined shape such as a sphere, a hemisphere, and a cylinder.

That is, the illumination information detector 110 displays the shade according to the location of the board 100 and the location of the illumination 20 installed in the indoor space 10, thereby displaying the direction of the light.

The illumination information detecting unit 110 may be integrally installed on the board 100 or may be detachably mounted on the board 100.

The illumination information detecting unit 110 may be formed by plating the outer surface with an arbitrary material, painting with the same color, or coloring with a certain pattern color so that arbitrary game information or the like is printed on the board 100 Or an object 140 such as a printing unit 130 or a play horse that displays a route, a background picture, and the like.

In the present embodiment, the illumination information detection unit 110 is configured as a three-dimensional model for estimating the illumination environment. However, the present invention is not limited thereto, and the illumination information detection unit 110 may be an optical sensor capable of detecting and estimating the directionality and brightness of illumination have.

The board 100 may include one or more frame markers 120 (not shown) so as to easily recognize the size, shape, coordinates, and orientation information of the board 100 in an image taken by the controller 300 through the camera 200. [ Is installed.

The frame marker 120 includes general positional information, and the position of the frame marker 120 is represented by, for example, four pieces of vertex information, so that the position of the frame marker 120 can be easily found in the augmented reality image. (120) may be composed of two or three.

The camera 200 outputs RGB signals of red (R), green (G), and blue (B) according to the intensity of light in an image signal having an arbitrary resolution, reads the object information in the form of light, And outputs the RGB signal according to the intensity of the inputted light. The board 100 and the board 100 are connected to each other at a position spaced apart from the board 100 by a predetermined distance, The image of the surrounding background is photographed.

In addition, the camera 200 may be a separate photographing means, or may be a photographing means installed in a portable terminal such as a smart phone.

The control unit 300 detects an illumination environment by analyzing the background image of the board 100 captured by the camera 200 and the background image of the board 100 and displays a virtual reality image of the virtual structure reflecting the detected illumination environment information And may be configured as a smart phone or the like together with the camera 200, or may be configured as a separate computing means.

That is, the control unit 300 recognizes the illumination information detection unit 110 installed on the board 100 by analyzing the image photographed by the camera 200, and detects shadow information formed around the illumination information detection unit 110 By extracting the direction information of the illumination, the illumination environment of the indoor space 10 in which the board 100 is installed can be recognized.

In addition, the controller 300 performs real-time brightness estimation using the automatic exposure information of the camera 200 to detect illuminance information or brightness information of the illuminant 20, Can be extracted.

In addition, the controller 300 may previously store shape information of the illumination information detector 110 installed in the board 100 and positional information of shadows formed in accordance with the illumination direction for quick recognition .

In addition, the controller 300 stores setting information related to the board type, material, and the like, which are preset with respect to the board 100, and stores the setting information of the board, the illumination environment information including the direction and brightness of the detected illumination To generate a virtual board image in which a board to be displayed is reconstructed in a certain form and texture.

The controller 300 detects the size, shape, coordinate and direction information of the board 100 using the frame marker 120 installed on the board 100, 0.0 > 140 < / RTI >

The control unit 300 stores setting information related to the type, material, and the like of the object 140 previously set in association with the object 140, and stores setting information of the stored object 140, Generates a virtual object image in which the shape and texture of the object to be displayed are reconstructed using the illumination environment information including brightness, and displays the generated virtual object image at the position of the estimated object 140. [

The controller 300 detects the position of the object 140 using the sensor unit 600 installed on the board 100 and detects the position of the object 140 disposed on the board 100 It is possible.

Also, the controller 300 may generate a dynamic image by synthesizing alpha channels (layers) with the reconstructed virtual board and virtual objects.

In the present exemplary embodiment, the control unit 300 automatically estimates the illumination environment information using the illumination information detection unit 110 installed in the board 100. However, the present invention is not limited to this, It is possible to reconstruct the virtual board image and the object image by detecting the illumination environment information inputted by the user.

That is, the user inputs the direction of illumination, the number of lights, the brightness information of the illumination, and the like around the board 100 in accordance with the position of the board 100 through the input unit 500, So that it is possible to provide illumination environment information.

The display unit 400 is configured to display an augmented reality image including a virtual board image and an object image generated by the controller 300, and includes a display device such as an LED and an LCD.

Although the camera 200, the control unit 300 and the display unit 400 are described as being separated from each other for convenience of explanation, the present invention is not limited thereto. For example, a smart phone, a tablet PC, PC or the like.

Next, a process of displaying an object augmented reality based on illumination information according to the present invention will be described with reference to Figs. 1 to 4. Fig.

When the board 100 is installed at a predetermined position of the indoor space 10, the controller 300 reads out the frame marker 120 from the image of the board 100 displaying the surrounding environment information photographed by the camera 200 Detects the board 100, and recognizes the basic information set in advance for the board 100 (S100).

When the board recognition in step S100 is completed, the control unit 300 determines whether the illumination environment can be automatically estimated using the illumination information detection unit 110 installed on the board 100 (S110).

As a result of the determination in step S110, if the automatic estimation is possible, the controller 300 detects illumination environment information about the board 100 including the direction of the illumination, the brightness of the illumination, etc. from the image photographed through the camera 200 (S120).

If it is difficult to acquire the shade information using the illumination information detection unit 110 as a result of the determination in step S110, a position or a direction in which the illumination 20 is installed, a direction in which the shade is generated, 20), and detects illumination environment information (S121).

Upon detecting the illumination environment information through the step S120 or S121, the controller 300 detects the shape and material information of the predetermined board, and reflects the detected illumination environment information to detect a direction in which shading occurs, (S130), and outputs the generated virtual board image through the display unit (400).

The control unit 300 recognizes whether the object 140 is positioned at an arbitrary position on the board 100 from the image photographed through the camera 200 in operation S140.

In the step of recognizing the object, the control unit 300 confirms presence or absence of an object corresponding to the photographed image using the shape information of the object 140 previously stored, The position of the object 140 is detected through the position estimation using the unit 600.

If the object 140 is recognized at an arbitrary position in step S140, the controller 300 reflects the illumination environment information detected in step S120 or S121 to detect the direction in which shading occurs, (S150) a reconstructed virtual object image so that a change, a texture, and the like may appear.

If a virtual object image is generated in step S150, the controller 300 generates a dynamic image by layering the virtual board image and the object image generated in steps S130 and S150 (S160) And outputs the dynamic image through the display unit 400.

Therefore, it is possible to provide a realistic image through natural synthesis of object and background by enhancing the object based on the position of traceable illumination and the light distribution information.

While the present invention has been particularly shown and described with reference to exemplary embodiments thereof, it will be understood by those skilled in the art that various changes and modifications may be made without departing from the spirit and scope of the invention as defined in the appended claims. It can be understood that

In the course of the description of the embodiments of the present invention, the thicknesses of the lines and the sizes of the components shown in the drawings may be exaggerated for clarity and convenience of explanation, , Which may vary depending on the intentions or customs of the user, the operator, and the interpretation of such terms should be based on the contents throughout this specification.

10: Indoor space
20: Lighting
100: Board
110:
120: frame marker
130:
140: object
200: camera
300:
400:
500: input unit
600:

Claims (17)

A board (100) for displaying surrounding illumination environment information;
A camera (200) for photographing the background of the board (100) and a background of the board (100);
The illumination environment is detected by analyzing the peripheral background image of the board 100 photographed by the camera 200 and the board 100 and the brightness of the board 100 is detected by reflecting the detected illumination environment information and preset board setting information. A controller 300 for generating a board image reconstructing the shape and texture of the board image; And
And a display unit (400) for displaying a board image reconstructing the shape and texture of the board (100) generated by the controller (300).
The method according to claim 1,
Further comprising an input unit (500) for allowing a user to input directions and the number of lights installed around the board (100).
The method according to claim 1,
Further comprising a sensor unit (600) installed on the board (100) and detecting a position of an object (140) disposed on the board (100).
4. The method according to any one of claims 1 to 3,
Wherein the board (100) includes an illumination information detection unit (110) for forming a shadow according to a direction of illumination so that direction information of the illumination is estimated.
5. The method of claim 4,
Wherein the board (100) further comprises one or more frame markers (120) indicating at least one of size, shape, coordinates, and orientation information of the board.
6. The method of claim 5,
Wherein the illumination information detection unit (110) is plated with a color of the same color or a predetermined pattern.
The method according to claim 6,
Wherein the illumination information detection unit (110) is integrally installed on the board (100) or is detachably installed on the board (100).
6. The method of claim 5,
Wherein the illumination information detection unit (110) is a stereoscopic model having a predetermined shape.
6. The method of claim 5,
Wherein the illumination information detection unit (110) comprises an optical sensor for detecting the brightness and direction of the illumination.
6. The method of claim 5,
Wherein at least one or more of the illumination information detecting unit 110 is installed at a center of the board 100 or at an arbitrary position of the board 100.
delete 6. The method of claim 5,
The controller 300 estimates the position of the object 140 located on the board 100 using at least one of size, shape, coordinates, and orientation information of the board 100. [ Based object augmented reality display device.
13. The method of claim 12,
The control unit 300 generates an object image reconstructing the shape and texture of the object 140 located on the board 100 and outputs the object image to the display unit 400. [ Device.
6. The method of claim 5,
Wherein the control unit (300) synthesizes an alpha channel with the board (100) and the object (140) to generate a dynamic image and outputs the dynamic image to the display unit (400).
a) recognizing an image of the board 100 displaying the surrounding illumination environment information taken by the camera 200 by the control unit 300 and basic information of the predetermined board 100;
b) the control unit (300) detects illumination environment information around the board (100); And
c) generating a board image by reconstructing the shape and texture of the board 100 according to the detected illumination environment information, and outputting the generated board image through the display unit 400; Augmented reality display method.
16. The method of claim 15,
The method of displaying an object augmented reality includes: d) the controller 300 recognizing the position of the object 140 and the object 140 on the board 100;
e) generating an object image by reconstructing the shape and texture of the object 140 using the illumination environment information detected by the controller 300 in step b); And
f) outputting the combined image of the board image and the object image generated in the step c) and the object image generated by the control unit 300 through the display unit 400, Way.
17. The method of claim 16,
The illumination environment information in the step b) may include at least one of shading information formed by the illumination information detection unit 110 installed on the board 100 in accordance with the illumination direction or illumination direction information of a user input by the user through the input unit 500 And displaying the object augmented reality based on the illumination information.

Images