KR20190006584A - Method and apparatus for displaying augmented reality object based on geometry recognition - Google Patents

Abstract

The present invention relates to a method to display an augmented reality object based on geographic information recognition, and to a device thereof. More specifically, according to an embodiment of the present invention, the method includes: a step of receiving an image about an actual space through a camera; a step of extracting pixel data information of the image about the actual space inputted through the camera; a step of recognizing marker information based on the extracted pixel data information, while determining an augmented reality object corresponding to the recognized marker information by exploring a pre-stored database; a step of analyzing geographic information about the actual space based on the extracted pixel data information, while converting the determined augmented reality object in accordance with the analyzed geographic information; and a step of combining the converted augmented reality object with the image about the actual space and displaying the same.

Description

[0001] METHOD AND APPARATUS FOR DISPLAYING AUGMENTED REALITY OBJECT BASED ON GEOMETRY RECOGNITION [0002]

The present invention relates to a method and apparatus for displaying an augmented reality object based on the recognition of terrain information, and more particularly, to a method and apparatus for recognizing a vertical and horizontal terrain information of an actual environment by applying the recognized augmented reality information And more particularly to a method and apparatus for displaying information.

2. Description of the Related Art [0002] In recent years, various image technologies have been developed and applied due to the rapid development of the image field. In particular, technologies for playing and viewing a virtual reality or augmented reality image through a computer or a user terminal are emerging.

Here, the term 'virtual reality' (VR) refers to a simulation technique that allows a user to experience an environment such as reality through a three-dimensional sight (Sight), which is not an actual reality, and augmented reality ) Refers to a simulation technique in which a virtual object is superimposed on a real world viewed by the user. The augmented reality is often called a mixed reality (MR) because of its feature of combining virtual objects having additional information in real time in real time into a single image.

The virtual reality and the augmented reality described above seem to be similar to each other, but they can be clearly distinguished according to whether the subject is virtual or real.

When we compare computer games with computer games, virtual reality game is a character that substitutes me in a virtual space against a virtual enemy. However, augmented reality game is a form in which I confront a virtual enemy in a real space do. Accordingly, when augmented reality technology is used, a virtual character or the like that does not actually exist in reality can be seen as actually existing on the screen.

Therefore, unlike the virtual reality technology configured by only a virtual environment, the augmented reality technology provides a user with an image in which a real environment and a virtual object are mixed with each other. Therefore, as a user, There is a merit that a better reality and additional information are provided when compared with a virtual reality composed of only the environment of the environment.

The past or present augmented reality is realized by a method of detecting an augmented reality marker together with an actual image from an image photographed by a camera, and synthesizing a virtual object according to the detected marker with the actual image and outputting the synthesized image.

That is, past or current augmented reality technology recognizes a marker for each frame of an image input from a camera to display a virtual object on the screen, and displays the size, position, and position of the virtual object so as to correspond to the type and position of the marker, And the virtual object is synthesized with the image at the calculated position.

However, in the case of the past or current marker-based augmented reality technology, it is difficult to recognize the terrain information including the depth or position of the marker three-dimensionally regardless of whether the marker is placed on the horizontal surface or the vertical wall surface .

Accordingly, the past or present augmented reality technology has an additional problem in that accurate spatial matching between actual objects and virtual objects in the image is not performed, resulting in lowering of realism and deterioration of interest of the user.

Therefore, a method for solving the above problems is required.

It is an object of the present invention to provide a method of displaying an augmented reality object based on topographic information recognition in order to solve the above problems.

It is another object of the present invention to provide a method of recognizing a terrain information recognition of an actual image and synthesizing and displaying the augmented reality object based on the recognized terrain information.

It is still another object of the present invention to provide an apparatus for realizing the above method.

The technical problems to be solved by the present invention are not limited to the technical problems and other technical problems which are not mentioned can be understood by those skilled in the art from the following description.

According to an aspect of the present invention, there is provided a method for displaying an augmented reality object based on topographic information recognition according to an embodiment of the present invention includes receiving an image of an actual space through a camera, Extracting pixel data information of an image with respect to a space, searching for a pre-stored database of an augmented reality object corresponding to the recognized marker information by recognizing marker information based on the extracted pixel data information, Converting the determined augmented reality object to correspond to the analyzed topographic information, and performing the step of converting the augmented reality object into the real space, based on the extracted pixel data information, And displaying the synthesized image on the display unit Can.

In the method of displaying an augmented reality object based on the recognition of the terrain information according to an embodiment of the present invention, the pixel data information may include RGB data of an image of the real space.

The method of displaying an augmented reality object based on the recognition of terrain information according to an embodiment of the present invention is characterized in that the terrain information analysis includes a step of analyzing a location of a pixel with respect to each of at least one object in the image, Determining a vanishing point in the image based on a position of each of the at least one object in the space, applying rotation about the image based on the determined vertical axis of the vanishing point, Analyzing the terrain information including the horizontal information, the vertical information, and the plane information of the actual space based on the coordinate change amount.

The method of displaying an augmented reality object based on the recognition of the terrain information according to an embodiment of the present invention may include the steps of converting the determined augmented reality object corresponding to the analyzed terrain information, And converting at least one of a position, a size, and a degree of rotation to be displayed by combining the determined augmented reality object according to the horizontal information, the vertical information, and the plane information of the actual space.

According to an aspect of the present invention, there is provided an augmented reality apparatus for displaying an augmented reality object based on geographical information recognition according to an embodiment of the present invention includes a camera unit for receiving an image of an actual space, A marker recognition unit for recognizing marker information based on the extracted pixel data information and searching for a database in which augmented reality objects corresponding to the recognized marker information are stored in advance, An image processing unit that performs geospatial information analysis on the real space based on the extracted pixel data information and converts the determined augmented reality object according to the analyzed topographical information, A display for compositing and displaying the image on the actual space It may be made, including the chin.

According to the present invention as described above, the following effects can be obtained. However, effects obtained through the present invention are not limited thereto.

First, according to the present invention, a method of displaying an augmented reality object based on the recognition of terrain information can be provided.

According to another aspect of the present invention, there is provided a method of recognizing a terrain information recognition of an actual image, and synthesizing and displaying the augmented reality object based on the recognized terrain information.

Third, according to the present invention, an apparatus for realizing the above-described method can be provided.

The effects obtained by the present invention are not limited to the above-mentioned effects, and other effects not mentioned can be clearly understood by those skilled in the art from the following description will be.

1 is a diagram illustrating a configuration of an augmented reality apparatus to which the present invention can be applied.
2 is a diagram illustrating a process of displaying an augmented reality object based on geographical information recognition according to an embodiment of the present invention.
FIG. 3 is another diagram for explaining a process of performing the terrain information recognition by the augmented reality device according to an embodiment of the present invention.
4 is a diagram illustrating an example of displaying an augmented reality object based on topographic information recognition according to an embodiment of the present invention.
FIG. 5 is another diagram illustrating an example of displaying an augmented reality object based on topographic information recognition according to an embodiment of the present invention. Referring to FIG.

Hereinafter, preferred embodiments according to the present invention will be described in detail with reference to the accompanying drawings. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS The following detailed description, together with the accompanying drawings, is intended to illustrate exemplary embodiments of the invention and is not intended to represent the only embodiments in which the invention may be practiced.

It is to be understood that both the foregoing general description and the following detailed description of the present invention are exemplary and explanatory and are intended to provide further explanation of the invention as claimed. Only.

In some instances, well-known structures and devices may be omitted or may be shown in block diagram form, centering on the core functionality of each structure and device, to avoid obscuring the concepts of the present invention. In the following description, the same components are denoted by the same reference numerals throughout the specification.

Throughout the specification, when an element is referred to as "comprising" or " including ", it is meant that the element does not exclude other elements, do.

Also, the term "part" in the description means a unit for processing at least one function or operation, which may be implemented by hardware, software, or a combination of hardware and software. Furthermore, the terms " a or an, "" one," and similar related terms, unless the context clearly dictates otherwise or clearly contradicted by context, As used herein.

In addition, specific terms used in embodiments of the present invention are provided to facilitate understanding of the present invention. Unless otherwise defined, all terms used herein, including technical or scientific terms, And have the same meaning as commonly understood by those of ordinary skill in the art. The use of such a specific term may be changed to other forms without departing from the spirit of the present invention.

Hereinafter, preferred embodiments according to the present invention will be described in detail with reference to the accompanying drawings. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS The following detailed description, together with the accompanying drawings, is intended to illustrate exemplary embodiments of the invention and is not intended to represent the only embodiments in which the invention may be practiced.

1 is a diagram illustrating a configuration of an augmented reality apparatus to which the present invention can be applied.

1, the augmented reality device to which the present invention can be applied includes a camera unit 100, a control unit 200, a marker recognition unit 300, an image processing unit 400, and a display unit 500 .

However, the augmented reality apparatus described below includes at least one of the components shown in FIG. 1, but is not limited thereto.

For example, the augmented reality device 1000 may be a mobile client on which an 'augmented reality application' or an 'augmented reality program' is executed. For example, the augmented reality device 1000 may be an ordinary augmented reality application using a mobile communication network such as WCDMA or LTE Such as a smart phone, a smart pad, a virtual reality (VR), an augmented reality (AR) device, a personal computer (PC), a notebook computer, a tablet personal computer, a terminal device, a mobile phone, Terminal, an MS (Mobile Station), an MSS (Mobile Subscriber Station), an SS (Subscriber Station), an AMS (Advanced Mobile Station), a WT (Wireless Terminal), a MTC (Machine- Machine, and a D2D (Device-to-Device) device.

At this time, the augmented reality application or the augmented reality program is distributed by the operator of the augmented reality service for the purpose of providing various augmented reality services. For example, the augmented reality application or the augmented reality program is distributed through a normal app store server (not shown) can do. An application or a program refers to an application program which is operated only by a mobile device using a development language provided by a mobile OS manufacturer. In the case of the application or the program, various functions can be implemented by supporting the hardware functions provided by the device.

However, the augmented reality device does not necessarily have to be configured as a mobile client, and may be a general PC (Personal Computer) having a camera and a display or any equivalent computing means.

The camera unit 100 may receive an image of an actual space and may be a conventional camera including a mono camera. In order to capture an actual space for realizing an augmented reality object (or augmented reality content) Which can be interpreted as a concept including a device having various image taking functions developed or developed in the future.

The augmented reality device of the present invention may include a marker recognition unit 300 and an image processing unit 400. The marker recognition unit 300 recognizes a marker from an image input through the camera unit 100 More specifically, it is possible to recognize the marker information based on the pixel data information in the image input by the camera unit 100, and to store the augmented reality object corresponding to the recognized marker information in advance in the database It can be determined by searching.

At this time, the database may previously store unique recognition information of each marker for the augmented reality and / or information about the augmented reality object corresponding thereto.

In addition, the image processing unit 400 performs topographic information analysis on the actual space based on the pixel data information in the image input by the camera unit 100, and performs the enhancement determined through the marker recognition unit 300 The real object can be converted corresponding to the analyzed terrain information.

The augmented reality device of the present invention may display an augmented reality interface and related information (for example, an image and / or augmented reality object for an actual space), a UI (User Interface), a GUI (Graphic User Interface) The display unit 500 may be included inside or outside the apparatus.

The display unit 500 may be a liquid crystal display, a thin film transistor-liquid crystal display, an organic light-emitting diode, a flexible display, a three-dimensional display (3D display), or may include two or more display units depending on the implementation. For example, the terminal 200 may be provided with an external display unit and an internal display unit at the same time.

The augmented reality apparatus may further include a controller 200. The controller may perform processing on various data or signals or perform control on the augmented reality interface and / or object based on the data or the signal .

In the present invention, the controller 200 includes: It is possible to extract pixel data information of an image of the actual space input through the camera unit 100. The camera unit 100, the marker recognition unit 300, the image processing unit 400, It is possible to control each of them so that each of the units 500 can perform the correct operation.

The controller may also be referred to as a processor, a controller, a microcontroller, a microprocessor, a microcomputer, etc., and the controller may be hardware or firmware, , Software, or a combination thereof.

The augmented reality device may further include a memory unit, and the controller 200 may store data or signals in the memory unit. The memory unit may be connected to the controller 200 and may be connected to an operating system, , Applications, and general files.

Meanwhile, the augmented reality device including various devices including the display unit 500 according to an embodiment of the present invention may further include various input / output units for providing various audiovisual stimuli to the user.

More specifically, the augmented reality device may include a general / standard image and signal input device such as a user input unit (e.g., a camera, a keyboard, a mouse, a joystick, a touch sensor, etc.) Based sensor capable of physically inputting a predetermined signal as a motion by using an input device such as a gyro change, an acceleration change, or the like, or may be connected or coupled via wire / wireless technology.

In addition, in the case of the display unit 500, it is possible to provide visualization of various augmented reality objects to the user, and to provide augmented reality objects to the user as well as to provide users with realistic audiovisual stimulation. (E.g., a display device, a sound device, etc.) to be developed or to be developed in the future.

In the present invention, the augmented reality device may further include a physically-based sensor, which can measure / recognize position, movement, change, etc. with respect to at least one or more objects in a user or an image, A sensor, a gyro sensor, a gravity sensor, a magnetic sensor, or a temperature sensor.

At this time, the acceleration sensor can measure how much the object is inclined about the ground. The acceleration sensor is widely used as a micro-electromechanical systems (MEMS) technology, which converts an acceleration change in one direction into an electric signal. The acceleration sensor is a gravity sensor . ≪ / RTI >

Meanwhile, when the beacon (BLE) of 4.0 or later is used together with the Bluetooth 4.2 capable of active sensing, the acceleration sensor can detect a specific movement of a certain object at a certain speed So that accurate time measurement in relation to movement can also be possible.

On the other hand, the gyro sensor is a sensor capable of measuring angular velocity or rotational force, and uses the principle of the gyroscope. In this case, the gyroscope is a device for automatically rotating the center of rotation when an external rotational force is applied to the center rotator. The restoring force of the gyroscope matches the quasi-rotational force from the outside, so that the angular velocity can be detected.

The augmented reality device can perform the augmented reality function without interworking with at least one external device by driving the augmented reality application or the augmented reality program. Although not shown in Fig. 1, It is also possible to provide some functions through interoperation with the server. For example, various virtual images used in the augmented reality may have a large amount of data or be changed in real time depending on the type and content of the provided image. In this case, As shown in FIG.

2 is a diagram illustrating a process of displaying an augmented reality object based on geographical information recognition according to an embodiment of the present invention.

Referring to FIG. 2, in order to display an augmented reality object based on the recognition of the terrain information according to an embodiment of the present invention, an image of the actual space may first be received through the camera unit.

Thereafter, the augmented reality device can extract pixel data information of an image of an actual space inputted through the camera, wherein the pixel data information includes RGB data information of the image of the actual space Lt; / RTI >

On the other hand, if the pixel data information of the image for the actual space is extracted, the augmented reality device (or marker recognition unit) recognizes the marker information based on the extracted pixel data information, The augmented reality object can be searched and determined in advance.

The determination may be made by searching a minutiae of the recognized marker information, searching the previously stored database based on the minutiae, and searching an augmented reality object corresponding to the recognized marker information.

According to the above search, when the augmented reality object corresponding to the recognized marker information is determined, the augmented reality device performs the terrain information analysis on the actual space based on the extracted pixel data information, The real object can be converted corresponding to the analyzed terrain information.

According to an embodiment of the present invention, for the topographic information analysis, as shown in FIG. 3 (a), the position of a pixel with respect to each of at least one or more objects in the image with respect to the real space, The vanishing point in the image can be determined based on the camera center point based on the position of each of the at least one object on the image.

In this case, a position where each of at least one or more objects in the image is located in the image may be considered with reference to the horizon line to determine a vanishing point in the image.

3 (b), when the rotation (for example, counterclockwise) is applied to the image based on the determined vertical axis of the vanishing point, the coordinates of the at least one object in the image As shown in FIG. 3 (c), when the coordinate change amount is used, for each of at least one object in the image, the terrain information including the horizontal information, the vertical information, Can be analyzed.

Meanwhile, if the pixel data information is extracted from the image of the actual space inputted through the camera unit and the topographic information analysis is performed on the actual space based on the extracted pixel data information, the augmented reality device (or the image processing unit) The augmented reality object to be synthesized on the Korean image can be converted corresponding to the analyzed terrain information.

At this time, the transformation may include at least one of a position, a size, and a degree of rotation of the augmented reality object to be displayed in the image of the actual space and displayed on the basis of the analyzed topographic information (i.e., And plane information, and the like).

Meanwhile, the augmented reality device can synthesize and display the converted augmented reality object on the image of the actual space.

Therefore, according to the present invention, it is possible for the augmented reality device to recognize the terrain information recognition of the actual image, and to display the composite augmented reality object based on the recognized terrain information, There is an effect that it is possible to physically express an object.

For example, when the augmented reality object is a clock or a navigation, the clock is generally displayed on a vertical wall surface. In the case of the navigation, it can be easily recognized by a user that the object is displayed on a horizontal plane. It is possible to display each of the augmented reality objects corresponding to the clock or the navigation on the vertical wall surface or the horizontal plane based on the recognized terrain information.

In the present invention, as described above, the augmented reality object is displayed by reflecting the topographic information. In this case, only the pixel data information of the image input by the camera without using an additional sensor such as a gravity sensor or a magnetic field sensor It is possible to recognize the terrain information by using the terrain information.

4 and 5 are views illustrating an example of displaying an augmented reality object based on the recognition of terrain information according to an embodiment of the present invention.

In the present invention, when the augmented reality device recognizes the topographic information on the actual space by using the pixel data information of the image input by the camera, a separate line and / or point (vertical, horizontal, flat, slope, area, etc.) can be visually expressed by using the map,

Further, in the present invention, the augmented reality device can display an additional augmented reality object with or without the augmented reality object registered in the marker, and the augmented reality object can also display the augmented reality object (or the terrain analysis result) Can be displayed.

That is, when a plurality of augmented reality objects having a shape of a spherical body are displayed as shown in Fig. 5 (a), a plurality of augmented reality objects having a shape of the spherical body Can be displayed as the augmented reality object of the virtual reality object falls directly in the depth direction of the screen.

However, when the terrain recognition result (or terrain analysis result) is reflected on the additional augmented reality object, as shown in Fig. 5 (b), a plurality of augmented reality objects in the shape of the spherical augmented object (E.g., skip, etc.) correspondingly, and it is possible to represent a more realistic augmented reality object according to an embodiment of the present invention.

On the other hand, the above-described method can be implemented in a general-purpose digital computer that can be created as a program that can be executed in a computer and operates the program using a computer-readable medium. Further, the structure of the data used in the above-described method can be recorded on a computer-readable medium through various means. Computer readable media for storing executable computer code for carrying out the various methods of the present invention include magnetic storage media (e.g., ROM, floppy disks, hard disks, etc.), optical readable media (e.g., Etc.).

It will be understood by those skilled in the art that various changes in form and details may be made therein without departing from the spirit and scope of the invention as defined by the appended claims. Therefore, the disclosed methods should be considered in an illustrative rather than a restrictive sense. It is intended that the scope of the invention be defined by the claims appended hereto and their equivalents falling within the scope of the invention.

The method and apparatus for displaying an augmented reality object based on the recognition of terrain information according to the present invention can be applied to various methods for displaying an augmented reality object.

100:
200:
300: marker recognition unit
400:
500:

Claims (1)

A method for displaying an augmented reality object based on topographic information recognition in an augmented reality device,
Receiving an image of an actual space through a camera;
Extracting pixel data information of an image of an actual space inputted through the camera;
Recognizing marker information based on the extracted pixel data information and searching for a database in which augmented reality objects corresponding to the recognized marker information are stored in advance;
Performing geospatial information analysis on the actual space based on the extracted pixel data information, and converting the determined augmented reality object corresponding to the analyzed geospatial information; And
And synthesizing and displaying the converted augmented reality object on an image of the real space,
The step of converting the determined augmented reality object according to the analyzed terrain information
Determining a vanishing point in the image based on the camera center point based on a position of a pixel for each of at least one object in the image with respect to the actual space and a position of each of the at least one object in the real space step;
Recognizing a rotation of the image based on the determined vertical axis of the vanishing point and measuring a coordinate change amount for each of at least one object in the image; And
And analyzing the terrain information including the horizontal information, the vertical information, and the plane information for the actual space for each of at least one or more objects in the image using the coordinate change amount, .

Images