KR101665399B1 - Object generation apparatus and method of based augmented reality using actual measured - Google Patents

Abstract

The present invention relates to an apparatus and method for generating an augmented reality based on an actual measurement so that an indoor 3D drawing and object information can be implemented as an augmented reality based on actual information, A conversion unit; A distance measuring unit for measuring a distance to the object to be measured; An angle measurement unit for measuring an angle of the camera at the time of shooting the measurement object; A distance value measured by the distance measuring unit and an angle value measured by the angle measuring unit are mapped and stored in the storage unit. When the user selects a specific object, A controller for implementing the selected specific object as an augmented reality on a 3D drawing; And a display unit for superimposing the specific object at a corresponding position of the indoor drawing displayed in the camera view according to the control of the controller and displaying the object as an augmented reality.

Description

[0001] The present invention relates to an apparatus and method for generating an augmented reality based on real-

The present invention relates to an apparatus and method for generating an augmented reality based on actual measurement, and more particularly, to an apparatus and method for generating an augmented reality based object through an actual measurement so that an indoor 3D drawing and object information can be implemented as an augmented reality And a method thereof.

Augmented reality is a technology that fuses and replaces virtual objects and information made with computer technology in the real world. It is also referred to as mixed reality (MR) because a virtual world with additional information in real time is added to the real world and displayed as a single image. Since its first use in Boeing's assembly of aircraft wires in 1990, technology research and development has been conducted mainly in the United States and Japan. Since the introduction of smart phones in the mid-2000s, and the introduction of this technology, attention has begun to be drawn.

Virtual reality (VR) technology allows users to be immersed in a virtual environment created by computer graphics, so that a real environment can not be seen. In contrast, augmented reality technology mixes virtual objects in a real environment, Additional realistic information can be provided. For example, if you turn the street around with a smartphone camera, information such as the location of a nearby shop, phone number, and map is displayed as a stereoscopic image, and weather information is displayed when the sky is illuminated.

A conventional technique for providing data based on an augmented reality is disclosed in the following Patent Document 1: Korean Patent Registration No. 10-1360999 (Registered on April 4, 2014); Patent Document 2: Korean Patent Application Publication No. 10-2013-0137077 Patent Document 3: Korean Patent Laid-open Publication No. 10-2014-0122302 (published on Apr. 20, 2014); Patent Document 4: Korean Patent Registration No. 10-1380854 (Apr. 27. Registration), Patent Document 5, Korean Patent Laid-Open Publication No. 10-2013-0025200 (Published on March 31, 2013), and Patent Document 6 (Korean Patent Publication No. 10-2013-0007142) (2013.01.18. Lt; / RTI >

The prior art disclosed in Patent Document 1 provides data obtained by photographing an experimental situation such as a scientific experiment or the like, by overlapping data measured from an experimental sensor with an augmented reality and real time.

The prior art disclosed in Patent Document 2 displays information of an outdoor or indoor language to be operated for a period of time required for service switching at the time of switching between the outdoor augmented reality service and the indoor augmented reality service, thereby providing a more natural service transition Can receive.

The prior art disclosed in Patent Document 3 provides an augmented reality-based bridge information providing and utilization performance evaluation system using a mobile terminal that can easily acquire visible information and invisible information about bridge information.

The prior art disclosed in Patent Document 4 provides an HTML document including program code for representing a 3D POI and a virtual object, and an increased reality content based on a web information structure using a 3D browser engine.

The conventional art disclosed in Patent Document 5 provides augmented reality information giving altitude information for each object scattered in an indoor environment through vector calculation using three-dimensional coordinate values.

The prior art disclosed in Patent Document 6 utilizes matching between a live view through a camera of a mobile terminal and a three-dimensional virtual view corresponding to the live view, And provides the information to the user of the mobile terminal as the augmented reality information.

Korea Registered Patent Registration No. 10-1360999 (Registered on April 4, 2014) Korean Patent Publication No. 10-2013-0137077 (Dec. 16, 2013) Korean Patent Laid-Open Publication No. 10-2014-0122302 (published on October 20, 2014) Korea Registered Patent Registration No. 10-1380854 (Registered on March 27, 2014) Korean Patent Publication No. 10-2013-0025200 (published on March 11, 2013) Korean Patent Publication No. 10-2013-0007142 (published on January 18, 2013)

However, the conventional arts as described above have a disadvantage in that they can be used only in a specific case in which the marker is present although the performance of the augmented reality implementation is excellent as a marker based incremental reality realization technique.

Accordingly, the present invention has been proposed in order to solve all the problems occurring in the conventional augmented reality technique, and it is an object of the present invention to provide an augmented reality based on real 3D images and object information, And an object generating method and apparatus therefor.

According to another aspect of the present invention, there is provided an apparatus for generating an augmented reality based on a real world, comprising: a 3D converter for rendering an indoor image using a drawing program; A distance measuring unit for measuring a distance to the object to be measured; An angle measurement unit for measuring an angle when the image of the measurement object is acquired; A distance value measured by the distance measuring unit and an angle value measured by the angle measuring unit are mapped and stored in the storage unit. When the user selects a specific object, A controller for implementing the selected specific object as an augmented reality on a 3D drawing; And a display unit for superimposing the specific object at a corresponding position of the indoor drawing displayed in the camera view according to the control of the control unit and displaying the superimposed reality as an augmented reality.

The control unit may include a 3D drawing extracting unit for extracting a 3D drawing stored in the storage unit; A designated position coordinate value extracting unit for extracting a coordinate value of a position designated by the user on the 3D drawing; A distance and angle information detector for detecting a distance and an angle of an actual position corresponding to the designated position; A user position calculation unit for detecting a position of the user; And an object generating unit for generating an object at a corresponding position of the camera based on the calculated user position.

The control unit may include a 3D drawing coordinate adjusting unit for re-adjusting the coordinates of the 3D drawing corresponding to the user position calculated by the user position calculating unit and responding to movement of the camera view; And an information input module for receiving operation information of the user.

The user position calculation unit calculates the current user position on the basis of the origin coordinates when the 3D drawing is produced by processing the coordinate values of the 3D drawing designated by the user and the coordinate values of the actual positions corresponding to the 3D drawing specified by the user with a trigonometric function .

According to another aspect of the present invention, there is provided a method of creating an augmented reality-based object through actual measurement, the method comprising: (a) rendering a 3D image of a room image using a drawing program; (b) extracting a coordinate value of a position designated by the user in the 3D drawing; (c) calculating a distance and an angle of an actual indoor position corresponding to the designated position; (d) calculating a position of the current user; (e) outputting a 3D view to the camera view, and creating an object at a corresponding position of the camera, thereby realizing an augmented reality.

In the step (d), the coordinate values of the 3D drawing designated by the user and the coordinate values of the actual positions corresponding to the 3D drawings designated by the user are processed by trigonometric functions to calculate the current user position on the basis of the origin coordinates upon production of the 3D drawing .

The method of generating an augmented reality-based object through actual measurement according to the present invention further comprises the steps of: (f) intervening between the step (d) and the step (e) Calculating a vector and adjusting the coordinates of the 3D drawing using the calculated motion vector to respond to movement of the camera view.

(E1) calculating a distance and an angle to a point in the actual room where the user selects the object, (e2) calculating an object placement coordinate based on the current position, (e3) calculating a corresponding position of the camera on the basis of the object arrangement coordinates, and creating an object as an augmented reality at the calculated position of the camera.

According to the present invention, it is possible to realize more realistic augmented reality by designating indoor 3D drawings and object information as augmented reality on the basis of actual information, designate an object creation position of augmented reality, There is an advantage in that object arrangement can be arranged without drawing information.

FIG. 1 is a block diagram of an augmented reality-based object generation apparatus through actual measurement according to a preferred embodiment of the present invention;
FIG. 2 is a block diagram of an embodiment of the control unit of FIG. 1;
3 is an explanatory diagram for explaining a method of detecting a user position in the present invention,
4 is a flowchart illustrating a method of creating an augmented reality-based object through actual measurement according to a preferred embodiment of the present invention.

Hereinafter, an apparatus and method for generating an augmented reality based on actual measurement according to a preferred embodiment of the present invention will be described in detail with reference to the accompanying drawings.

FIG. 1 is a configuration diagram of an augmented reality-based object generating apparatus through actual measurement according to a preferred embodiment of the present invention.

An augmented reality-based object generation apparatus according to the present invention includes a camera 101, an image processing unit 102, a 3D conversion unit 103, a distance measurement unit 104, an angle measurement unit 105, a control unit 106 A storage unit 107, a display unit 108, an input unit 109, a laser pointer 110, and a frame forming unit 111.

The camera 101 plays a role of acquiring an image of a room or a ship, a plant, an automobile, a media, a game, etc. for realizing an augmented reality, and the image processing unit 102 And processes the acquired image using a normal image processing technique. Here, the objects for realizing the augmented reality are various such as indoor, shipbuilding, plant, automobile, media, game and the like, and the methods for implementing the augmented reality are all the same as described above. .

Here, the camera 101 may use a plurality of cameras or a panoramic camera so as to acquire 3D images.

The distance measuring unit 104 measures the distance to the object to be measured, and an infrared sensor, an ultrasonic sensor, or the like can be used. To measure the distance, infrared or ultrasonic waves are emitted to the measurement object and the reflected signal is processed to measure the distance.

The frame forming unit 111 forms a surface by connecting a plurality of pieces of positional information using the distance measured by the distance measuring unit 104 as positional information, and based on the formed surface information, And forms a frame.

The 3D conversion unit 103 serves to render an actual indoor image processed by the image processing unit 102 or an indoor frame image formed by the frame forming unit 110 using a drawing program.

The angle measuring unit 105 measures the angle of the camera 101 (for example, a photographing angle of a smartphone) at the time of photographing the measurement object. The angle measuring unit 105 may be a geomagnetic sensor, a gyro sensor Sensor, an acceleration sensor, or the like.

The storage unit 107 stores the distance and angle information between the 3D drawing and the indoor position generated by measuring the room, and the display unit 108 is displayed on the camera view under the control of the controller 106 The input unit 109 plays the role of receiving the operation information operated by the user and delivering it to the control unit 106. The control unit 106 receives the operation information,

The control unit 106 maps the distance value measured by the distance measurement unit 104 and the angle value measured by the angle measurement unit 105 with the drawing converted by the 3D conversion unit 103 and outputs the distance value to the storage unit 107 And when the user selects a specific object, the selected specific object is implemented as an augmented reality on a 3D drawing designed based on the actual measurement.

2, the control unit 106 includes a 3D drawing extraction unit 111 for extracting a 3D drawing stored in a storage unit; A designated position coordinate value extraction unit (112) for extracting a coordinate value of a position designated by the user on the 3D drawing; A distance and angle information detecting unit (113) for detecting a distance and an angle of an actual position corresponding to the designated position; A user position calculation unit 114 for detecting the position of the user; An object creating unit (116) for creating an object at a corresponding position of the camera based on the detected user position; A 3D drawing coordinate adjusting unit 115 for re-adjusting the coordinates of the 3D drawing corresponding to the user position calculated by the user position calculating unit 114 and responding to the movement of the camera view; And an information input module 117 for inputting operation information of the user.

The user position calculation unit 114 processes the coordinate values of the 3D drawing designated by the user and the coordinate values of the actual positions corresponding to the 3D drawing designated by the user with the trigonometric function to determine the current user position .

The augmented reality-based object generation apparatus through actual measurement according to the preferred embodiment of the present invention will be described in detail as follows.

First, in order to create an object based on an augmented reality, an actual room for implementing an augmented reality is photographed through a camera 101 to acquire an indoor image, or an indoor frame image corresponding to an actual room is acquired through a frame forming unit 110 . When the camera 101 is used, it is preferable to use a plurality of cameras or a panoramic camera. The frame forming unit 110 obtains a plurality of distance values (position coordinates) with respect to the measurement object (indoor wall surface) measured in conjunction with the distance measuring unit 104, . Then, each of the surfaces is combined into a frame to form a room in the same room as the actual room. Here, the frame means a wall surface structure formed by the position coordinate of the wall structure of the actual room.

The image processing unit 102 processes the indoor image acquired by the camera 101 using a normal image processing technique.

Next, the 3D conversion unit 103 shapes the actual indoor image processed by the image processing unit 102 or the frame formed by the frame formation unit 110 in 3D using a drawing program (CAD program) . The indoor image photographed here may be used as 2D drawings.

Here, the distance measuring unit 104 measures a distance of a measurement object (a specific position in a room) using an infrared sensor, a distance measuring method using an ultrasonic sensor or a laser pointer, or the like. In addition, the angle measuring unit 105 may measure the angle of the camera 101 (e.g., a photographing angle of the smartphone) at the time of capturing the image of the measurement object by using a geomagnetism sensor, a gyro sensor, Measure the angle.

The control unit 103 maps the distance measurement information and the angle measurement information to the 3D drawing information generated through the 3D conversion unit 103 and stores the distance measurement information and the angle measurement information in the storage unit 107.

Then, when a user selects a specific object through the camera of the smartphone, that is, a specific object (for example, furniture, shipbuilding, plant, automobile, media, game, The distance information of the specific object and the camera, and the photographing angle information of the camera are calculated and stored.

Next, when the user selects a 3D drawing for the actual room stored in the storage unit 1070, the 3D drawing extraction unit 111 extracts the 3D drawing stored in the storage unit 107 and displays it on the screen. When the user designates a specific position in the state that the 3D drawing is displayed, the coordinate value for the designated position is extracted by the designated position coordinate value extraction unit 112. Here, Coordinate values for each position are generated together and stored. Therefore, the coordinate value for the designated position can be quickly extracted as soon as the user designates the position.

Then, the distance and the angle of the actual position corresponding to the 3D plane specified by the distance and angle information detecting unit 113 are calculated. For example, when photographing through a camera in a room, distance information between a camera and a specific position in a room and a photographing angle of the camera are calculated.

Next, the user position calculation unit 114 detects the position of the user based on the calculated information. For example, the user position calculation unit 114 processes the coordinate values of the 3D drawing designated by the user and the coordinate values of the actual positions corresponding to the 3D drawing specified by the user with the trigonometric function as shown in FIG. 3, The current user position is calculated based on the reference.

Then, the 3D drawing coordinates adjusting unit 115 adjusts the coordinates of the 3D drawing according to the user position calculated by the user position calculating unit 114 so as to respond to the movement of the camera view. For example, motion vectors are obtained in response to the angle and motion of a user using a geomagnetic sensor, a gyro sensor, and an acceleration sensor of a smartphone, and the coordinate values of the 3D drawing are readjusted based on the motion vector thus calculated, The coordinates to be calculated are calculated. That is, the generation position of the object to be generated as the augmented reality is designated.

Next, the object generating unit 116 generates an object at a corresponding position of the camera based on the detected user position, and implements the object as an augmented reality.

FIG. 4 is a flowchart illustrating a method of generating an augmented reality-based object through actual measurement according to a preferred embodiment of the present invention, wherein S represents a step.

The method for creating an augmented reality-based object through actual measurement according to the present invention includes the steps of: (a) creating 3D views of an image obtained for implementing an augmented reality using a drawing program (S101 to S103); (b) extracting a coordinate value of a position designated by the user in the stored 3D drawing (S104 to S105); (c) calculating a distance and an angle of an actual indoor position corresponding to the designated position (S106); (d) calculating the position of the current user (S107); (e) calculating a motion vector according to the user position calculated in the step (d), and adjusting the coordinates of the 3D drawing using the calculated motion vector to react to movement of the camera view (steps S108 to S109); (f) outputting a 3D view to the camera view and creating an object at a corresponding position of the camera to implement an augmented reality (S110 to S113).

The step (d) S107 processes the coordinate value of the 3D drawing designated by the user and the coordinate value of the actual position corresponding to the 3D drawing designated by the user using the trigonometric function to generate the current user position .

The step (e) includes the steps of: (e1) calculating a distance and an angle to a point in the actual room in which the object selected by the user is to be arranged (S111); (e2) calculating object placement coordinates based on the current position (S112); (e3) calculating a corresponding position of the camera based on the object arrangement coordinates, and creating an object as an augmented reality at the corresponding position of the camera (S113).

A method for generating an augmented reality based object through actual measurement according to a preferred embodiment of the present invention will be described in detail as follows.

First, in step S101, in order to create an object based on an augmented reality, an actual room for implementing an augmented reality is photographed through a camera 101 to acquire an indoor image or correspond to an actual room through a frame forming unit 110 Thereby forming an indoor frame image. When the camera 101 is used, it is preferable to use a plurality of cameras or a panoramic camera. The frame forming unit 110 obtains a plurality of distance values (position coordinates) with respect to the measurement object (indoor wall surface) measured in conjunction with the distance measuring unit 104, . Then, each of the surfaces is combined into a frame to form a room in the same room as the actual room. Here, the frame means a wall surface structure formed by the position coordinate of the wall structure of the actual room.

In step S102, the image processing unit 102 performs image processing on the obtained image using a normal image processing technique. Then, the 3D conversion unit 103 shapes the actual indoor image processed by the image processing unit 102 in 3D using a drawing program (CAD program). The indoor image photographed here may be used as 2D drawings. At this time, the distance measuring unit 104 measures the distance of a measurement object (a specific position in the room) using an infrared sensor, an ultrasonic sensor, or a distance measuring method using a laser pointer, and transmits the distance to the controller 106. In addition, the angle measuring unit 105 measures the angle of the camera 101 (for example, a photographing angle of the smartphone) at the time of photographing the measurement object by using a geomagnetism sensor, a gyro sensor, an acceleration sensor, .

In step S103, the control unit 103 maps the distance measurement information and the angle measurement information to the 3D drawing information generated through the 3D conversion unit 103, and stores the mapping information in the storage unit 107. [

Thereafter, when a user selects a specific object through the camera of the smartphone, that is, when a specific object (e.g. furniture to be placed in the room) is photographed with the smartphone, the distance information of the specific object and the camera, Information and the like are calculated and stored.

Next, when the user selects a 3D drawing for the actual room stored in the storage unit 107, the 3D drawing stored in the storage unit 107 is extracted from the 3D drawing drawing unit 111 and displayed on the screen in step S104.

When the user designates a specific position in a state where the 3D drawing of the room is displayed on the screen, the coordinate value for the designated position is extracted from the designated position coordinate value extraction unit 112 in step S105. In this case, coordinate values for respective positions are generated and stored together when the indoor photographed image is produced as a 3D drawing, so that coordinate values for the designated position can be extracted quickly as soon as the user designates the position.

Then, in step S106, the distance and the angle of the actual position corresponding to the 3D plane designated by the distance and angle information detecting unit 113 are calculated. For example, when photographing through a camera in a room, distance information between a camera and a specific position in a room and a photographing angle of the camera are calculated.

Next, in step S107, the user position calculation unit 114 detects the position of the user on the basis of each of the calculated information. For example, the user position calculation unit 114 processes the coordinate values of the 3D drawing designated by the user and the coordinate values of the actual positions corresponding to the 3D drawing specified by the user with the trigonometric function as shown in FIG. 3, The current user position is calculated based on the reference.

Then, in steps S108 and S109, the 3D drawing coordinate adjusting unit 115 adjusts the coordinates of the 3D drawing corresponding to the user position calculated by the user position calculating unit 114 to react to the movement of the camera view. For example, motion vectors are obtained in response to the angle and motion of a user using a geomagnetic sensor, a gyro sensor, and an acceleration sensor of a smartphone, and the coordinate values of the 3D drawing are readjusted based on the motion vector thus calculated, The coordinates to be calculated are calculated. That is, the generation position of the object to be generated as the augmented reality is designated.

Next, in steps S110 to S113, the object creating unit 116 creates an object at a corresponding position of the camera based on the detected user position, and implements the object as an augmented reality.

For example, a 3D view is output to the camera view in step S110, and a distance and an angle to the point where the object (furniture) is placed are calculated in step S111. Thereafter, in step S112, the object placement coordinates are calculated based on the current position. In step S113, an object (e.g., furniture) is generated at a corresponding position of the camera and the object is displayed through the augmented reality.

Although the present invention has been described in detail with reference to the above embodiments, it is needless to say that the present invention is not limited to the above-described embodiments, and various modifications may be made without departing from the spirit of the present invention.

The present invention is applied to a technique for creating objects such as furniture, ships, plants, media, automobiles, games, and the like through EV technology.

101: Camera
102:
103: 3D conversion section
104: distance measuring unit
105:
106:
107:
108:
109:
110:
111: 3D drawing extracting unit
112: designated position coordinate value extracting unit
113: Distance and angle information detection unit
114: User position calculation unit
115: 3D drawing coordinate adjustment unit
116:
117: Information input module

Claims (8)

A camera for photographing indoor images and images of specific objects;
A 3D conversion unit for converting an image of the room taken by the camera into a 3D drawing using a drawing program;
A distance measuring unit measuring a distance between the camera and a specific position of the room and measuring a distance between the camera and the object;
An angle measurement unit for measuring an angle of the camera when capturing a specific position of the room and measuring an angle of the camera when the object is captured;
A distance value measured by the distance measuring unit and an angle value measured by the angle measuring unit are mapped to the 3D drawing transformed by the 3D transforming unit and stored in the storage unit. When the camera photographs the object, A controller configured to implement the object photographed on the object as an augmented reality; And
And a display unit for superimposing the object at a corresponding position of the indoor drawing displayed in the camera view according to the control of the control unit and displaying the object as an augmented reality,
Wherein the control unit comprises: a 3D drawing extracting unit for extracting a 3D drawing stored in a storage unit; A designated position coordinate value extracting unit for extracting a coordinate value of a position designated by the user on the 3D drawing; A distance and angle information detector for detecting a distance and an angle of an actual position corresponding to the designated position; A user position calculation unit for detecting a position of the user; And an object generator for generating an object at a corresponding position of the camera based on the calculated user position.
delete [2] The apparatus of claim 1, wherein the control unit comprises: a 3D drawing coordinate adjusting unit for re-adjusting the coordinates of the 3D drawing corresponding to the user position calculated by the user position calculating unit and responding to movement of the camera view; And an information input module for receiving user operation information. The apparatus for generating an object based on augmented reality through actual measurement.
[2] The apparatus of claim 1, wherein the user location calculator processes the coordinate value of the 3D drawing designated by the user and the coordinate value of the actual location corresponding to the 3D drawing designated by the user with a trigonometric function, Based on an actual state of the object.
(a) converting an image of a room photographed by a camera and an image of a specific object into a 3D drawing using a drawing program and storing the 3D drawing;
(b) extracting a coordinate value of a position designated by the user in the 3D drawing;
(c) calculating a distance and an angle of the indoor position corresponding to the designated position;
(d) calculating a position of the current user; And
(e) outputting a 3D view to a camera view and creating an object at a corresponding position of the camera to implement an augmented reality,
In the step (d), the coordinate value of the 3D drawing designated by the user and the coordinate value of the actual position corresponding to the 3D drawing designated by the user are processed by the trigonometric function to calculate the current user position on the basis of the origin coordinate upon production of the 3D drawing A method for object creation based on augmented reality based on actual measurement.
delete The method of claim 5, further comprising the steps of: (f) calculating a motion vector according to the user position calculated in step (d) intervening between step (d) and step (e) Further comprising the step of adjusting coordinates of the 3D drawing to respond to the movement of the camera view.
[6] The method of claim 5, wherein the step (e) includes: (e1) calculating a distance and an angle to a point in the actual room where the user selects the object; (e2) calculating object placement coordinates based on the current position; (e3) calculating a corresponding position of the camera on the basis of the object arrangement coordinates, and creating an object as an augmented reality at a corresponding position of the calculated camera.

Images