KR102344817B1 - Method of compensating the orientation of augmented reality using more than two markers, recording medium electronic device - Google Patents

Abstract

A method of compensating the orientation of augmented reality using more than two markers according to an embodiment of the present invention includes the steps of: recognizing the real coordinate of a first marker given to a first object in real space and the real coordinate of a second marker given to a second object adjacent to the first object, respectively; generating virtual coordinates in a virtual space with respect to the real coordinates of the first marker and the second marker; calculating a real space direction vector between two real coordinates and a virtual space direction vector between two virtual coordinates; calculating an angle between the real space direction vector and the virtual space direction vector to calculate a coordinate error between the real space and the virtual space; and generating a rotation angle for correcting the coordinate error and then correcting the direction of the virtual space direction vector with the generated rotation angle.

Description

Augmented reality orientation correction method using two or more markers, a recording medium storing a program for implementing the same, and an electronic device executing the same

The present invention relates to an augmented reality orientation correction method using two or more markers, a recording medium storing a program for implementing the same, and an electronic device executing the same.

In the existing augmented reality system, the compass function of the location sensor built into the smart device is used or the method of aligning the orientation of the virtual world by acquiring GPS coordinates is used.

However, since both data have a large error, if the orientation is adjusted using this, a phenomenon occurs that the positions of the real world and virtual objects are misaligned. There is a problem that it is difficult to use in the system.

US Patent Publication US 9619942

The problem to be solved by the present invention is to reduce the azimuth error between the real world image and the virtual world corresponding to the image when driving the augmented reality system on a smart device, thereby providing an augmented reality screen with high positioning accuracy. An object of the present invention is to provide an augmented reality orientation correction method using the above markers, a recording medium storing a program for implementing the same, and an electronic device executing the same.

In an augmented reality orientation correction method using two or more markers according to an embodiment of the present invention for solving the above problems, a first marker given to a first object in real space and a second object adjacent to the first object are provided. recognizing the real coordinates of each of the second markers; generating virtual coordinates in a virtual space with respect to the real coordinates of the first marker and the second marker; calculating a real space direction vector between two real coordinates and a virtual space direction vector between two virtual coordinates; checking a coordinate error between the real space and the virtual space by calculating an angle difference between the real space direction vector and the virtual space direction vector; and after generating a rotation angle for correcting the identified coordinate error, correcting the direction of the virtual space direction vector with the generated rotation angle.

In an embodiment, the first marker and the second marker include an object recognition code and x, y, z coordinate information.

In one embodiment, the generating of the virtual coordinates may include the location of the intersection of the mesh generated in the virtual space and the straight line generated to face the location of the marker on the screen with the user's location on the virtual space as the starting point. It is characterized in that it is a step of generating virtual coordinates for an object in space.

In an embodiment, the real space direction vector is calculated as a difference between the real coordinates of the second marker and the real coordinates of the first marker, and the virtual space direction vector is the difference between the virtual coordinates of the second marker and the virtual coordinates of the first marker. It is characterized in that it is calculated by car.

In an embodiment, the real space direction vector and the virtual space direction vector are two-dimensional vectors having X-axis and Y-axis values.

In one embodiment, the generating of the rotation angle for correcting the identified coordinate error is characterized in that the step of generating the rotation angle θ using Equation 2 below.

[Equation 2]

Here, Vr is a real direction vector, Vv is a virtual space direction vector, Xr and Yr are the x and y values of Vr, respectively, and Xv and Yv represent the x and y values of Vv, respectively. The angle θ indicates a counterclockwise direction when it is (+) and a clockwise direction when it is (-).

The recording medium for solving the above problem is a recording medium in which a program for implementing the augmented reality orientation correction method using two or more markers according to any one of claims 1, 2, and 4 to 6 is stored. do.

When the augmented reality orientation correction method using two or more markers according to an embodiment of the present invention is used, when a virtual object corresponding to the coordinates is drawn on the real world image in the augmented reality system implementing the real coordinate space, the existing technology When using , there is an advantage that it can be expressed with higher positioning accuracy.

1 is an exemplary diagram illustrating a virtual space provided by the Unity engine.
2 is an exemplary diagram of an AR Foundation framework.
3 is a block diagram of a program for executing the augmented reality orientation correction method using two or more markers according to an embodiment of the present invention.
4 is a flowchart of an augmented reality orientation correction method using two or more markers according to an embodiment of the present invention.
5 is a live-action view illustrating a state in which the target area (marker) in real space is recognized by the electronic device executing the augmented reality orientation correction program using two or more markers according to an embodiment of the present invention.
6 is an exemplary view of virtual reality in which TM information of an object displayed in a virtual space displayed in the electronic device shown in FIG. 5 is displayed.

Since the present invention can have various changes and can have various forms, specific embodiments are illustrated in the drawings and described in detail in the text. However, this is not intended to limit the present invention to the specific disclosed form, it should be understood to include all modifications, equivalents and substitutes included in the spirit and scope of the present invention.

Terms such as first, second, etc. may be used to describe various elements, but the elements should not be limited by the terms. The above terms are used only for the purpose of distinguishing one component from another. For example, without departing from the scope of the present invention, a first component may be referred to as a second component, and similarly, a second component may also be referred to as a first component.

The terms used in the present application are only used to describe specific embodiments, and are not intended to limit the present invention. The singular expression includes the plural expression unless the context clearly dictates otherwise. In the present application, terms such as “comprise” or “have” are intended to designate that a feature, number, step, operation, component, part, or combination thereof described in the specification is present, and includes one or more other features or It should be understood that the existence or addition of numbers, steps, operations, components, parts, or combinations thereof does not preclude the possibility of addition.

Unless defined otherwise, all terms used herein, including technical or scientific terms, have the same meaning as commonly understood by one of ordinary skill in the art to which this invention belongs.

Terms such as those defined in commonly used dictionaries should be interpreted as having a meaning consistent with the meaning in the context of the related art, and should not be interpreted in an ideal or excessively formal meaning unless explicitly defined in the present application. does not

Hereinafter, an augmented reality orientation correction method using two or more markers according to an embodiment of the present invention will be described in more detail with reference to the accompanying drawings.

First, in the augmented reality orientation correction method using two or more markers according to an embodiment of the present invention, when implementing an augmented reality system driven on a smart device, two or more markers are used as a means of matching the orientation of the real world and the orientation of the virtual world. This is a method for correcting the azimuth accuracy of the virtual world using

The augmented reality technology described in the present invention is a technology that superimposes a virtual object every frame on the image of the real world coming through a video camera, and the effect of making the object feel as if it actually exists in the real world. technology that provides

The augmented reality system referred to in the present invention is a system implemented using a framework called ARFoundation provided by the Unity game engine and runs on a smart device equipped with Android OS (see FIG. 2 ).

ARFoundation expresses augmented reality by floating a three-dimensional virtual coordinate space on the device screen and outputting a video camera image captured in real time to the background. At this time, the initial coordinate axis direction of the virtual space is adjusted based on the direction the smart device looks at at the moment the system is driven.

1 is an exemplary diagram illustrating a virtual space provided by the Unity engine, and FIG. 2 is an exemplary diagram of the AR Foundation architecture.

Referring to FIG. 1 , a red arrow indicates a horizontal axis of the plane, a blue arrow indicates a vertical axis of the plane, and a green arrow indicates a height.

In ARFoundation, at the same time as the augmented reality system is driven, the direction of the height axis is aligned with the real world by using the gravity sensor of the smart device.

However, since the direction required for the horizontal and vertical axes of the plane varies depending on the purpose of the user or application, the horizontal axis and the front direction are aligned with the vertical axis for the right direction of the device at the moment the system is operated.

This means that if the orientation of the device does not directly face the north of the real world, the horizontal and vertical axes of the virtual space may not coincide with the east and north of the real world. Therefore, in order to express coordinates and directions that coincide with the real world in augmented reality, a process of matching the orientations of the virtual space and the real space is separately required.

Accordingly, in the present invention, as a preliminary preparation step for aligning the orientation, a process of attaching a marker or QR code to a target area (object) in the real world and storing the marker or QR information in a database is required.

After attaching the QR code to the target area where the augmented reality will be used and fixing it so that it does not move, the coordinates of the attached location are measured and stored in the DB together with the contents indicated by the QR code. A schematic view of the DB table can be configured as shown in Table 1 below.

[Table 1]

Here, ID is a unique number given to each row in the DB, Code is the content (number or string) contained in the QR code itself, and X, Y, Z are the measured x, y, and z coordinates (TM coordinates). do.

When the augmented reality system is activated, the first step of recognizing the space in the real world should be performed. ARFoundation provides a technology called Plane Detection, which makes it possible to detect horizontal and vertical planes from camera images.

Using this technology, it is possible to detect road floors or building walls in the real world, and create a mesh that mimics the shapes of floors and walls detected in virtual space. When the mesh generation around the marker or QR code to be recognized is completed, it moves on to the marker or QR code recognition step of this application in earnest.

For reference, the augmented reality orientation correction program using two or more markers introduced in the present invention includes the recognition unit 120 , the virtual coordinate generation unit 130 , the direction vector calculation unit 140 , the coordinate error check unit 150 , and the coordinates. It may be configured as an error correction unit 160 .

The recognition unit 120 performs a function of recognizing the real coordinates of each of a first marker provided to a first object in real space and a second marker provided to a second object adjacent to the first object.

The virtual coordinate generating unit 130 performs a function of generating virtual coordinates in a virtual space with respect to the real coordinates of the first marker and the second marker.

The direction vector calculator 140 calculates a real space direction vector between two real coordinates and a virtual space direction vector between two virtual coordinates.

The coordinate error check unit 150 calculates an angular difference between the real space direction vector and the virtual space direction vector to check the coordinate error between the real space and the virtual space.

The coordinate error correcting unit 160 generates a rotation angle for correcting the identified coordinate error, and then performs a function of correcting the direction of the virtual space direction vector with the generated rotation angle.

The description of the detailed functions of each component of the augmented reality orientation correction program using the two or more markers described above will be described in more detail through the augmented reality orientation correction method using two or more markers to be described later.

4 is a flowchart illustrating an augmented reality orientation correction method using two or more markers according to an embodiment of the present invention.

Referring to FIG. 4 , in the augmented reality orientation correction method S700 using two or more markers according to an embodiment of the present invention, first, a first marker provided to a first object in real space and a second adjacent to the first object The real coordinates of each of the second markers assigned to the object are recognized (S710).

Here, the first marker and the second marker include an identification code and x, y, z coordinate information.

Thereafter, virtual coordinates in the virtual space for the real coordinates of the first marker and the second marker are generated (S720). Here, in the step of generating the virtual coordinates in the virtual space (S720), the position of the intersection point where the mesh created in the virtual space intersects the straight line created to face the position of the marker on the screen with the user position in the virtual space as a starting point. and generating virtual coordinates for the object in the virtual space.

Here, a mesh refers to a mass of 3D objects that can exist in virtual space, and is an object that can visualize a 3D shape on a screen and perform physical simulations such as collision detection.

Thereafter, a real space direction vector between the two real coordinates and a virtual space direction vector between the two virtual coordinates are calculated ( S730 ).

For reference, the first recognized first marker is used for aligning the orientation, and the second marker is used for aligning the orientation and positioning of the object.

If the real world coordinates obtained by recognizing each marker are R1 and R2, respectively, and the coordinates in the virtual space are V1 and V2, the real direction vector Vr and the virtual space direction vector Vv can be obtained by the following Equation 1.

[Equation 1]

At this time, since the direction of the height axis is already aligned with the real world, the z-coordinates of R1, R2, V2, and V1 are excluded from the calculation. Therefore, Vr and Vv take the form of a two-dimensional vector having x and y values.

Next, a coordinate error between the real space and the virtual space is checked by calculating the angular difference between the real space direction vector and the virtual space direction vector (S740).

Here, the coordinate error means the degree of misalignment between the real space direction vector and the virtual space direction vector.

That is, in order to correct the erroneous information, the real space direction vector (Vr) is projected into the virtual space, and the virtual space is returned until the projected real space direction vector matches the virtual space direction vector (Vv). You can make it look north of this real world.

Finally, after generating a rotation angle for correcting the identified coordinate error, a process of correcting the direction of the virtual space direction vector using the generated rotation angle (S750) is performed.

The step S750 is a process of correcting the virtual space direction vector based on calculating a rotation angle for aligning the real space direction vector (Vr) with the virtual space direction vector (Vv) in the direction of the virtual space direction vector (Vv). In the process of generating the rotation angle, the rotation angle θ may be generated using Equation 2 below.

[Equation 2]

Here, Xr and Yr are the x and y values of Vr, respectively, and Xv and Yv are the x and y values of Vv, respectively. Angle θ indicates counterclockwise when (+) and clockwise when (-)

Meanwhile, at the moment of recognizing the first marker in the process S710, the values of the real coordinate R1 and the virtual coordinate V1 of the first marker are calculated and stored, and the virtual coordinate of the second marker is recognized the moment the second marker is recognized. After calculating (V2), real coordinates (R2), and rotation angle (θ) values, when all values are determined, the process proceeds to the step of correcting the orientation and outputting the virtual object.

Accordingly, if the space plane is rotated by θ using the virtual coordinate V2 position of the second marker in the virtual space, the orientation of the virtual space and the real space can be matched.

In addition, if the relative positions of virtual objects are calculated based on the real coordinates (R2) coordinates of the real space, and the objects are placed in the calculated positions based on the virtual coordinates (V2) coordinates of the virtual space, the real world coordinates and orientation are reflected. Augmented reality systems can be implemented.

Factors that can cause errors in calculation results include measurement errors of the QR code attachment position and errors due to plane detection accuracy.

However, even considering the error, it was confirmed that it showed higher accuracy than the method using the device compass and the method using GPS coordinates.

For reference, the present invention has given an example using two markers to more conveniently explain the augmented reality orientation correction method using two or more markers, but is not limited to two, and three, four or more markers are used. It can be used to calculate the relative positions of virtual objects based on the real coordinates of the real space, and place the objects at the calculated positions based on the virtual coordinates of the virtual space to reflect real world coordinates and orientations. That is, it is to be specified that the correction method presented in the present invention is only an example.

Therefore, using the augmented reality orientation correction method using two or more markers according to an embodiment of the present invention, when the virtual object corresponding to the coordinates on the real world image in the augmented reality system for obtaining the real coordinate space is like that, the existing technology It has the advantage of showing higher positioning accuracy than the case of using it.

Meanwhile, the method according to an embodiment of the present invention may be implemented in the form of program instructions that can be executed through various computer means and recorded in a computer-readable medium. The computer-readable medium may include program instructions, data files, data structures, etc. alone or in combination.

The program instructions recorded on the medium may be specially designed and configured for the present invention, or may be known and available to those skilled in the art of computer software. Examples of computer-readable recording media include magnetic media such as hard disks, floppy disks and magnetic tapes, optical media such as CD-ROMs and DVDs, and magnetic media such as floppy disks. - Includes magneto-optical media, and hardware devices specially configured to store and execute program instructions, such as ROM, RAM, flash memory, and the like. Examples of program instructions include not only machine code generated by a compiler, but also high-level language code that can be executed by a computer using an interpreter or the like. The hardware devices described above may be configured to operate as one or more software modules to carry out the operations of the present invention, and vice versa.

The invention has been described above for the purpose of method steps presenting the performance of specific functions and their relationships. The boundaries and order of these functional components and method steps have been arbitrarily defined herein for convenience of description.

Alternative boundaries and orders may be defined so long as the specific functions and relationships are properly performed. Any such alternative boundaries and orders are therefore within the scope and spirit of the claimed invention. Additionally, boundaries of these functional components have been arbitrarily defined for convenience of description. Alternative boundaries can be defined as long as any important functions are properly performed. Likewise, flowchart blocks may also be arbitrarily defined herein to represent any significant functionality. For extended use, the flowchart block boundaries and order may have been defined and still perform some important function. Alternative definitions of both functional components and flowchart blocks and sequences are therefore within the scope and spirit of the claimed subject matter.

The invention may also have been described, at least in part, in terms of one or more embodiments. Embodiments of the present invention are used herein to represent the present invention, aspects thereof, features thereof, concepts thereof, and/or examples thereof. A physical embodiment of an apparatus, article of manufacture, machine, and/or process embodying the present invention may include one or more aspects, features, concepts, examples, etc. described with reference to one or more embodiments described herein. may include Moreover, throughout the drawings, embodiments may incorporate the same or similarly named functions, steps, modules, etc., which may use the same or different reference numbers, as such, the functions, The steps, modules, etc. may be the same or similar functions, steps, modules, etc. or different ones.

In the above, the present invention has been described in detail with reference to the embodiments, but those of ordinary skill in the art to which the present invention pertains may make various substitutions, additions and modifications within the scope not departing from the technical spirit described above. Of course, it should be understood that such modified embodiments also fall within the protection scope of the present invention as defined by the appended claims below.

100: Augmented reality orientation correction program using two or more markers
120: recognition unit
130: virtual coordinate generator
140: direction vector calculation unit
150: coordinate error check unit
160: coordinate error correction unit

Claims (8)

recognizing the real coordinates of each of a first marker provided to a first object in real space and a second marker provided to a second object adjacent to the first object;
generating virtual coordinates in a virtual space with respect to the real coordinates of the first marker and the second marker;
calculating a real space direction vector between two real coordinates and a virtual space direction vector between two virtual coordinates;
checking a coordinate error between the real space and the virtual space by calculating an angle difference between the real space direction vector and the virtual space direction vector; and
After generating a rotation angle for correcting the identified coordinate error, correcting the direction of the virtual space direction vector with the generated rotation angle;
The step of generating the virtual coordinates is
Two or more steps of generating virtual coordinates for an object in the virtual space at the location of the intersection of the mesh created in the virtual space and the straight line created to point to the position of the marker on the screen with the user's position in the virtual space as the starting point Augmented reality orientation correction method using markers.
According to claim 1,
The first marker and the second marker are
Augmented reality orientation correction method using two or more markers including the object's unique number (ID), recognition code, x, y, z coordinate information.
delete According to claim 1,
The real space direction vector is calculated by the difference between the real coordinates of the second marker and the first marker, and the virtual space direction vector is calculated by the difference between the virtual coordinates of the second marker and the first marker. Augmented reality orientation correction method using markers.
5. The method of claim 4,
The augmented reality orientation correction method using two or more markers, wherein the real space direction vector and the virtual space direction vector are two-dimensional vectors having X-axis and y-axis values.
According to claim 1,
The step of generating a rotation angle for correcting the identified coordinate error is
Augmented reality orientation correction method using two or more markers, characterized in that the step of generating the rotation angle (θ) using Equation 2 below.
[Equation 2]

Here, Vr is a real direction vector, Vv is a virtual space direction vector, Xr and Yr are the x and y values of Vr, respectively, and Xv and Yv represent the x and y values of Vv, respectively. The angle θ indicates a counterclockwise direction when it is (+) and a clockwise direction when it is (-).
A recording medium storing a program for implementing the augmented reality orientation correction method using two or more markers according to any one of claims 1, 2, and 4 to 6.
Electronic devices capable of calculation and display for a video camera and 3D graphics to execute a program for implementing the augmented reality orientation correction method using two or more markers according to any one of claims 1, 2, and 4 to 6 device.

Images