WO2020101197A1

WO2020101197A1 - Method and system for sharing augmented reality content

Info

Publication number: WO2020101197A1
Application number: PCT/KR2019/013848
Authority: WO
Inventors: 서석은; 정연운; 최필균
Original assignee: 유엔젤주식회사
Priority date: 2018-11-15
Filing date: 2019-10-22
Publication date: 2020-05-22

Abstract

The present invention relates to a method and a system for sharing augmented reality content. The method according to the present invention comprises: a step in which a first user terminal and a second user terminal recognize a reference object positioned in an actual space in which augmented reality content is reproduced; a step of obtaining, at a reference point of a local coordinate system of the first user terminal, a first reference coordinate corresponding to the position of the reference object; a step of obtaining, at a reference point of a local coordinate system of the second user terminal, a second reference coordinate corresponding to the position of the reference object; a step of obtaining a coordinate for sharing which is made by converting the coordinate of an object to be shared with respect to the first reference coordinate, in the local coordinate system of the first user terminal; and a step in which the second user terminal places the object to be shared at a coordinate which is obtained by applying the second reference coordinate to the coordinate for sharing transmitted from the first user terminal.

Description

Augmented reality content sharing method and system

The present invention relates to an augmented reality content sharing method and system.

Augmented Reality (AR) is a field of virtual reality that is a computer graphic technique that synthesizes virtual objects or information in a real environment to look like objects existing in the original environment. It is also referred to as Mixed Reality (MR) because it shows the virtual world with additional information in real time in one image.

However, as disclosed in Korean Registered Patent No. 1,788,248, when a large number of users gather in a real space that reproduces augmented reality content and share the augmented reality content using their own user terminal, the world coordinate system in that space is shared. When each of the augmented reality content was applied in a non-existing state, there was a problem that the augmented reality content could be seen at different locations.

The present invention has been devised in consideration of the above situation, and the technical problem of the present invention is to augmented reality that allows a shared object, which is augmented reality content, to be placed in the same location in a real space, and can be synchronized to a direction in which the object to be shared is directed. It is to provide a content sharing method and system.

The method for sharing augmented reality content according to the present invention for solving the above technical problem is a step in which a first user terminal and a second user terminal recognize a reference object located in a real space reproducing augmented reality content, and the first user terminal Obtaining a first reference coordinate corresponding to a location of the reference object from a reference point of the local coordinate system, obtaining a second reference coordinate corresponding to a location of the reference object from a reference point of the local coordinate system of the second user terminal, the Obtaining coordinates for sharing the coordinates of the object to be shared in the local coordinate system of the first user terminal based on the first reference coordinate, and the second user terminal is configured to share coordinates transmitted from the first user terminal. And disposing the shared target object at coordinates obtained by applying the second reference coordinates.

The method includes obtaining a first reference rotation value corresponding to the direction of the reference object in the local coordinate system of the first user terminal, and a second corresponding to the direction of the reference object in the local coordinate system of the second user terminal. The method may further include obtaining a reference rotation value.

The step of obtaining the coordinates for sharing may include obtaining coordinates to which the first reference rotation value is applied to coordinates of an object to be shared in the local coordinate system of the first user terminal, and to coordinates to which the first reference rotation value is applied. And obtaining the shared coordinates by subtracting the first reference coordinates.

The step of arranging the object to be shared may include obtaining coordinates to which the second reference rotation value is applied to the shared coordinates in the local coordinate system of the second user terminal, and to coordinates to which the second reference rotation value is applied. And disposing the shared object at coordinates obtained by adding second reference coordinates.

The first user terminal and the second user terminal recognizing the reference object located in the real space reproducing the augmented reality content may include recognizing an object of a predetermined shape, and indicating an object of the recognized predetermined shape. The method may include displaying on the screen, and confirming that the recognized object of the predetermined type is a reference object from the user.

The augmented reality content sharing system according to the present invention for solving the above technical problem, a first user terminal recognizing a reference object located in a real space reproducing augmented reality content, and a second user terminal recognizing the reference object It includes.

The first user terminal obtains first reference coordinates corresponding to the location of the reference object at a reference point of the local coordinate system of the first user terminal, and obtains coordinates of an object to be shared in the local coordinate system of the first user terminal. The shared coordinates converted based on the 1 reference coordinate are obtained, and the second user terminal obtains a second reference coordinate corresponding to the location of the reference object at a reference point of the local coordinate system of the second user terminal, and the first The object to be shared is disposed at coordinates obtained by applying the second reference coordinates to the sharing coordinates transmitted from the user terminal.

According to another embodiment of the present invention, there is provided a computer-readable recording medium containing a program for performing augmented reality content sharing method.

The program corresponds to a command set recognized by a first user terminal and a second user terminal in a reference object located in a real space reproducing augmented reality content, and corresponds to the location of the reference object at a reference point of the local coordinate system of the first user terminal A set of instructions for obtaining a first reference coordinate, a set of instructions for obtaining a second reference coordinate corresponding to the location of the reference object at a reference point of the local coordinate system of the second user terminal, an object to be shared in the local coordinate system of the first user terminal A command set for obtaining coordinates for sharing obtained by converting the coordinates of the first reference coordinates based on the first reference coordinates, and the second user terminal is obtained by applying the second reference coordinates to the shared coordinates transmitted from the first user terminal A set of instructions for arranging the object to be shared may be included in the coordinates.

According to the present invention, it is to provide a method and system for sharing augmented reality content that allows an object to be shared, which is augmented reality content, to be placed in the same location in a real space, and can be synchronized in a direction toward the object to be shared.

1 is a block diagram showing the configuration of an augmented reality content sharing system according to an embodiment of the present invention.

2 illustrates that a target object, which is augmented reality content, is displayed at different locations for each user terminal.

3 is a flowchart provided to describe the operation of the augmented reality content sharing system according to an embodiment of the present invention.

FIG. 4 illustrates that an object to be shared, which is augmented reality content, is displayed at the same location for each user terminal in the augmented reality content sharing system according to an embodiment of the present invention.

5 is a view provided to explain matters to be considered for the direction of a reference object according to another embodiment of the present invention.

6 is a flowchart provided to describe the operation of the augmented reality content sharing system according to another embodiment of the present invention.

Hereinafter, embodiments of the present invention will be described in detail with reference to the accompanying drawings. The configuration of the present invention and the effect of the action will be clearly understood through the following detailed description.

Referring to FIG. 1, the augmented reality content sharing system according to the present invention may include a plurality of

user terminals

100a, 100b, ..., 100n capable of sharing augmented reality content. And it may include a server 200 capable of providing augmented reality content to a plurality of user terminals (100a, 100b, ..., 100n).

The plurality of

user terminals

100a, 100b, ..., 100n and the server 200 may exchange various information and data through the communication network 10.

The communication network 10 does not cover a local area network (LAN), a metropolitan area network (MAN), a wide area network (WAN), the Internet, etc. It doesn't matter which communication method you use.

A plurality of user terminals (100a, 100b, ..., 100n), such as a smart phone, tablet PC, HMD (Head mounted Display) terminal, such as by synthesizing things or information in the real space to look like objects existing in the original environment It may be implemented as an information communication terminal supporting computer graphics technology.

The server 200 provides contents based on virtual reality and augmented reality, and at the same time, accepts each user's motion based on the scenario and changes the state of the content accordingly to provide a service using augmented reality and virtual reality. Can be. In particular, an object image to be shared corresponding to augmented reality content may be provided according to a request from the

user terminals

100a, 100b, ..., 100n.

When a plurality of

user terminals

100a, 100b, ..., and 100n arrange augmented reality content based on each local coordinate system without sharing the world coordinate system in the real space, in the real space as illustrated in FIG. Augmented reality content will be shown in different locations.

For example, in the real space, the local coordinate system 1 reference point of the user terminal 100a and the local coordinate system 2 reference point of the user terminal 100b have the same coordinates (0,0,0), but different positions in the real space. Therefore, if the user object 100a and the user terminal 100b are respectively applied in their local coordinate system by applying the object coordinates to be shared as (10,0,0), the user of the user terminal 100a (hereinafter referred to as 'user 1') To the user of the user terminal 100b (hereinafter referred to as 'user 2'), the objects to be shared (S1, S2) appear to be arranged at different locations in the actual space.

Hereinafter, a method of synchronizing coordinates of an object to be shared without using a world coordinate system when a plurality of

user terminals

100a, 100b, ..., 100n transmit coordinates of an object to be shared.

3 is a flowchart provided to describe the operation of the augmented reality content sharing system according to an embodiment of the present invention, and FIG. 4 is augmented reality content sharing target in the augmented reality content sharing system according to an embodiment of the present invention It shows that objects are displayed at the same location for each user terminal.

1, 3 and 4, first, the user terminal 100a and the user terminal 100b may recognize a reference object R located in a real space that reproduces augmented reality content (S305). 4 shows a case where the trash can is a reference object (R). The reference object R may be determined by selecting one of objects located in an actual space between users of the user terminal 100a and the user terminal 100b. In addition, a method in which a reference object R is displayed on the screens of the user terminal 100a and the user terminal 100b is set as a reference object. Alternatively, the shape of the reference object may be determined in advance according to an embodiment. In this case, the user terminal 100a and the user terminal 100b recognize a predetermined type of object and display an indication on the screen, so that the user can check whether the object is the reference object.

Next, the user terminal 100a may obtain a first reference coordinate corresponding to the position of the reference object R at a reference point of its local coordinate system (hereinafter referred to as local coordinate system 1) (S310). 3 shows that (10, -10,10) is obtained as the first reference coordinate.

Similarly, the user terminal 100b may obtain a second reference coordinate corresponding to the position of the reference object R at a reference point of its local coordinate system (hereinafter referred to as local coordinate system 2) (S315). 3 shows that (10, -10, -10) is obtained as the second reference coordinate.

In addition, the user terminal 100a may obtain the coordinates for sharing by converting the coordinates of the object S to be shared in the local coordinate system 1 by applying Equation 1 based on the first reference coordinate (S320).

[Equation 1]

Shared coordinates = object coordinates to be shared in the local coordinate system 1-first reference coordinate

Therefore, in the local coordinate system 1, if the coordinate of the object to be shared (S) is (20, -10,10), and the first reference coordinate is (10, -10,10), the coordinate for sharing is (10,0,0) Can be obtained.

Next, the user terminal 100a may transmit the sharing coordinates obtained for the object to be shared S to the user terminal 100b (S325). In step S325, the transmission of the coordinates for sharing may be directly transmitted through short-range communication between the user terminal 100a and the user terminal 100b, or may be transmitted through the server 200. Meanwhile, graphic image information for rendering the object S to be shared on the screen of the user terminal may also be transmitted together with the coordinates for sharing. Alternatively, the coordinates for sharing may be directly shared between user terminals, and graphic image information may be shared through the server 200.

Thereafter, the user terminal 100b may apply the second reference coordinate to the sharing coordinates transmitted from the user terminal 100a to place the object to be shared in the coordinates obtained by Equation 2 (S330).

[Equation 2]

Object coordinates to be shared in local coordinate system 2 = coordinates for sharing + second reference coordinates

Therefore, if the coordinates for sharing are (10,0,0) and the second reference coordinates are (10, -10, -10), the coordinates of the object S to be shared in the local coordinate system 2 are (20, -10,-) 10).

In this way, as illustrated in FIG. 4, the location in the real space of the shared object S viewed by the user 1 and the location in the real space of the shared object S viewed by the user 2 are the same.

5 is a view provided to explain consideration of a direction of a reference object according to another embodiment of the present invention, and FIG. 6 is provided to describe the operation of the augmented reality content sharing system according to another embodiment of the present invention It is a flow chart.

As illustrated in FIG. 5, it is shown that the reference object not only exists in different locations, but also lies in different directions based on each user. For example, in FIG. 5, the reference object R is on the right side in the direction that user 1 is looking at, and the vertex RP is pointing at the 12 o'clock direction. On the other hand, with respect to the direction viewed by the user 2, the reference object R is located on the left side, and the vertex RP faces the middle direction between 12 o'clock and 3 o'clock.

Therefore, it is desirable not only to allow the user 1 and the user 2 to be placed in the same position in the real space, but also to consider the rotation of the object according to the viewing direction.

1 and 6, first, the user terminal 100a and the user terminal 100b may recognize a reference object R located in a real space that reproduces augmented reality content (S605). In FIG. 6, the first user terminal is a user terminal 100a, and the second user terminal is a user terminal 100b.

Next, the user terminal 100a may obtain first reference coordinates corresponding to the position of the reference object R at a reference point of its local coordinate system (hereinafter referred to as 'local coordinate system 1') (S610).

Similarly, the user terminal 100b may obtain a second reference coordinate corresponding to the position of the reference object R at a reference point of its local coordinate system (hereinafter referred to as 'local coordinate system 2') (S615).

Meanwhile, the user terminal 100a may obtain a first reference rotation value corresponding to the direction of the reference object R in the local coordinate system 1 (S620). Step S620 may be performed simultaneously with step S610. Specifically, the direction of the vertex RP in the local coordinate system 1 may be obtained as a first reference rotation value.

Similarly, the user terminal 100b may obtain a second reference rotation value corresponding to the direction of the reference object R in the local coordinate system 2 (S625). Step S625 may be performed simultaneously with step S615. Specifically, the direction of the vertex RP in the local coordinate system 2 can be obtained as the second reference rotation value.

Next, the user terminal 100a obtains the coordinates to which the first reference rotation value is applied to the coordinates of the object to be shared in the local coordinate system 1 (S630), and subtracts the first reference coordinate to the coordinates obtained in step S630 for sharing. Coordinates can be obtained (S635). Steps S630 and S635 may be performed by Equation (3).

[Equation 3]

Shared coordinates = (first reference rotation value * coordinates of the object to be shared in the local coordinate system 1)-first reference coordinate

Converting the object coordinates to be shared to the coordinates for sharing in the local coordinate system 1 by Equation 3 is as follows. In the local coordinate system 1, the shared object is rotated as much as the reference object R is rotated so that the vertex RP is directed in the z-axis direction. Next, the shared object is moved as much as the reference object R is moved to the reference point. Through the process, the coordinates for sharing can be obtained.

Next, the user terminal 100a may transmit or transmit the shared coordinates obtained for the object to be shared to the user terminal 100b (S640).

Subsequently, the user terminal 100b obtains the coordinates to which the second reference rotation value is applied in the local coordinate system 2 to the shared coordinates transmitted from the user terminal 100a (S645), and the second reference coordinates to the coordinates to which the second reference rotation value is applied. The object to be shared can be arranged at the coordinates obtained by adding (S650). Steps S645 and S650 may be performed by Equation (4).

[Equation 4]

Object coordinate to be shared in local coordinate system 2 = (-2nd reference rotation value * sharing coordinate) + 2nd reference coordinate

It is as follows to obtain the coordinates of the object to be shared in the local coordinate system 2 by processing the coordinates for sharing in the local coordinate system 2 by Equation 4. In the local coordinate system 2, the object to be shared is rotated as much as the reference object R is rotated so that the vertex RP is directed in the z-axis direction. Next, the object to be shared is reversed as much as the reference object R is moved to the reference point. Through the process of moving, it is possible to obtain the location of the shared object in the local coordinate system 2.

In this way, the user 1 and the user 2 may not only appear that the object to be shared is in the same position in the real space, but also reflect the direction in which the object to be shared is facing.

The embodiments described above may be implemented by hardware components, software components, and / or combinations of hardware components and software components. For example, the devices, methods, and components described in the embodiments include, for example, processors, controllers, arithmetic logic units (ALUs), digital signal processors (micro signal processors), microcomputers, and field programmable gates (FPGAs). It can be implemented using one or more general purpose computers or special purpose computers, such as arrays, programmable logic units (PLUs), microprocessors, or any other device capable of executing and responding to instructions. The processing device may run an operating system (OS) and one or more software applications running on the operating system. In addition, the processing device may access, store, manipulate, process, and generate data in response to the execution of the software. For convenience of understanding, a processing device may be described as one being used, but a person having ordinary skill in the art, the processing device may include a plurality of processing elements and / or a plurality of types of processing elements. It can be seen that may include. For example, the processing device may include a plurality of processors or a processor and a controller. In addition, other processing configurations, such as parallel processors, are possible.

The software may include a computer program, code, instruction, or a combination of one or more of these, and configure the processing device to operate as desired, or process independently or collectively You can command the device. Software and / or data may be interpreted by a processing device, or to provide instructions or data to a processing device, of any type of machine, component, physical device, virtual equipment, computer storage medium or device. , Or may be permanently or temporarily embodied in the transmitted signal wave. The software may be distributed over networked computer systems, and stored or executed in a distributed manner. Software and data may be stored in one or more computer-readable recording media.

The method according to the embodiment may be implemented in the form of program instructions that can be executed through various computer means and recorded on a computer-readable medium. The computer-readable medium may include program instructions, data files, data structures, or the like alone or in combination. The program instructions recorded in the medium may be specially designed and configured for the embodiments or may be known and usable by those skilled in computer software. Examples of computer-readable recording media include magnetic media such as hard disks, floppy disks, and magnetic tapes, optical media such as CD-ROMs, DVDs, and magnetic media such as floptical disks. -Hardware devices specifically configured to store and execute program instructions such as magneto-optical media, and ROM, RAM, flash memory, and the like. Examples of program instructions include high-level language codes that can be executed by a computer using an interpreter, etc., as well as machine language codes produced by a compiler. The hardware device described above may be configured to operate as one or more software modules to perform the operations of the embodiments, and vice versa.

As described above, although the embodiments have been described with limited drawings, those skilled in the art may apply various technical modifications and variations based on the above. For example, the described techniques are performed in a different order than the described method, and / or the components of the described system, structure, device, circuit, etc. are combined or combined in a different form from the described method, or other components Alternatively, even if substituted or substituted by equivalents, appropriate results can be achieved.

Claims

The first user terminal and the second user terminal recognize the reference object located in the real space to reproduce the augmented reality content,

Obtaining a first reference coordinate corresponding to the location of the reference object from a reference point of the local coordinate system of the first user terminal,

Obtaining a second reference coordinate corresponding to the location of the reference object from a reference point of the local coordinate system of the second user terminal,

Obtaining coordinates for sharing the coordinates of an object to be shared in the local coordinate system of the first user terminal based on the first reference coordinates, and

The second user terminal disposing the sharing target object in coordinates obtained by applying the second reference coordinates to the sharing coordinates transmitted from the first user terminal.

Augmented reality content sharing method comprising a.
In claim 1,

Obtaining a first reference rotation value corresponding to the direction of the reference object in the local coordinate system of the first user terminal, and

Obtaining a second reference rotation value corresponding to the direction of the reference object in the local coordinate system of the second user terminal

Augmented reality content sharing method further comprising.
In claim 2,

The step of obtaining the coordinates for sharing,

Obtaining coordinates to which the first reference rotation value is applied to coordinates of an object to be shared in the local coordinate system of the first user terminal, and

Obtaining the shared coordinates by subtracting the first reference coordinates to coordinates to which the first reference rotation value is applied.

Augmented reality content sharing method comprising a.
In claim 3,

The step of arranging the object to be shared,

Obtaining coordinates to which the second reference rotation value is applied to the shared coordinates in the local coordinate system of the second user terminal, and

Disposing the shared target object in coordinates obtained by adding the second reference coordinates to coordinates to which the second reference rotation value is applied.

Augmented reality content sharing method comprising a.
In claim 1,

The first user terminal and the second user terminal recognize the reference object located in the real space to reproduce the augmented reality content,

Recognizing a predetermined type of object,

Displaying an indication indicating the recognized predetermined type of object on a screen, and

Receiving a confirmation from the user that the recognized predetermined object is a reference object

Augmented reality content sharing method comprising a.
A first user terminal that recognizes a reference object located in a real space that reproduces augmented reality content, and

A second user terminal recognizing the reference object

Including,

The first user terminal obtains first reference coordinates corresponding to the location of the reference object at a reference point of the local coordinate system of the first user terminal, and obtains coordinates of an object to be shared in the local coordinate system of the first user terminal. 1 Calculate the shared coordinates converted based on the reference coordinates,

The second user terminal obtains second reference coordinates corresponding to the location of the reference object from a reference point of the local coordinate system of the second user terminal, and the second reference coordinates to shared coordinates transmitted from the first user terminal. Augmented reality content sharing system to place the object to be shared at the coordinates obtained by applying.
In claim 6,

The first user terminal obtains a first reference rotation value corresponding to the direction of the reference object in the local coordinate system of the first user terminal,

The second user terminal, augmented reality content sharing system for obtaining a second reference rotation value corresponding to the direction of the reference object in the local coordinate system of the second user terminal.
In claim 7,

The first user terminal obtains coordinates to which the first reference rotation value is applied to coordinates of an object to be shared in the local coordinate system of the first user terminal, and the first reference coordinates to coordinates to which the first reference rotation value is applied. Augmented reality content sharing system to obtain the coordinates for sharing by subtracting.
In claim 8,

The second user terminal obtains coordinates to which the second reference rotation value is applied to the shared coordinates in the local coordinate system of the second user terminal, and applies the second reference coordinates to coordinates to which the second reference rotation value is applied. Augmented reality content sharing system for placing the object to be shared in the coordinates obtained by adding.
In claim 6,

The first user terminal and the second user terminal,

An augmented reality content sharing system, each of which recognizes a predetermined type of object, displays an indication indicating the recognized predetermined type of object on a screen, and is confirmed by the user that the recognized predetermined type of object is a reference object.
In a computer-readable recording medium containing a program for performing augmented reality content sharing method,

The above program,

A set of instructions that the first user terminal and the second user terminal recognize a reference object located in a real space that reproduces augmented reality content,

A set of instructions for obtaining a first reference coordinate corresponding to the location of the reference object from a reference point of the local coordinate system of the first user terminal,

A set of instructions for obtaining a second reference coordinate corresponding to the location of the reference object at a reference point of the local coordinate system of the second user terminal,

In the local coordinate system of the first user terminal, a set of commands for obtaining coordinates for sharing obtained by converting coordinates of an object to be shared based on the first reference coordinates, and

The second user terminal is a computer-readable recording medium including a set of instructions for placing the object to be shared at coordinates obtained by applying the second reference coordinates to the shared coordinates transmitted from the first user terminal.