WO2021075878A1

WO2021075878A1 - Augmented reality record service provision method and user terminal

Info

Publication number: WO2021075878A1
Application number: PCT/KR2020/014096
Authority: WO
Inventors: 이재만
Original assignee: 주식회사 도넛
Priority date: 2019-10-18
Filing date: 2020-10-15
Publication date: 2021-04-22
Also published as: KR102204721B1

Abstract

Disclosed are an augmented reality record service provision method and user terminal. The augmented reality record service provision method comprises the steps of: generating a terrain feature image by means of a user of the user terminal photographing desired terrain features, and acquiring current location information; generating a marker by extracting a terrain feature, selected by the user, from the generated terrain feature image and combining the extracted terrain feature with the acquired location information; receiving inputted personal content created by the user; generating an augmented reality record by linking the inputted personal content with the generated marker; and setting a viewing condition for the generated augmented reality record.

Description

Augmented Reality Records Service Providing Method and User Terminal

The present invention relates to a method of providing an augmented reality recording service and a user terminal.

In general, augmented reality (AR) refers to a technology field that provides a new paradigm that can be used for human-computer interaction and communication. Here, augmented reality is one of virtual reality that combines the real world that the user sees with the user's eyes and the virtual world with additional information as a single image. Augmented reality, a concept that complements the real world with a virtual world, uses a virtual environment created with computer graphics, but it may be different from general virtual reality in that it is based on the real environment. Computer graphics serve to provide additional information necessary for the real environment. That is, by overlapping the 3D virtual image with the real image viewed by the user, the distinction between the real environment and the virtual screen becomes ambiguous.

In the field of augmented reality, research on technologies necessary to match live images and virtual images so that additional information can be provided to users through a method of displaying new additional information such as virtual graphic images on images input from cameras. Is actively in progress.

On the other hand, people generally have a basic desire to keep a record of that place when they travel to a new place or a place they want to visit. Therefore, people often leave personal marks on the main features of the place through graffiti or damage. As a result, there has been a problem of damaging the environment or public order, such as damage to important natural objects or artificial objects, and the spread of graffiti in public places.

Therefore, there is a need to develop a method that does not harm the environment or public order while satisfying the personal desire to leave a record in a specific place by utilizing augmented reality.

In the present invention, a marker is created using a feature image taken at a user's desired location and geographical location information of the feature, and the augmented reality record is created by linking and storing the content produced by the user to the generated marker. , An augmented reality recording service providing method and a user terminal for displaying a marker on a map using geographic location information to view the augmented reality records created by a user or another user in the future, and outputting the augmented reality records according to the user input. To provide.

According to an aspect of the present invention, a method for providing an augmented reality (AR) recording service performed by a user terminal is disclosed.

A method for providing an augmented reality recorded service according to an embodiment of the present invention includes the steps of generating a feature image by photographing a feature desired by a user of the user terminal, and obtaining current location information, from the generated feature image. Extracting the feature selected by the user, generating a marker by combining the extracted feature and the acquired location information, receiving the personal content created by the user, and receiving the input personal content And creating an augmented reality record by linking with the generated marker, and setting a viewing condition of the generated augmented reality record.

The generating of the marker includes: outputting the feature image, selecting and inputting the feature by dragging an edge or region of the feature from the output feature image, and receiving the feature from the feature image. And extracting the selected feature.

The personal content includes at least one of text, photos, audio, and video including content that the user wants to leave at a corresponding place, and the user creates and inputs the personal content using the user terminal, or The stored personal content is input.

The viewing conditions include a viewing permission range, viewing start and end date and time, and photographing direction information.

The viewing permission range is set to be shared so that the generated augmented reality record is shared with all others, set to a personal use that is not shared with others at all, or set to a friend so that only a designated portion is shared.

The photographing direction information is calculated when the current location information is obtained when photographing the feature, and includes an azimuth, a pitch, and a roll of the user terminal when photographing the feature.

The setting of the viewing condition includes setting augmented reality screen setting information by performing setting of an augmented reality screen based on the generated feature image.

The augmented reality screen setting information includes information on an icon overlapping the image of the terrain feature and information on a special effect produced when the icon appears or the linked personal content is output.

The augmented reality recording service providing method includes the steps of outputting a marker map indicating the location where the marker is generated, and activating the camera as the user moves to the location where the desired marker exists and inputs a scan command, thereby activating the desired marker. Scanning whether or not a feature corresponding to is photographed, when a feature corresponding to the desired marker is detected, outputting an augmented reality screen according to the augmented reality screen setting information, and executed through the output augmented reality screen If the command is input, further comprising the step of outputting the personal content.

The outputting of the marker map includes the step of expanding and displaying the marker map by a preset ratio around the selected location when one of the locations where the marker is generated displayed on the marker map is selected by the user, In the marker map, the location where the marker is created is displayed as an icon having a different color or shape according to the viewing permission range of each marker set for personal, friend, or public use.

The outputting of the marker map includes outputting a notification message and a marker map indicating the location where the desired marker is generated when the user enters a location where the desired marker exists within a preset distance. .

In the scanning step, when a feature corresponding to the desired marker is photographed in a direction according to the photographing direction information corresponding to the desired marker, it is determined that the feature corresponding to the desired marker is detected.

According to another aspect of the present invention, a user terminal providing an augmented reality (AR) recording service is disclosed.

A user terminal according to an embodiment of the present invention includes a memory for storing a command and a processor for executing the command, wherein the command generates a feature image by photographing a feature desired by a user of the user terminal, Acquiring current location information, extracting a feature selected by the user from the generated feature image, and creating a marker by combining the extracted feature and the obtained location information, the An augmented reality record comprising receiving personal content created by a user, creating an augmented reality record by linking the input personal content with the generated marker, and setting viewing conditions of the generated augmented reality record Perform the service provision method.

The augmented reality recording service providing method and the user terminal according to an embodiment of the present invention generate a marker using the geographical feature image and the geographical location information of the feature captured at the user's desired place, and use the generated marker to the user. Create an augmented reality record by connecting and storing the content created by the user, displaying a marker on the map using geographic location information, and outputting the augmented reality record according to the user input, so that a user or another user is created later. Augmented reality records can be viewed.

1 is a flow chart showing a method of providing an augmented reality (AR) recording service according to an embodiment of the present invention.

2 and 3 are diagrams for explaining a method of providing an augmented reality recording service according to an embodiment of the present invention of FIG. 1.

4 is a flow chart showing a method of providing an augmented reality recording service according to another embodiment of the present invention.

5 and 6 are views for explaining a method of providing an augmented reality recording service according to another embodiment of the present invention of FIG. 4.

7 is a diagram schematically illustrating a configuration of a user terminal that performs a method of providing an augmented reality recorded service according to an embodiment of the present invention.

Singular expressions used in the present specification include plural expressions unless the context clearly indicates otherwise. In the present specification, terms such as “consisting of” or “comprising” should not be construed as necessarily including all of the various elements or various steps described in the specification, and some of the elements or some steps It may not be included, or it should be interpreted that it may further include additional components or steps. In addition, terms such as "... unit" and "module" described in the specification mean a unit that processes at least one function or operation, which may be implemented as hardware or software or a combination of hardware and software. .

Hereinafter, various embodiments of the present invention will be described in detail with reference to the accompanying drawings.

1 is a flowchart showing a method of providing an augmented reality (AR) recording service according to an embodiment of the present invention, and FIGS. 2 and 3 are a flowchart illustrating a method of providing an augmented reality recording service according to an embodiment of the present invention of FIG. It is a drawing for explanation.

The method of providing an augmented reality recording service according to an embodiment of the present invention may be implemented in software such as an application and driven in an electronic device such as a PC or a smartphone. Hereinafter, an electronic device that performs a method of providing an augmented reality recording service according to an embodiment of the present invention will be described collectively as a user terminal.

Meanwhile, the user terminal according to an embodiment of the present invention may perform a method of providing an augmented reality recording service in connection with a server. For example, an application that performs a method of providing an augmented reality recording service mounted on a user terminal can be provided from a server, installed, and managed, and requests for large-capacity operations that are difficult to be performed on the user terminal to the server or various data generated. You can also upload it to the server and save it.

Hereinafter, a method of providing an augmented reality recording service according to an embodiment of the present invention will be described with reference to FIG. 1, but with reference to FIGS. 2 and 3.

In step S110, the user terminal generates a feature image by photographing a feature desired by the user according to the user's manipulation. At this time, the user terminal acquires current geographic location information along with photographing a feature. For example, the user terminal may acquire current geographic location information using a GPS module.

In step S120, the user terminal extracts the feature selected by the user from the generated feature image, and creates a marker by combining the extracted feature and the acquired geographic location information.

For example, referring to FIG. 2, the user terminal may output an intro screen as shown in FIG. 2A according to a user's input of an application driving command. The intro screen of FIG. 2A may include a first touch button 10 for inputting a command for performing marker creation and a second touch button 20 for inputting a command for checking the generated marker. have.

Thereafter, when the first touch button 10 is input, the user terminal may activate the camera and then output a camera view screen as shown in FIG. 2B. The camera view screen may include a third touch button 30 that outputs an image input to an activated camera and inputs a command for generating a marker, as shown in FIG. 2B.

Thereafter, when the camera of the user terminal is manipulated to point to the corresponding feature 40 in order to photograph the feature 40 desired by the user, the user terminal captures and outputs a feature image including the feature 40. I can. Subsequently, the user terminal is a method of dragging the edge or area of the feature 40 desired by the user from the output feature image, and after receiving the desired feature 40 selected from the user, the third touch button 30 When this is input, a marker may be generated by extracting the selected feature 40 from the feature image.

In step S130, the user terminal receives the personal content created by the user. For example, personal content may include text, photos, audio, video, and the like, and may include content that the user wants to leave at a corresponding place. The user can create and input personal content using the user terminal, or can input previously stored personal content.

In step S140, the user terminal creates an augmented reality record by linking the input personal content with a marker.

In step S150, the user terminal sets a viewing condition of the generated augmented reality record. Here, the viewing conditions may include a viewing permission range, viewing start date and end date and time, and photographing direction information. For example, the viewing permission range may be set as public so that the created augmented reality record is shared with others, privately not shared with others at all, or set for friends so that only a designated part such as family or friends is shared. have. Further, the photographing direction information may be calculated when the current geographic location information is obtained using a GPS module when capturing a terrain feature. The photographing direction information may include an azimuth angle, a pitch, and a roll of the user terminal when capturing a feature. Here, the user terminal may include a magnetic field sensor, an acceleration sensor, a gyroscope sensor, a motion sensor, etc., and photographing direction information may be calculated using these sensors.

For example, referring to FIG. 3, after generating a marker, the user terminal may output a screen as shown in (a) of FIG. 3 in order to generate an augmented reality record and set a viewing condition. That is, the screen of Fig. 3 (a) includes a plurality of touch buttons 50 for inputting commands for creating and inputting personal content, and a plurality of touch buttons 50 for inputting commands for setting and checking viewing conditions. It may include a button 60.

In this case, the user terminal may set the augmented reality screen setting information by performing the setting of the augmented reality screen based on the generated terrain feature image.

For example, the user terminal may output a screen as shown in (b) of FIG. 3 including a plurality of touch buttons 70 for inputting a command for performing the setting of an augmented reality screen. Through this, the user terminal may set the icon 71 in the form of a ribbon as shown in (b) of FIG. 3 to overlap and output on the feature image according to the user input. In addition, the user terminal may set a special effect produced when an icon appears or a linked personal content is output according to a user input. That is, an icon may be set to appear while approaching from a distance, or an effect such as a written text being written in the air may be produced.

In step S160, the user terminal uploads an augmented reality record including the reading condition to the server according to the setting of the reading condition.

4 is a flowchart illustrating a method of providing an augmented reality recorded service according to another embodiment of the present invention, and FIGS. 5 and 6 are diagrams for explaining a method of providing an augmented reality recorded service according to another embodiment of the present invention of FIG. to be.

The method of providing an augmented reality recording service according to another embodiment of the present invention of FIG. 4 shows a method for a user or another user to view the augmented reality recorded material generated in FIG. 1 and uploaded to the server later.

Hereinafter, a method of providing an augmented reality recording service according to another embodiment of the present invention will be described with reference to FIG. 4, but reference will be made to FIGS. 5 and 6.

In step S410, the user terminal outputs a marker map indicating the location where the marker is generated according to the input of the user's marker search command.

For example, referring to FIG. 5, when a second touch button 20 for inputting a command for checking a marker on the intro screen of FIG. 2 (a) is input, the user terminal A search screen displayed as an icon can be output as shown in (a) of FIG. 5. Here, the search screen may include

touch buttons

80, 90, and 100 for inputting a search command for a marker in which a viewing permission range is set to be for personal use, friend use, or public use. In addition, the user terminal may display a location where a marker is created with an icon having a different color or shape according to the viewing permission range of each marker set for personal, friend, or public use. Thereafter, when the user terminal selects one of the icons displayed on the marker map of FIG. 5(a) from the user, as shown in FIG. 5(b), the marker map is enlarged and displayed by a preset ratio around the selected icon. can do.

Accordingly, the user can check the marker map output to the user terminal and move to the location where the desired marker is generated.

Thereafter, when the user enters a location where a desired marker exists within a preset distance, the user terminal may output a marker map indicating the location at which the marker is generated along with a notification message. In this case, the user terminal may output a marker map enlarged by a preset ratio around an icon indicating the location of the corresponding marker.

In step S420, the user terminal moves to the location where the desired marker exists and activates the camera according to the input of a scan command to scan whether a feature corresponding to the desired marker is photographed. At this time, the user terminal determines that the terrain feature corresponding to the desired marker is detected in the case where the feature corresponding to the desired marker is photographed in the direction according to the photographing direction information among the viewing conditions of the augmented reality record corresponding to the desired marker. I can.

For example, referring to FIG. 6, when the user terminal receives a selection input of an icon selected by the user in FIG. 5 (b), as shown in (b) of FIG. 6, a marker corresponding to the selected icon is activated by activating the camera. It is possible to scan whether a feature is being photographed and output the scanned screen.

In step S430, when a terrain feature corresponding to a marker desired by the user is detected, the user terminal outputs an augmented reality screen according to preset augmented reality screen setting information.

In step S440, when an execution command is input from the user through the output augmented reality screen, the user terminal outputs personal content included in the augmented reality record.

For example, referring to FIG. 6, when a terrain feature corresponding to the marker of the selected icon is detected, the user terminal displays a ribbon-shaped icon 71 according to the augmented reality screen setting information, as shown in FIG. 6(b). It is possible to output an augmented reality screen that is overlapped with the feature image. Subsequently, the user terminal may output the personal content 72 according to the user's input of the execution command.

Referring to FIG. 7, a user terminal according to an embodiment of the present invention includes a processor 710, a memory 720, a camera unit 730, a location information acquisition unit 740, a sensor unit 750, and a display unit 760. ), a communication unit 770 and an interface unit 780.

The processor 710 may be a CPU or semiconductor device that executes processing instructions stored in the memory 720.

The memory 720 may include various types of volatile or nonvolatile storage media. For example, the memory 720 may include ROM, RAM, or the like.

For example, the memory 720 may store instructions for performing a method of providing an augmented reality recording service according to an embodiment of the present invention.

The camera unit 730 is activated in response to an input of a driving command signal to capture an object, thereby obtaining an image. For example, the camera unit 730 may generate a feature image by capturing a feature desired by the user according to a user manipulation.

The location information acquisition unit 740 acquires the current location information of the user terminal. For example, the location information acquisition unit 740 may acquire current geographic location information of the user terminal using a GPS module.

The sensor unit 750 measures the direction of the user terminal. For example, the sensor unit 750 may include a magnetic field sensor, an acceleration sensor, a gyroscope sensor, a motion sensor, and the like, and may calculate photographing direction information of the user terminal using these sensors.

The display unit 760 displays information received or processed by the user terminal. For example, the display unit 760 may be implemented as a liquid crystal display or the like.

In addition, the display unit 760 may be used as a user input unit by applying a touch screen in which sensors for detecting a touch motion form a mutual layer structure.

The communication unit 770 is a means for transmitting and receiving data with other devices through a communication network.

The interface unit 780 may include a network interface and a user interface for accessing a network.

Meanwhile, the constituent elements of the above-described embodiment can be easily grasped from a process perspective. That is, each component can be identified as a respective process. In addition, the process of the above-described embodiment can be easily grasped from the viewpoint of the components of the device.

In addition, the above-described technical contents may be implemented in the form of program instructions that can be executed through various computer means and recorded in a computer-readable medium. The computer-readable medium may include program instructions, data files, data structures, and the like alone or in combination. The program instructions recorded on the medium may be specially designed and configured for the embodiments, or may be known and usable to those skilled in computer software. Examples of computer-readable recording media include magnetic media such as hard disks, floppy disks, and magnetic tapes, optical media such as CD-ROMs and DVDs, and magnetic media such as floptical disks. -A hardware device specially configured to store and execute program instructions such as magneto-optical media, and ROM, RAM, flash memory, and the like. Examples of program instructions include not only machine language codes such as those produced by a compiler, but also high-level language codes that can be executed by a computer using an interpreter or the like. The hardware device may be configured to operate as one or more software modules to perform the operations of the embodiments, and vice versa.

The above-described embodiments of the present invention are disclosed for the purpose of illustration, and those skilled in the art who have ordinary knowledge of the present invention will be able to make various modifications, changes, and additions within the spirit and scope of the present invention. It should be seen as falling within the scope of the following claims.

Claims

In the method of providing an augmented reality (AR) recorded material service performed by a user terminal,

Generating a feature image by photographing a feature desired by a user of the user terminal, and obtaining current location information;

Extracting the feature selected by the user from the generated feature image, and creating a marker by combining the extracted feature and the obtained location information;

Receiving the personal content created by the user;

Creating an augmented reality record by linking the input personal content with the generated marker; And

An augmented reality recording service providing method comprising the step of setting a viewing condition of the generated augmented reality recording.
The method of claim 1,

Generating the marker,

Outputting the feature image;

Selecting and inputting the feature by dragging an edge or region of the feature from the output feature image; And

And extracting the selected and input feature from the feature image.
The method of claim 1,

The personal content includes at least one of text, photo, audio, and video including content that the user wants to leave at a corresponding place,

Wherein the user creates and inputs the personal content using the user terminal or inputs the previously stored personal content.
The method of claim 1,

The viewing conditions are augmented reality recorded service providing method, characterized in that it comprises a viewing permission range, viewing start date and time and end date and time, and photographing direction information.
The method of claim 4,

The viewing permission range is set as public so that the generated augmented reality record is shared with others, is set for personal use that is not shared with others at all, or is set for a friend so that only a designated part is shared. How to provide the service.
The method of claim 4,

The photographing direction information,

It is calculated when acquiring the current location information when photographing the terrain feature,

An augmented reality recorded service providing method comprising an azimuth, a pitch, and a roll of the user terminal when photographing the terrain feature.
The method of claim 4,

The step of setting the viewing conditions,

And setting augmented reality screen setting information by performing setting of an augmented reality screen based on the generated terrain feature image.
The method of claim 7,

The augmented reality screen setting information includes information on an icon overlapping the image of the terrain and information on a special effect produced when the icon appears or the linked personal content is output. .
The method of claim 7,

The augmented reality recording service providing method,

Outputting a marker map indicating the location where the marker is generated;

Activating a camera as the user moves to a location where a desired marker exists and inputs a scan command to scan whether a feature corresponding to the desired marker is photographed;

Outputting an augmented reality screen according to the augmented reality screen setting information when a terrain feature corresponding to the desired marker is detected; And

When an execution command is input through the output augmented reality screen, the method of providing an augmented reality recorded service further comprising the step of outputting the personal content.
The method of claim 9,

The step of outputting the marker map,

When one of the locations in which the markers displayed on the marker map are generated is selected by the user, expanding and displaying the marker map by a preset ratio centered on the selected location,

The marker map is an augmented reality recording service providing method, characterized in that the location where the marker is created is displayed as an icon having a different color or shape according to the viewing permission range of each marker set for personal, friend or public use.
The method of claim 9,

The step of outputting the marker map,

Providing an augmented reality recording service comprising the step of outputting a notification message and a marker map indicating the location where the desired marker is generated when the user enters the location where the desired marker exists within a preset distance Way.
The method of claim 9,

The scanning step,

Providing an augmented reality recording service, characterized in that when a feature corresponding to the desired marker is photographed in a direction according to the photographing direction information corresponding to the desired marker, it is determined that the feature corresponding to the desired marker has been detected. Way.
In a user terminal providing an augmented reality (AR: Augmented Reality) recording service,

A memory for storing an instruction; And

Including a processor that executes the instruction,

The above command is:

Generating a feature image by photographing a feature desired by a user of the user terminal, and obtaining current location information;

Extracting a feature selected by the user from the generated feature image, and creating a marker by combining the extracted feature and the obtained location information;

Receiving the personal content created by the user;

Creating an augmented reality record by linking the input personal content with the generated marker; And

A user terminal, characterized in that performing a method of providing an augmented reality recording service comprising the step of setting a viewing condition of the generated augmented reality recording.