KR20200137523A - System for Sharing Augmented Reality contents and Driving method thereof - Google Patents

Abstract

The present invention relates to an augmented reality (AR) content sharing system and a driving method thereof. The AR content sharing system comprises: an object loading unit which imports an AR virtual object on the basis of a pre-stored image; an object recognition unit which recognizes an object in the real world; a work creation unit which creates an AR work by augmenting the real world object recognized by the object recognition unit with the virtual object loaded by the loading unit through a predefined method; and an upload unit which uploads the AR work created by the work creation unit to a social network services/sites (SNS) server. The AR content sharing system enables further advancement of AR, which has the advantage of imparting a sense of immersion and realism, and thus has the effect of facilitating the creation of AR works by individuals, and encouraging the creation and sharing of diverse and creative works obtained by applying AR.

Description

TECHNICAL FIELD [System for Sharing Augmented Reality contents and Driving method thereof]

The present invention relates to an augmented reality content sharing system and a driving method thereof, and to an augmented reality content sharing system and a driving method for uploading the created augmented reality content to a social network server and sharing it with others.

In general, augmented reality (AR) technology refers to a technology that superimposes a virtual object made in computer graphics on an object existing in the real world and provides it to a user, and the recent implementation of augmented reality is such as a smartphone or smart glass. As it extends to end point devices, it is a technology in which research and development is being concentrated.

A technology that superimposes virtual objects on the real world seen by the user is called'Augmented Reality'. It is also called Mixed Reality (MR) because it combines the real world with the virtual world with additional information in real time and shows it as a single image.

Augmented reality, a concept that complements the real world with a virtual world, uses a virtual environment made with computer graphics, but the protagonist is the real environment. Computer graphics serve to provide additional information necessary for the real environment.

Unlike virtual reality technology composed of only virtual environments, such augmented reality technology provides an image in which a real environment and a virtual object are mixed, so that a user can see the real environment, providing a better sense of reality and additional information.

Augmented reality technology is used for remote medical diagnosis, broadcasting, architectural design, and manufacturing process management. In particular, with the recent widespread use of smartphones, it has entered the stage of full-scale commercialization, and various products are also being developed in the game and mobile solution industry and education fields.

In addition, various types of augmented reality are provided on the go by using smart glasses, but when the amount of augmented reality is large, unwanted augmented reality is provided, so it is difficult to easily find a desired augmented reality.

Meanwhile, with the recent advent of Internet services such as Facebook, YouTube, and wikipedia, users are not only using the already created content, but also directly creating, posting and sharing multimedia content. As can be seen from the rapid growth in the market shown by Second Life, a real study case, the boundary between virtual and real spaces gradually disappears.

KR 1020190051779 A KR 1020180058895 A

The present invention is derived from such a technical background, and aims to provide an augmented reality content sharing system and a driving method thereof capable of providing an easy and convenient user interface for a user to directly produce an augmented reality work according to his or her preference. There is this.

Furthermore, it is intended to provide an augmented reality content sharing system and a driving method that can share with others by uploading a self-made augmented reality work directly to one's social network and create a publicity effect for one's work.

In addition, an object of the present invention is to provide an augmented reality content sharing system and a driving method thereof that can prevent the occurrence of copyright infringement problem for an augmented reality work created by an individual.

The present invention for achieving the above object includes the following configuration.

That is, the augmented reality content sharing system according to an embodiment of the present invention includes an object loading unit that loads an AR (Augmented Reality) virtual object based on a previously stored image, an object recognition unit that recognizes an object in the real world, and the object. A work generation unit for generating an augmented reality work by augmenting the virtual object loaded by the loading unit to an object in the real world recognized by the recognition unit through a predefined method, and a social network for the augmented reality work generated by the work generation unit It characterized in that it includes an upload unit for uploading to a service (SNS: Social Network Services/Sites) server.

On the other hand, the driving method of the augmented reality content sharing system includes a step in which an object loading unit loads an augmented reality (AR) work object stored in advance, an object recognition unit recognizes an object in the real world, and a work creation unit in the object recognition unit Creating an augmented reality work by augmenting the virtual object loaded by the loading unit to the recognized real world object through a predefined method, and the upload unit providing the augmented reality work generated by the work generation unit through a social network service (SNS :Social Network Services/Sites) It characterized in that it comprises the step of uploading to the server.

According to the present invention, as augmented reality with the advantage of giving immersion and realism further develops, it is possible to facilitate the creation of augmented reality works by individuals and to encourage the creation and sharing of various and creative works applying augmented reality. It works. That is, an effect of providing an augmented reality content sharing system and a driving method thereof capable of providing an easy and convenient user interface when a user directly produces an augmented reality work according to his or her preference is derived.

Furthermore, it is possible to provide an augmented reality content sharing system and a driving method for sharing with others by uploading a directly created augmented reality work on their own social network and producing a promotional effect for their work.

In addition, it is possible to provide an augmented reality content sharing system and a driving method thereof that can prevent the occurrence of a copyright infringement problem for a directly produced augmented reality work.

1 is a block diagram showing the configuration of an augmented reality content sharing system according to an embodiment of the present invention;
2 to 5 are exemplary views for explaining a process of creating a virtual object object in the form of a three-dimensional figure,
6 to 11 are exemplary diagrams for explaining a process of generating an augmented reality work according to an embodiment of the present invention;
12 is an exemplary view showing a state in which an augmented reality work is uploaded according to an embodiment of the present invention;
13 is a flowchart of a method of driving an augmented reality content sharing system according to an embodiment of the present invention.

It should be noted that the technical terms used in the present invention are only used to describe specific embodiments, and are not intended to limit the present invention. In addition, the technical terms used in the present invention should be interpreted as generally understood by those of ordinary skill in the technical field to which the present invention belongs, unless otherwise defined in the present invention, and is excessively comprehensive. It should not be construed as a human meaning or an excessively reduced meaning.

Hereinafter, preferred embodiments of the present invention will be described in detail with reference to the accompanying drawings.

1 is a block diagram showing the configuration of an augmented reality content sharing system according to an embodiment of the present invention.

As can be seen from FIG. 1, the augmented reality content sharing system according to an embodiment of the present invention includes a communication unit 100, a storage unit 110, an object loading unit 120, an object recognition unit 130, and a work creation unit 140. ) And an upload unit 150.

The augmented reality content sharing system according to an embodiment of the present invention may perform data communication with the SNS server 20 to download or upload augmented reality content.

In one embodiment, the augmented reality content sharing system is a personal computer such as a notebook, a desktop, or a laptop, or a personal communication system (PCS), a global system for mobile communications (GSM), a personal digital cellular (PDC), PHS (Personal Handyphone System), PDA (Personal Digital Assistant), IMT (International Mobile Telecommunication)-2000, CDMA (Code Division Multiple Access)-2000, W-CDMA (W-Code Division Multiple Access), Wibro (Wireless Broadband Internet) It may be one of all types of handheld-based wireless communication devices such as a terminal, a smartphone, a smartpad, and a tablet PC, but the present invention is not limited thereto.

That is, the augmented reality content sharing system is interpreted to cover all technical configurations that can be carried by an individual and equipped with a dedicated application for sharing augmented reality content according to an embodiment.

In one aspect, the augmented reality content sharing-only application includes a function of selecting an augmented reality (AR) work to be loaded, a function of recognizing and selecting objects in the real world, and a function of confirming the generated augmented reality work.

In addition, augmented reality is produced for sharing augmented reality content, and various user interfaces required for uploading and downloading it are provided.

The communication unit 100 may include a communication module for supporting wired communication and a mobile communication module for supporting wireless communication. The mobile communication module includes technical standards or communication methods for mobile communication (for example, GSM (Global System for Mobile Communication), CDMA (Code Division Multi Access), CDMA2000 (Code Division Multi Access 2000)), EVDO (Enhanced Voice -Data Optimized or Enhanced Voice-Data Only), WCDMA (Wideband CDMA), HSDPA (High Speed Downlink Packet Access), HSUPA (High Speed Uplink Packet Access), LTE (Long Term Evolution), LTE-A (Long Term Evolution) Advanced), etc.), transmits and receives radio signals to and from the base station, external user terminals, or participant terminals.

The object loading unit 120 loads an augmented reality (AR) virtual object based on a previously stored image. At this time, the augmented reality virtual object loaded may be a part of an image or a character stored in the storage unit 110, and it is interpreted to encompass all types and types of objects that can be used as virtual objects without any limitation. In addition, the object loading unit 120 may download an augmented reality work object from a social network service (SNS) server.

For example, it is possible to download an image uploaded to a friend's SNS as an augmented reality work object. That is, it is possible to create an augmented reality work by creating a reality image uploaded by a friend at a remote location as a virtual object and applying it to the reality image captured by the person. Accordingly, it is possible to produce augmented reality works that can more realistically communicate with others in real time.

The virtual space stored in the storage unit 110 of the augmented reality content sharing system 10 may be generated based on a scanned image of the real space. Here, the scanned image of the real space is scanned by the augmented reality content sharing system 10 itself, received from another augmented reality content sharing system 10, or any external server not shown in the drawing. It may be transmitted from the back, but is not limited thereto.

In one aspect of the present invention, the object loading unit 120 selects a background image to be inserted into at least one of the three-dimensional expanded view-shaped templates among pre-stored images, and includes at least a three-dimensional figure composed of the selected background image. Some forms of virtual object objects can be created.

It is also possible to create a virtual object object by inserting another virtual object object into a virtual object object that is at least a part of the three-dimensional figure generated by the object loading unit 120.

2 to 5 are exemplary diagrams for explaining a process of creating a virtual object object in the form of a three-dimensional figure.

First, the user may set the format of the virtual object object through an augmented reality content sharing application driven by the augmented reality content sharing system 10. It can be selected as a single video format, or a three-dimensional figure format such as a cube shape. However, it is not limited thereto and is interpreted to cover various modifications.

When the 3D model format is selected, different background images may be selected on each side of the 3D model as shown in FIG. 3. In this case, selecting the background image of each side of the three-dimensional figure may be setting a color or pattern or selecting an image previously stored in the storage unit 110 of the augmented reality content sharing system 10.

Then, the virtual object object in the form of a three-dimensional figure can be created as in FIG. 5 through the standby screen as in FIG. 4.

The object recognition unit 130 recognizes an object in the real world. Here, recognition refers to the recognition of information that the attribute extraction unit to be described later can contribute in extracting attributes. Recognition is, for example, the appearance representing attributes such as size, color, material, weight, etc. of real-world objects, and all other attributes that can identify various real-world objects (e.g., barcode, QR code, trademark Etc.). Accordingly, the object recognition unit may include a separate component capable of recognizing each attribute. For example, if you need to recognize the size of a real-world object, a camera with built-in size measurement software, a camera with a built-in color discrimination software if you need to recognize the color of a real-world object, or a material discrimination software if you need to recognize the material of a real-world object. It may include a built-in camera, a scale that can measure weight if it is necessary to recognize the weight of real-world objects.

At this time, a camera or a scale is just a kind of input device for recognizing the property of a real-world object, and in order to extract the property by the property extraction unit to be described later through this, the content recognized by the input device must be converted into data. Accordingly, the object recognition unit may additionally include a configuration such as a converter (not shown) that converts content recognized by a camera or a scale into data, and software built into the aforementioned input device may perform a data conversion function. .

For example, the object recognition unit 130 may recognize a structure of an actual space such as a distance, an object, etc. based on an image captured by a camera and convert it into data.

The work creation unit 140 generates an augmented reality work by augmenting the virtual object loaded by the object loading unit 120 to the real world object recognized by the object recognition unit 130 through a predefined method.

6 to 11 are exemplary diagrams for explaining a process of creating an augmented reality work according to an embodiment of the present invention.

As shown in FIG. 6, a user interface for uploading to SNS through an augmented reality content sharing application is provided, and basic information of an augmented reality work is input. In addition, as shown in FIG. 7, it is possible to immediately check the input contents as augmented reality contents.

In this case, as shown in FIG. 8, image information for generating a virtual object of a three-dimensional figure such as a cube form is additionally input in addition to the virtual object. In one embodiment, it is possible to select and insert the same or different images on a template in the form of a three-dimensional diagram through an augmented reality content sharing application as shown in FIGS. 9 and 10.

In addition, as shown in FIG. 11, it is possible to immediately check the augmented image when a virtual object in the form of a three-dimensional figure is inserted and a virtual object in the form of a single image is inserted.

That is, it is possible to produce while checking in real time the form of an augmented reality work created before uploading to an actual SNS or the like.

In addition, according to an additional aspect of the present invention, the augmented reality work generated by the work creation unit 140 is characterized in that the identification information of the author who created the work is visually displayed.

The work creation unit 140 includes identification information of the author in the generated augmented reality work. In this case, the author may be a user who has a terminal device implementing the augmented reality content sharing system according to an embodiment. That is, it is preferable that the augmented reality work created in the augmented reality content sharing system is implemented in a technical configuration that cannot be separated or deleted by others other than the user who has the augmented reality content sharing system.

For example, the work creation unit 140 may receive a character capable of displaying an author or select a signature image.

It is possible to prevent the occurrence of a copyright problem for an augmented reality work created by including a character or a signature image selected by the logged-in user, that is, the copyright holder based on the information logged in the augmented reality content sharing application.

For example, the work creation unit 140 may include a property extraction unit 142 that extracts a property from a real-world object recognized by the object recognition unit 130, and the real-world object extracted by the property extraction unit 142 According to the determination result of the attribute determination unit 144 and attribute determination unit 144 that determines whether the attribute of is identical to the previously stored attribute information dependent on the identifier declared in the augmented reality work object, the virtual object in the real world It characterized in that it comprises an augmented unit 146 for augmenting the object through a predefined method in the augmented reality work.

The attribute extracting unit 142 extracts attributes from an object in the real world recognized by the object recognition unit 130. More specifically, the object recognition unit 130 recognizes an object in the real world and extracts an attribute capable of representing the characteristic of the object in the real world from the converted data. For example, if the object recognition unit 130 recognizes a road, a building, or a tree that is an object in the real world, the attribute extraction unit 142 determines what the size of the road, building, or tree is, and what the specific size or shape is. You can extract attributes such as.

The attribute determination unit 144 determines whether the attribute of the real-world object extracted by the attribute extraction unit 142 matches attribute information dependent on the identifier declared in the augmented reality work.

The augmentation unit 146 augments an object in the real world through a method predefined in the augmented reality work according to the determination result of the attribute determination unit 144.

For example, the attribute determination unit 144 determined that the attribute of the real-world object extracted from the attribute extracting unit 142 matches attribute information subordinate to the identifier "car", and the augmented reality work If the object is pre-defined to augment the rainy screen, it is augmented as if there is a car on a rainy street.

Meanwhile, the augmented screen may be displayed through an augmented reality content sharing-only application in the display means of the augmented reality content sharing system 10 according to an embodiment of the present invention.

The upload unit 150 uploads the augmented reality work generated by the work creation unit 140 to a social network server (SNS: Social Network Services/Sites) through the communication unit 100.

12 is an exemplary view showing a state in which an augmented reality work is uploaded according to an embodiment of the present invention. As illustrated in FIG. 12, the upload unit 150 may upload the augmented reality work generated by the work creation unit 140 to a user account.

In one embodiment, the upload unit 150 may upload the augmented reality work generated by the work creation unit 140 in the form of a video through an augmented reality content sharing application. That is, when the augmented reality work is a photo file, it is possible to designate the augmented reality work in the form of a cube or a single view and upload it as a video file including a plurality of images (photos).

In addition, the augmented reality content sharing application according to an embodiment may provide a multi-view function to simultaneously display two or more of a plurality of augmented reality works on the output screen of the augmented reality work. That is, it is possible to execute two or more augmented reality works uploaded to SNS on one augmented reality screen. Accordingly, it is possible to enjoy various augmented reality screens by selecting and executing a plurality of augmented reality works uploaded by the person or others.

According to an additional aspect of the present invention, the augmented reality content sharing system 10 may further include an inertial sensor unit according to an embodiment. The inertial sensor unit may be implemented by an inertial measurement unit (IMU) capable of measuring the movement of the augmented reality content sharing system 10, such as acceleration and angular velocity.

In this case, the current position or the current gaze direction of the augmented reality content sharing system 10 is determined by combining information on the motion detected by the inertial sensor unit on the real world object recognized by the object recognition unit 130 I can.

According to an embodiment, a user may generate augmented reality content using the augmented reality content sharing system 10 in a virtual space modeling a real space, and a remote SNS friend may receive such augmented reality content. Accordingly, the accuracy, speed, or efficiency of information exchange between acquaintances through SNS is improved, and realistic communication becomes possible.

In addition, in an embodiment, a user may create an augmented reality work using the augmented reality content sharing system 10 in a virtual space modeling a real space. In this case, the augmented reality work is stored with the user's current location information matched.

The augmented reality content sharing system 10 matches and stores the location information of the user terminal possessed by the user who created the augmented reality work with the augmented reality work. At this time, the location information of the user terminal may include receiving GPS information of the user terminal from the user terminal or receiving location information from the user through a user interface such as a map.

Then, after that, it is possible for the user to create an augmented reality work or to check the augmented reality content at the location uploaded to the SNS.

For example, if a user creates and uploads his/her photo as an augmented reality work while traveling in Tokyo, a follower who follows the user, an SNS friend, or the user himself/herself can check the augmented reality content at the corresponding place in Tokyo.

In one embodiment, an augmented reality content sharing-only application identifies the current user location, and if there is an augmented reality work uploaded from the location by the person or SNS friend the other day, it is displayed as a notification message or immediately displays the augmented reality work Can be printed.

13 is a flowchart of a method of driving an augmented reality content sharing system according to an embodiment of the present invention.

First, the object loading unit of the augmented reality content sharing system loads the previously stored AR (Augmented Reality) work object (S200).

The object loading unit loads an AR (Augmented Reality) virtual object based on a previously stored image. At this time, the augmented reality virtual object loaded may be a part of an image stored in the storage unit or a character, and it is interpreted to encompass all types and types of objects that can be used as virtual objects without any limitation. In addition, the object loading unit may download an augmented reality work object from a social network server (SNS: Social Network Services/Sites).

Then, the object recognition unit recognizes an object in the real world (S210). The object recognition unit recognizes objects in the real world. Here, recognition refers to the recognition of information that the attribute extraction unit to be described later can contribute in extracting attributes. Recognition is, for example, the appearance representing attributes such as size, color, material, weight, etc. of real-world objects, and all other attributes that can identify various real-world objects (e.g., barcode, QR code, trademark Etc.).

After that, the work generation unit augments the virtual object loaded by the loading unit on the real world object recognized by the object recognition unit through a predefined method to generate an augmented reality work (S220). In this case, according to an additional aspect of the present invention, the augmented reality work generated by the work generation unit may visually display identification information of the author who generated the work.

Then, the upload unit uploads the augmented reality work generated by the work creation unit to a social network server (SNS: Social Network Services/Sites) (S230). In an embodiment, the upload unit may upload the augmented reality work generated by the work creation unit to the user account.

The above-described method may be implemented as an application or in the form of program instructions that may be executed through various computer components and recorded in a computer-readable recording medium. The computer-readable recording medium may include program instructions, data files, data structures, etc. alone or in combination.

The program instructions recorded in the computer-readable recording medium may be specially designed and constructed for the present invention, and may be known and usable to those skilled in the computer software field.

Examples of computer-readable recording media include magnetic media such as hard disks, floppy disks, and magnetic tapes, optical recording media such as CD-ROMs and DVDs, and magnetic-optical media such as floptical disks. media), and a hardware device specially configured to store and execute program instructions such as ROM, RAM, flash memory, and the like.

Examples of the program instructions include not only machine language codes such as those produced by a compiler, but also high-level language codes that can be executed by a computer using an interpreter or the like. The hardware device may be configured to operate as one or more software modules to perform processing according to the present invention, and vice versa.

Although the above has been described with reference to embodiments, those skilled in the art will understand that various modifications and changes can be made to the present invention without departing from the spirit and scope of the present invention described in the following claims. I will be able to.

10: augmented reality content sharing system 20: SNS server
100: communication unit 110: storage unit
120: object loading unit 130: object recognition unit
140: work creation unit 150: upload unit

Claims (7)

An object loading unit that loads an augmented reality (AR) virtual object based on a previously stored image;
An object recognition unit for recognizing an object in the real world;
A work creation unit for generating an augmented reality work by augmenting a virtual object loaded by the loading unit to an object of the real world recognized by the object recognition unit through a predefined method; And
An augmented reality content sharing system comprising: an upload unit for uploading the augmented reality work generated by the work creation unit to a social network service (SNS) server.
The method of claim 1,
The object loading unit is configured to select a background image to be inserted into at least one of the three-dimensional diagram-shaped templates from among pre-stored images, and to generate a virtual object object in the form of at least a part of the three-dimensional diagram composed of the selected background image. Augmented reality content sharing system characterized by.
The method of claim 1
An augmented reality content sharing system, characterized in that, in the augmented reality work generated by the work creation unit, identification information of the author who generated the work is visually displayed.
The method of claim 1,
The augmented reality content sharing system, characterized in that the object loading unit downloads the augmented reality work object from a social network service (SNS: Social Network Services/Sites) server.

The method of claim 1, wherein the work creation unit,
A property extraction unit for extracting a property from an object in the real world recognized by the object recognition unit,
An attribute determination unit that determines whether the attribute of the real-world object extracted by the attribute extracting unit matches pre-stored attribute information dependent on the identifier declared in the augmented reality work object; and
An augmented reality content sharing system comprising an augmented unit for augmenting a virtual object to the real world object through a predefined method in the augmented reality work according to a determination result of the attribute determination unit.
The method of claim 1,
The augmented reality content sharing system is dedicated to augmented reality content sharing that performs a function of selecting an augmented reality (AR) work to be loaded, a function of recognizing and selecting objects in the real world, and a function of checking the generated augmented reality work. Augmented reality content sharing system, characterized in that the application is installed and driven.
In the driving method of the augmented reality content sharing system according to any one of claims 1 to 6,
Loading a previously stored augmented reality (AR) work object by an object loading unit;
Recognizing an object in the real world by an object recognition unit;
Generating an augmented reality work by augmenting a virtual object loaded by the loading unit to an object of the real world recognized by the object recognition unit through a predefined method by the work generation unit; And
A method of driving an augmented reality content sharing system comprising: uploading, by an upload unit, the augmented reality work generated by the work generation unit to a social network server (SNS: Social Network Services/Sites).

Images