WO2019017582A1

WO2019017582A1 - Method and system for collecting cloud sourcing-based ar content templates and automatically generating ar content

Info

Publication number: WO2019017582A1
Application number: PCT/KR2018/006020
Authority: WO
Inventors: 정유진; 김정희
Original assignee: 네이버랩스 주식회사
Priority date: 2017-07-18
Filing date: 2018-05-28
Publication date: 2019-01-24
Also published as: KR20190009081A; KR101989089B1

Abstract

A content generation method may comprise the steps of: with respect to an augmented reality (AR) content generated by combining a virtual content with a background image, associating and storing background information that is information related to the background image and virtual content information that is information related to the virtual content; analyzing the background information and the virtual content information so as to determine a ranking of the virtual content information; when a specific background image is input from an electronic device, recommending, for the specific background image, at least one piece of virtual content information by using the ranking of the virtual content information; and generating a new AR content by applying, to the specific background image, virtual content information selected by a user of the electronic device from the recommended virtual content information.

Description

A method and system for collecting AR content templates based on cloud sourcing and automatically generating AR contents

The following description relates to a technique for generating Augmented Reality (AR) content.

Recently, as the Internet has developed, the digital contents industry has also become remarkable, because digital contents are being activated in the tool of the Internet.

In addition, digital cameras and camcorders that have camera and camcorder functions, and digital cameras that are widely inexpensive, among the functions of mobile communication terminals that are widely used by people all over the world, UGC (User Generated Contents) is becoming a mega trend for the general public as contents such as photographs and videos that are directly produced and edited by the user.

Furthermore, interest in AR content is increasing as interest in Augmented Reality (AR) increases. For example, in Korean Patent Laid-Open Publication No. 10-2002-0078141 (Oct. 18, 2002), a plurality of video cameras are used to acquire an object movie and display it through a network in real time through a viewer A virtual reality image creating system and method are disclosed.

In AR UGC service based on 3D contents, users have difficulty in creating content by editing / manipulating various contents in real space / real space in real space. There is a limit in size or coverage.

The present invention provides a method and system for collecting content information generated in real situations or backgrounds by users and providing it as content presets that are automatically edited and applied to real space by other users.

A method of generating content by computer, the method comprising: associating background information, which is information related to the background image, with virtual content information that is information related to the virtual content, with respect to AR (Augmented Reality) content generated by combining virtual content with a background image ; Analyzing the background information and the virtual content information to determine a ranking of the virtual content information; Recommending at least one virtual content information using the ranking of the virtual content information for the specific background image when a specific background image is input from the electronic device; And generating new AR contents by applying the virtual content information selected from the recommended user among the recommended virtual content information to the specific background image.

According to an aspect of the present invention, the storing step may store at least one of the type and number of objects in the background image, the layout information, the context information of the background image, the location information, and the time information as the background information.

According to another aspect, the storing step may store the parameter value of the virtual content combined with the background image as the virtual content information.

According to another aspect of the present invention, the determining step includes vectorizing the background information and the virtual content information, and embedding space in which a background vector representing the background information and a virtual content vector representing the virtual content information are mapped, ; And assigning a ranking to the virtual content information based on a distance between a vector representing the specific background image and a vector representing the specific background image in the embedding space after mapping the embedded background image to the embedding space.

According to another aspect, the recommending step may recommend the at least one virtual content information using ranking according to at least one of a frequency of use of the virtual content information, a freshness, a location similarity, and a user profile have.

According to another aspect, in the recommending step, the virtual content vector in the embedding space among the virtual content information may be adjusted to rank the virtual content information within a certain distance from the vector representing the specific background image, It is possible to exclude at least some of the virtual content information whose virtual content vector is within a certain distance from the vector representing the specific background image from the recommendation target.

According to another aspect, in the recommending step, the virtual content information having the same content parameter among the virtual content information may be excluded from the recommendation target.

According to another aspect, in the recommending step, the virtual content information selected from the user of the electronic device among the recommended virtual content information may be provided with a preview applied to the specific background image.

According to another aspect of the present invention, when the AR content is 3D character content generated by using the face image as the background image, the storing step may analyze the face image and store the facial expression information acquired from the analysis result together .

According to another aspect, in the recommending step, when a specific face photograph is input from the electronic device, 3D character content having facial expression information corresponding to a facial expression of the specific face photograph may be recommended.

A computer-implemented content creation system, comprising: at least one processor configured to execute computer-readable instructions, the at least one processor comprising: A content storage unit for associating background information, which is information related to a background image, with virtual content information that is information related to the virtual content; A content ranking unit for analyzing the background information and the virtual content information to determine a ranking of the virtual content information; A content recommending unit for recommending at least one virtual content information using the ranking of the virtual content information with respect to the specific background image when a specific background image is input from the electronic device; And a content generation unit for generating new AR contents by applying the virtual content information selected from the user of the electronic device among the recommended virtual content information to the specific background image.

According to embodiments of the present invention, AR contents are easily and conveniently generated by collecting content information generated by users in real situations or backgrounds and providing them as content presets that are automatically edited and applied to other users .

1 is a diagram illustrating an example of a network environment according to an embodiment of the present invention.

2 is a block diagram for explaining an internal configuration of an electronic device and a server in an embodiment of the present invention.

FIG. 3 is a diagram illustrating an example of a component that a processor of a server according to an embodiment of the present invention may include.

4 is a flowchart illustrating an example of a method that a server according to an embodiment of the present invention can perform.

5 to 6 are views for explaining an example of a process of collecting background information and virtual content information utilized in AR content production according to an embodiment of the present invention.

7 to 8 are views for explaining an example of a process of assigning ranking of virtual content information according to an embodiment of the present invention.

9 to 11 are views for explaining an example of a process of recommending virtual content information according to an embodiment of the present invention.

DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS Hereinafter, embodiments of the present invention will be described in detail with reference to the accompanying drawings.

Embodiments of the present invention relate to techniques for generating AR content, and more particularly, to a method and system for collecting AR content templates based on cloud sourcing and automatically generating AR contents therefrom.

Embodiments including those specifically disclosed herein may automatically generate new AR content using a content template created by an AR content by another user, thereby increasing content expansion, efficiency, convenience, cost reduction, etc. In the aspects of the present invention.

1 is a diagram illustrating an example of a network environment according to an embodiment of the present invention. 1 shows an example in which a plurality of

electronic devices

110, 120, 130, 140, a plurality of

servers

150, 160, and a network 170 are included. 1, the number of electronic devices and the number of servers are not limited to those shown in FIG.

The plurality of

electronic devices

110, 120, 130, 140 may be a fixed terminal implemented as a computer device or a mobile terminal. A mobile phone, a tablet PC, a navigation device, a computer, a notebook, a digital broadcasting terminal, a PDA (personal digital assistant), a PMP (personal digital assistant) Portable Multimedia Player). For example, the first electronic device 110 may communicate with other

electronic devices

120, 130, 140 and / or the

servers

150, 160 via the network 170 using a wireless or wired communication scheme.

The communication method is not limited, and may include a communication method using a communication network (for example, a mobile communication network, a wired Internet, a wireless Internet, a broadcasting network) that the network 170 may include, as well as a short-range wireless communication between the devices. For example, the network 170 may be a personal area network (LAN), a local area network (LAN), a campus area network (CAN), a metropolitan area network (MAN), a wide area network (WAN) , A network such as the Internet, and the like. The network 170 may also include any one or more of a network topology including a bus network, a star network, a ring network, a mesh network, a star-bus network, a tree or a hierarchical network, It is not limited.

Each of the

servers

150 and 160 is a computer device or a plurality of computers that communicate with a plurality of

electronic devices

110, 120, 130 and 140 through a network 170 to provide commands, codes, files, Lt; / RTI > devices.

For example, the server 160 may provide a file for installation of an application to the first electronic device 110 connected via the network 170. [ In this case, the first electronic device 110 can install an application using a file provided from the server 160. [ The server 150 is connected to the server 150 according to the control of an operating system (OS) included in the first electronic device 110 or at least one program (for example, a browser or an installed application) Services and contents can be provided. For example, if the first electronic device 110 transmits a service request message to the server 150 via the network 170 under the control of the application, the server 150 transmits a code corresponding to the service request message to the first And the first electronic device 110 can provide contents to the user by displaying and displaying a screen according to the code according to the control of the application.

2 is a block diagram for explaining an internal configuration of an electronic device and a server in an embodiment of the present invention. In FIG. 2, the internal configuration of the first electronic device 110 as an example of one electronic device and the server 150 as an example of one server will be described. Other

electronic devices

120, 130, 140 or server 160 may have the same or similar internal configurations.

The first electronic device 110 and the server 150 may include

memories

211 and 221,

processors

212 and 222,

communication modules

213 and 223 and input /

output interfaces

214 and 224. The

memories

211 and 221 may be a computer-readable recording medium and may include a permanent mass storage device such as a random access memory (RAM), a read only memory (ROM), and a disk drive. The

memory

211 and 221 may store an operating system or at least one program code (for example, codes for an application installed and driven in the first electronic device 110). These software components may be loaded from a computer readable recording medium separate from the

memories

211 and 221. [ Such a computer-readable recording medium may include a computer-readable recording medium such as a floppy drive, a disk, a tape, a DVD / CD-ROM drive, and a memory card. In other embodiments, the software components may be loaded into

memory

211, 221 via

communication modules

213, 223 rather than a computer readable recording medium. For example, at least one program may be a program installed by a file distribution system (for example, the server 160 described above) that distributes installation files of developers or applications, May be loaded into the

memory

211, 221 based on the application described above.

Processors

212 and 222 may be configured to process instructions of a computer program by performing basic arithmetic, logic, and input / output operations. The instructions may be provided to the

processors

212 and 222 by the

memories

211 and 221 or the

communication modules

213 and 223. For example, the

processor

212, 222 may be configured to execute a command received in accordance with a program code stored in a recording device, such as the

memory

211, 221.

The

communication modules

213 and 223 may provide a function for the first electronic device 110 and the server 150 to communicate with each other through the network 170 and may be connected to other electronic devices ) Or another server (e.g., server 160). The request generated by the processor 212 of the first electronic device 110 in accordance with the program code stored in the recording device such as the memory 211 is transferred to the server 170 via the network 170 under the control of the communication module 213. [ (150). Conversely, control signals, commands, contents, files, and the like provided under the control of the processor 222 of the server 150 are transmitted to the communication module 223 of the first electronic device 110 via the communication module 223 and the network 170, May be received by the first electronic device (110) via the second network (213). For example, the control signal or command of the server 150 received through the communication module 213 may be transmitted to the processor 212 or the memory 211, and contents or files may be transmitted to the first electronic device 110, May also be stored as a storage medium.

The input / output interface 214 may be a means for interfacing with the input / output device 215. For example, the input device may include a device such as a keyboard or a mouse, and the output device may include a device such as a display for displaying a communication session of the application. As another example, the input / output interface 214 may be a means for interfacing with a device having integrated functions for input and output, such as a touch screen. The processor 212 of the first electronic device 110 may use the data provided by the server 150 or the second electronic device 120 in processing the instructions of the computer program loaded into the memory 211. For example, A service screen or contents configured by the user can be displayed on the display through the input / output interface 214. [ The input / output interface 224 can also output information configured using the data provided by the server 150 in processing the instructions of the computer program loaded in the memory 221 by the processor 222 of the server 150 have.

Also, in other embodiments, the first electronic device 110 and the server 150 may include more components than the components of FIG. However, there is no need to clearly illustrate most prior art components. For example, the first electronic device 110 may be implemented to include at least some of the input / output devices 215 described above, or may be implemented as a transceiver, a Global Positioning System (GPS) module, a camera, And may further include components. More specifically, when the first electronic device 110 is a smart phone, it may be an acceleration sensor, a gyro sensor, a camera, various physical buttons, buttons using a touch panel, input / output ports, It is understood that various components such as a vibrator may be further included in the first electronic device 110.

Hereinafter, a method and system for collecting AR content templates based on cloud sourcing and automatically generating AR contents will be described.

In an environment where general users create AR contents, it is important to match a specific space, location, and situation in the real world with virtual content matching. There is a difficulty in expressing an AR authoring requiring various data searching and manipulation when the user writes in real space or in real time. Accordingly, it is possible to collect the content information generated by the users in the actual situation or background, and to provide the content information as a content preset that other users can directly use to author the AR content.

FIG. 3 is a diagram illustrating an example of a component that a server of a server according to an exemplary embodiment of the present invention may include; FIG. 4 illustrates an example of a method that a server can perform according to an exemplary embodiment of the present invention; Fig.

3, the processor 222 of the server 150 includes a content storage unit 310, a content ranking unit 320, a content recommendation unit 330, and a content generation unit 340 can do. Such processor 222 and components of the processor 222 may control the server 150 to perform the steps S410 through S440 included in the method of FIG. At this time, the components of the processor 222 and the processor 222 may be implemented to execute instructions according to the code of the operating system and the code of at least one program that the memory 221 contains. In addition, components of processor 222 may be representations of different functions performed by processor 222 in accordance with control commands provided by the operating system or by at least one program. For example, the content storage unit 310 may be used as a functional representation for controlling the processor 222 to collect and store AR contents according to the control command described above.

In step S410, the content storage unit 310 may associate and store the background information included in the content and the virtual content information for each of the AR content generated in the AR UGC service environment based on the 3D content. The server 150 serves as a platform for providing AR UGC services based on 3D content to a plurality of

electronic devices

110, 120, 130 and 140 as clients. At this time, It is possible to provide a 3D content editing tool capable of applying positioning, movement, orientation, various styles and motion sets in a background image. A user can create AR contents by synthesizing various virtual contents matching the background image by using a 3D content editing tool. The virtual contents may include a human real picture, an unrealistic character, a general object, and the like, and may include text information such as a caption, and motion or movement information of the character. The AR contents are designed to be combined into a background image and a virtual image, and may include background information and virtual content information. At this time, the background information may be information on the background image used to generate the AR contents. Background information may be spatial information about a space, and may include object information in a background or angle information of parameters of 3D contents. The content storage unit 310 may analyze the information of the background image such as the actual background or the virtual background used for generating the AR content, and store the analyzed information as background information. For example, the content storage unit 310 recognizes an object in the background image through image analysis and stores the type and number of the recognized object, the layout information, background information of the background, position information (e.g., global positioning system Etc.), time information, and the like. The virtual content information may be information on virtual content that can be overlaid on the background image. In other words, the content storage unit 310 stores the position, size, number, movement, tracking, motion, and style information of the virtual content manually edited in the background image through the 3D content editing tool to create the AR content The virtual content information can be stored together with the background information.

It is possible to collect various information such as GIFs (graphic interchange format) and moving picture data on the web, as well as AR contents generated by users, and to collect information available for creating new AR contents based on the analyzed contents.

In operation S420, the content ranking unit 320 may analyze the background information and the virtual content information to determine the ranking of the virtual content information based on the background information. The content ranking unit 320 may correlate the background information and the virtual content information to give a ranking on which content is mainly enhanced on a certain background and how the content is edited. At this time, the content ranking unit 320 may individually rank various parameters (e.g., number, size, position, motion, style, etc.) of the virtual content used to generate the AR content. A concrete method of assigning the ranking of the virtual content information will be described below again.

In step S430, when a specific background image is input from the electronic device 110, the content recommendation unit 330 may generate at least one virtual content information using the ranking of the virtual content information for the background information corresponding to the specific background image And can be recommended to the user of the electronic device 110. If the user selects a background image from the electronic device 110 or captures an actual background through the camera, the content recommendation unit 330 analyzes the background image and stores the most matching background information stored in step S410 It is possible to provide some virtual contents as recommended contents applicable to the background image by using the ranking of the virtual contents information based on the selected and selected background information. In other words, the content recommendation unit 330 uses the AR content produced by other users to directly apply the virtual content, which is mainly used in the background matched with the background image selected by the user of the electronic device 110, to the background image It can be provided as a possible content preset. At this time, the content recommending unit 330 may recommend the virtual content according to the overall situation of the background image or the main object among the objects in the background image. For example, the content recommending unit 330 may analyze a scene corresponding to a specific background image, and recommend a virtual content frequently used in a scene most similar to the scene. As another example, the content recommendation unit 330 may recommend virtual content that is frequently used at a location related to a specific background image based on the location information included in the background information. As another example, it is also possible to combine two or more of the content recommendation criteria, for example, to recommend a virtual content frequently used in a specific scene at a specific location with a combination of scenes and locations. At this time, the content recommendation unit 330 may sequentially provide the next ranked virtual content information according to the user selection based on the ranking of the virtual content information. The content recommendation unit 330 uses the caption information collected as the virtual content information as well as the 3D content as the content template applicable to the background image specified by the user of the electronic device 110, Can be provided together.

In step S440, the content generation unit 340 generates the virtual content information selected by the user of the electronic device 110 among the virtual content information recommended in step S430 as the background image specified by the user of the electronic device 110 The new AR contents can be generated. The user of the electronic device 110 can confirm the recommended content template for a specific background image, select a desired content template among the selected content template, and easily edit and create his or her own AR contents based on the template. At this time, the content generation unit 340 may provide a 3D content editing tool capable of positioning, moving, orienting, and editing various styles and motion sets of a recommended content template. The user can further create new AR contents by further editing the position or motion of the virtual content in the content template recommended for the specific background image by using the 3D content editing tool.

5, a user may create an AR content 500 by synthesizing a virtual content 520 that matches a background image 510 using a 3D content editing tool provided by a server 150. FIG. When the user generates the AR content 500 as the UGC contents, the content storage unit 310 stores information about the background image 510 included in the AR content 500 (type, number, placement information, background information, (E.g., position, size, number, movement, tracking motion, style information, etc.) of virtual content 520 manually edited Content information) can be recorded. Accordingly, the content storage unit 310 analyzes the AR content generated by the users and generates background information 610, which is information on the background image, and information on the virtual content, which is applied to the background image, The virtual content information 620 can be simultaneously recorded.

The content ranking unit 320 can vectorize the background information and the virtual content information stored by the content storage unit 310, respectively. For example, the background information may include [background image, position information, time information], and the virtual content information may include [kind of content, number, size, position, motion]. FIG. 7 exemplarily shows a state in which the background information 610 and the virtual content information 620 of FIG. 6 are vectorized and mapped into a vector space. The content ranking unit 320 uses the deep learning learning model to generate an embedding space such that the background vector 710 and the virtual content vector 720 are mapped into one space as shown in FIG. You can learn. At this time, it is possible to learn using a loss function (for example, triplet loss) so that backgrounds and contents that are generated as AR contents and related to each other are close to each other, and non-related backgrounds and contents of different AR contents are mapped away.

Referring to FIG. 8, when the background image specified by the user is received, the content ranking unit 320 maps the specific background vector 801 to the embedded embedding space previously obtained after vectorizing the specific background image, The virtual content vector 720 and the virtual content vector 720. [ For example, a higher rank may be assigned to a virtual content vector 720 closer to a specific background vector 801 based on a specific background vector 801. It is also possible to assign a rank based on the frequency of use of the virtual content information, the latestness, the location similarity, the user profile, and the like.

The content recommendation unit 330 can recommend virtual content frequently used by other users on a background similar to the background image specified by the user by using the ranking of the virtual content information. It is possible to exclude similar contents among the high ranked virtual contents in order to provide various contents as much as possible for a specific background image. For example, the content recommendation unit 330 may adjust the ranking of the contents in the embedding space to a certain distance from the specific background vector 801 or adjust the ranking of the contents within a predetermined distance from the specific background vector 801 It is possible to exclude the remainder excluding the arbitrarily selected at least one content from the recommendation object. As another example, it is possible to exclude contents having the same parameters such as the type and number of contents from virtual contents.

The content recommendation unit 330 can apply various recommendation criteria to the ranking of the virtual content information. For example, the content recommendation unit 330 may notify the user of the frequency of use, freshness, positional similarity, user profile, The virtual content information can be recommended according to the ranking.

9, when the user inputs a new background image 910, the content recommending unit 330 may activate a 'virtual content recommendation' menu 911 for recommending virtual content information. When the user operates the 'virtual content recommendation' menu 911 in a state that a new background image 910 is input, the content recommendation unit 330 may display the virtual content 910 applicable to the background image 910 as shown in FIG. 10, (1011) can be recommended. At this time, the virtual content 1011 is a content template directly applicable to the background image 910, and includes the history content of the AR content generated by the other users, which has the history similar to the background image 910 . The virtual content 1011 can be sequentially provided according to rankings based on the frequency of use, freshness, position similarity, and user profile of the virtual content information.

Referring to FIG. 10, the content recommending unit 330 may provide a preview function for showing a specific content 1020 selected by the user on the background image 910 of the virtual content 1011 . Then, the content recommendation unit 330 may sequentially provide the next ranked content according to the user's selection of the virtual content 1011. If the user selects the next ranked virtual content 1120 as shown in FIG. 11 The content 1120 can provide a preview of how the content 1120 is applied on the background image 910. The recommended virtual content 1011 for the background image 910 input by the user includes parameter values manually edited by other users (position, size, number, movement, tracking, motion, style information, etc.) And can provide a preview of the parameter applied to the background image 910 as it is.

As a kind of AR contents, 3D facial modeling or 3D avatar can be generated by using the face photograph of the user. The user can share contents among users in a service environment in which contents are produced by applying various motions and styles to a background image which is an actual real-time camera image, thereby applying user's face photographs to contents created by other users, / RTI >

For example, the user A may combine various styles and motions using 2D photographs on which his / her face is imaged to create 3D character contents. At this time, the server 150 may convert the 3D character contents created by the user A into video And store various parameters (character size, position, motion, style, movement path, etc.) of the contents when storing the contents.

When the user B captures his / her face image with the camera image, the server 150 can recommend the 3D character content produced by the user A as virtual content applicable to the face photograph of the user B. The user B can request the content embedding (e.g., replay) by selecting the 3D character content created by the user A among the recommended contents, and the server 150 can transmit the stored parameters for the content to the electronic device of the user B , The face of the 3D character content created by the user A can be automatically replaced with the face of the user B caught by the camera image. The content embedding function can share not only the character but also the entire content scene in 3D form.

The server 150 may analyze face photographs when storing 3D character contents using face photographs, store facial expression information acquired from the analysis results together, and recommend contents using facial expression information. In other words, the server 150 can recommend the 3D character content having the most similar appearance to the expression shown in the face photograph of the user B as the virtual content applicable to the face photograph of the user B. For example, when the facial expression of the user B caught by the camera image is anger, the server 150 recommends the 3D character content having the facial expression information corresponding to the anger among the 3D character contents stored on the server 150 to the user B A new 3D character content can be constructed by applying a model with the closest expression to the face photograph of the user B. [

Accordingly, the user can apply the 3D character content created by another user to his / her character and reproduce new contents based on the 3D character content, which can be effectively used for viral marketing.

As described above, according to the embodiments of the present invention, AR contents are easily and conveniently generated by collecting content information generated by users in real situations or backgrounds and providing them as content presets that are automatically edited and applied to other users can do.

The apparatus described above may be implemented as a hardware component, a software component, and / or a combination of hardware components and software components. For example, the apparatus and components described in the embodiments may be implemented as a processor, a controller, an arithmetic logic unit (ALU), a digital signal processor, a microcomputer, a field programmable gate array (FPGA), a programmable logic unit, a microprocessor, or any other device capable of executing and responding to instructions. The processing device may execute an operating system (OS) and one or more software applications running on the operating system. The processing device may also access, store, manipulate, process, and generate data in response to execution of the software. For ease of understanding, the processing apparatus may be described as being used singly, but those skilled in the art will recognize that the processing apparatus may have a plurality of processing elements and / As shown in FIG. For example, the processing unit may comprise a plurality of processors or one processor and one controller. Other processing configurations are also possible, such as a parallel processor.

The software may include a computer program, code, instructions, or a combination of one or more of the foregoing, and may be configured to configure the processing device to operate as desired or to process it collectively or collectively Device can be commanded. The software and / or data may be embodied in any type of machine, component, physical device, computer storage media, or device for interpretation by a processing device or to provide instructions or data to the processing device have. The software may be distributed over a networked computer system and stored or executed in a distributed manner. The software and data may be stored on one or more computer readable recording media.

The method according to an embodiment may be implemented in the form of a program command that can be executed through various computer means and recorded in a computer-readable medium. At this time, the medium may be a program that continuously stores a computer executable program, or temporarily stores the program for execution or downloading. Further, the medium may be a variety of recording means or storage means in the form of a combination of a single hardware or a plurality of hardware, but is not limited to a medium directly connected to any computer system, but may be dispersed on a network. Examples of the medium include a magnetic medium such as a hard disk, a floppy disk and a magnetic tape, an optical recording medium such as CD-ROM and DVD, a magneto-optical medium such as a floptical disk, And program instructions including ROM, RAM, flash memory, and the like. As another example of the medium, a recording medium or a storage medium that is managed by a site or a server that supplies or distributes an application store or various other software is also enumerated.

While the present invention has been particularly shown and described with reference to exemplary embodiments thereof, it is to be understood that the invention is not limited to the disclosed exemplary embodiments. For example, it is to be understood that the techniques described may be performed in a different order than the described methods, and / or that components of the described systems, structures, devices, circuits, Lt; / RTI > or equivalents, even if it is replaced or replaced.

Therefore, other implementations, other embodiments, and equivalents to the claims are also within the scope of the following claims.

Claims

A computer-generated content generating method,

Associating background information, which is information related to the background image, with virtual content information, which is information related to the virtual content, in association with AR (Augmented Reality) content generated by combining virtual content with a background image;

Analyzing the background information and the virtual content information to determine a ranking of the virtual content information;

Recommending at least one virtual content information using the ranking of the virtual content information for the specific background image when a specific background image is input from the electronic device; And

Generating new AR content by applying virtual content information selected from a user of the electronic device among the recommended virtual content information to the specific background image

And generating the content.
The method according to claim 1,

Wherein the storing step comprises:

Storing at least one of the type and number of objects in the background image, the layout information, the context information of the background image, the location information, and the time information as the background information

And generating the content.
The method according to claim 1,

Wherein the storing step comprises:

Storing the parameter value of the virtual content combined with the background image as the virtual content information

And generating the content.
The method according to claim 1,

Wherein the determining comprises:

Vectorizing the background information and the virtual content information to learn an embedding space in which a background vector representing the background information and a virtual content vector representing the virtual content information are mapped; And

Assigning a ranking to the virtual content information based on a distance between a vector representing the specific background image and a vector representing the specific background image in the embedded space after being mapped to the embedded space

And generating the content.
The method according to claim 1,

Wherein the recommending step comprises:

Recommending the at least one virtual content information using rankings according to at least one of a frequency of use of the virtual content information, a freshness, a location similarity, and a user profile

And generating the content.
5. The method of claim 4,

Wherein the recommending step comprises:

A virtual content vector in the embedding space among the virtual content information is adjusted to rank the virtual content information within a certain distance from a vector representing the specific background image or the virtual content vector in the embedding space is adjusted to a vector And at least a part of the virtual content information within a predetermined distance from the recommended object

And generating the content.
The method according to claim 1,

Wherein the recommending step comprises:

The virtual content information having the same content parameter among the virtual content information is excluded from the recommendation object

And generating the content.
The method according to claim 1,

Wherein the recommending step comprises:

The virtual content information selected from the user of the electronic device among the recommended virtual content information provides a preview applied to the specific background image

And generating the content.
The method according to claim 1,

Wherein the storing step comprises:

And if the AR content is 3D character content generated by using a face photograph as the background image, the facial photograph is analyzed and the facial expression information obtained from the analysis result is stored together

And generating the content.
10. The method of claim 9,

Wherein the recommending step comprises:

When a specific face photograph is input from the electronic device, recommending 3D character content having facial expression information corresponding to a facial expression of the specific face photograph

And generating the content.
A computer-implemented content creation system,

At least one processor configured to execute computer readable instructions,

Lt; / RTI >

Wherein the at least one processor comprises:

A content storage unit for associating background information, which is information related to the background image, with virtual content information, which is information related to the virtual content, in association with AR content generated by combining virtual content with a background image;

A content ranking unit for analyzing the background information and the virtual content information to determine a ranking of the virtual content information;

A content recommending unit for recommending at least one virtual content information using the ranking of the virtual content information with respect to the specific background image when a specific background image is input from the electronic device; And

Generating a new AR content by applying virtual content information selected from a user of the electronic device among the recommended virtual content information to the specific background image,

The content generation system comprising:
12. The method of claim 11,

The content-

Vectorizing the background information and the virtual content information to learn an embedding space in which a background vector representing the background information and a virtual content vector representing the virtual content information are mapped,

Assigning a ranking to the virtual content information based on a distance between the specific background image and a vector representing the specific background image in the embedding space after mapping the specific background image to the embedding space

And the content generation system.
12. The method of claim 11,

The content recommendation unit,

Recommending the at least one virtual content information using rankings according to at least one of a frequency of use of the virtual content information, a freshness, a location similarity, and a user profile

And the content generation system.
13. The method of claim 12,

The content recommendation unit,

A virtual content vector in the embedding space among the virtual content information is adjusted to rank the virtual content information within a certain distance from a vector representing the specific background image or the virtual content vector in the embedding space is adjusted to a vector And at least a part of the virtual content information within a predetermined distance from the recommended object

And the content generation system.
12. The method of claim 11,

The content storage unit,

If the AR content is 3D character content generated by using a face photograph as the background image, analyzes the face photograph and stores facial expression information acquired from the analysis result together,

The content recommendation unit,

When a specific face photograph is input from the electronic device, recommending 3D character content having facial expression information corresponding to a facial expression of the specific face photograph

And the content generation system.