WO2016053029A1

WO2016053029A1 - Method and system for generating message including virtual space and virtual object, and computer-readable recording medium

Info

Publication number: WO2016053029A1
Application number: PCT/KR2015/010396
Authority: WO
Inventors: 전재웅
Original assignee: (주) 애니펜
Priority date: 2014-10-01
Filing date: 2015-10-01
Publication date: 2016-04-07
Also published as: KR20160039776A; KR101659066B1

Abstract

According to one aspect of the present invention, a method for generating a message including a virtual space and a virtual object comprises the steps of: (a) generating a virtual space corresponding to a real world space within a real world image by performing 3D space recognition and 3D mapping for the real world image inputted from a camera; (b) generating a virtual object corresponding to a real world object within the real world image by performing 3D object recognition and 3D mapping for the real world image; and (c) generating a message by dynamically combining an additional object inputted by a user with the virtual space and/or the virtual object.

Description

Method, system and computer readable recording medium for generating a message comprising a virtual space and a virtual object

The present invention relates to a method, a system and a computer readable recording medium for generating a message comprising a virtual space and a virtual object.

In the conventionally introduced messenger service or social network service (SNS), a user may include content such as an emoticon or a sticker indicating his or her current situation or emotional state in addition to text in a message. In recent years, as the performance of the user terminal device is improved and the types of content that the user encounters become diverse, there is a situation in which a user's request to include different content in a message exists.

Accordingly, the present inventors have developed a technology that enables a user to generate various types of messages that are dynamically combined with virtual spaces and virtual objects.

The object of the present invention is to solve all the above-mentioned problems.

In addition, the present invention generates a virtual space corresponding to the real world space in the real world image by performing 3D spatial recognition and 3D mapping on the real world image input from the camera, 3D object recognition and 3D mapping are performed to create a virtual object corresponding to the real world object in the real world image, and dynamically combine the additional object input by the user with at least one of the above virtual space and the virtual object. To generate a message, so that the user can generate various types of messages that are dynamically combined with virtual space and virtual objects.

Representative configuration of the present invention for achieving the above object is as follows.

According to an aspect of the present invention, there is provided a method for generating a message including a virtual space and a virtual object, comprising: (a) performing 3D space recognition and 3D mapping on a real world image input from a camera; Generating a virtual space corresponding to the real world space in the real world image, (b) performing 3D object recognition and 3D mapping on the real world image to perform a virtual object corresponding to the real world object in the real world image ( a virtual object), and (c) generating a message by dynamically combining an additional object input by a user with at least one of the virtual space and the virtual object.

According to another aspect of the present invention, there is provided a method for generating a message including a virtual space and a virtual object, the method comprising: (a) 3D space recognition for a real world image input from a camera of a terminal device of a first user; Generating a virtual space corresponding to the real world space in the real world image by performing 3D mapping; (b) performing 3D object recognition and 3D mapping on the real world image to perform a real world in the real world image Creating a virtual object corresponding to the object, and (c) dynamically adding an additional object input in association with the real world object by at least one of the virtual space and the virtual object by a terminal device of a second user. By combining to generate a message provided on the terminal device of the first user.

According to still another aspect of the present invention, there is provided a system for generating a message including a virtual space and a virtual object, wherein 3D space recognition and 3D mapping are performed on a real world image input from a camera. A virtual space processor that generates a virtual space corresponding to the real world space in the real world image, and performs a 3D object recognition and 3D mapping on the real world image to perform a virtual object corresponding to the real world object in the real world image. There is provided a system including a virtual object processing unit for generating a, and an additional object management unit for generating a message by dynamically combining the additional object input by the user with at least one of the virtual space and the virtual object.

According to another aspect of the present invention, a system for generating a message including a virtual space and a virtual object, 3D space recognition and 3D mapping of the real world image input from the camera of the terminal device of the first user a virtual space processor for generating a virtual space corresponding to the real world space in the real world image by performing mapping, and performing 3D object recognition and 3D mapping on the real world image to correspond to the real world object in the real world image A virtual object processing unit for generating a virtual object, and an additional object input in association with the real world object by a terminal device of a second user by dynamically combining with at least one of the virtual space and the virtual object; The additional object manager which generates a message provided on the terminal device of the first user. The system is also provided.

In addition, there is further provided a computer readable recording medium for recording another method, system for implementing the present invention and a computer program for executing the method.

According to the present invention, the user can create a message that he wants to convey in various forms that are dynamically combined with a virtual space or a virtual object, so that various forms of communication can be achieved between users and a new way of message. The effect of being able to provide a service is achieved.

In addition, according to the present invention, the effect of enabling a user to deliver a message via a virtual object corresponding to a real world object in a real world image is achieved by simply performing a simple operation.

1 is a diagram illustrating a schematic configuration of an entire system for generating and managing a message including a virtual space and a virtual object according to an embodiment of the present invention.

2 is a diagram illustrating an internal configuration of a service providing system according to an embodiment of the present invention.

3 is a diagram illustrating a process of generating a message including a virtual space and a virtual object according to an embodiment of the present invention.

100: network

200: service delivery system

210: virtual space processing unit

220: virtual object processing unit

230: additional object management unit

240: communication unit

250: control unit

300: user terminal device

400: known messenger server or SNS server

DETAILED DESCRIPTION The following detailed description of the invention refers to the accompanying drawings that show, by way of illustration, specific embodiments in which the invention may be practiced. These embodiments are described in sufficient detail to enable those skilled in the art to practice the invention. It should be understood that the various embodiments of the present invention are different but need not be mutually exclusive. For example, certain shapes, structures, and characteristics described herein may be embodied in other embodiments without departing from the spirit and scope of the invention with respect to one embodiment. In addition, it is to be understood that the location or arrangement of individual components within each disclosed embodiment may be changed without departing from the spirit and scope of the invention. The following detailed description, therefore, is not to be taken in a limiting sense, and the scope of the present invention, if properly described, is defined only by the appended claims, along with the full range of equivalents to which such claims are entitled. Like reference numerals in the drawings refer to the same or similar functions throughout the several aspects.

DETAILED DESCRIPTION Hereinafter, exemplary embodiments of the present invention will be described in detail with reference to the accompanying drawings so that those skilled in the art can easily implement the present invention.

Configuration of the entire system

1 is a diagram illustrating a schematic configuration of an entire system for generating and managing a message including a virtual object according to an embodiment of the present invention.

As shown in Figure 1, the entire system according to an embodiment of the present invention, the communication network 100, the service providing system 200, the user terminal device 300 and a known messenger server 400 or SNS server 400.

First, according to an embodiment of the present invention, the communication network 100 may be configured to include both wired and wireless communication and may be configured as a communication network including a mobile communication network. More specifically, the communication network 100 according to the present invention is a known wireless LAN (WLAN), Code Division Multiple Access (CDMA), Wideband Code Division Multiple Access (WCDMA), Global System for Mobile communications (GSM) or Long LTE (Long). Term Evolution) It should be understood that the concept includes all the communication networks.

Next, according to an embodiment of the present invention, the service providing system 200 performs a three-dimensional (3D) spatial recognition and 3D mapping of the real world image input from the camera in the real-world image Create a virtual space corresponding to the real world space, perform 3D object recognition and 3D mapping on the real world image to generate a virtual object corresponding to the real world object in the real world image, By generating the message by dynamically combining the additional object input by the at least one of the above virtual space and the virtual object, the user can create a variety of types of messages dynamically combined with the virtual space and the virtual object Can be done.

The internal configuration of the service providing system 200 will be described in more detail in the section "Configuration of the service providing system" to be described later.

Next, the user terminal device 300 according to an embodiment of the present invention is a digital device including a function to enable communication after connecting to the communication network 100, a memory such as a smartphone, tablet, desktop, laptop, etc. Any portable digital device having a means and a microprocessor capable of computing may be adopted as the user terminal device 300 according to the present invention. Meanwhile, the user terminal device 300 according to an embodiment of the present invention may include a necessary program such as an application, a web browser, a widget, and the like for supporting a user to generate a message including a virtual space and a virtual object. Can be.

Next, according to an embodiment of the present invention, the messenger server 400 or the SNS server 400 performs communication with the service providing system 200 and / or the user terminal device 300 through the communication network 100. . For example, the messenger server 400 or the SNS server 400 performs a function of transferring a message generated by the user terminal device 300 and / or the service providing system 200 from one user to another user. Can be.

Configuration of Service Delivery System

Hereinafter, the internal configuration of the service providing system performing important functions for the implementation of the present invention and the function of each component will be described.

2, the service providing system 200 according to an exemplary embodiment of the present invention may include a virtual space processor 210, a virtual object processor 220, an additional object manager 230, a communicator 240, and a controller ( 250). According to an embodiment of the present invention, the virtual space processor 210, the virtual object processor 220, the additional object manager 230, the communicator 240, and the controller 250 may be an external system (not shown). Program modules). Such program modules may be included in the service providing system 200 in the form of an operating system, an application program module, and other program modules, and may be physically stored on various known storage devices. In addition, these program modules may be stored in a remote storage device that can communicate with the service providing system 200. On the other hand, such program modules include, but are not limited to, routines, subroutines, programs, objects, components, data structures, etc. that perform particular tasks or execute particular abstract data types, described below, in accordance with the present invention.

First, according to an embodiment of the present invention, the virtual space processor 210 performs 3D space recognition and 3D mapping on a real world image input from a camera to correspond to the real world space in the real world image. A function to create a virtual space can be performed. Here, the real world image may be an image photographed by a camera provided in the terminal device 300 of the user who wants to write a message, or may be an image previously stored in the user terminal device 300 or the service providing system 200. .

Specifically, according to an embodiment of the present invention, the virtual space processor 210 may include Augmented Reality (AR) scanning technology or SLAM (Simultaneous Localization And Mapping) technology, marker and QR code recognition. Recognition of real-world space in real-world images (ie, 3D space recognition) using recognition techniques, etc., matching the 3D coordinate system of real-world space in real-world images with the 3D coordinate system of virtual space (ie, 3D mapping) A virtual space corresponding to the 3D real world space within the space may be generated.

In the above, an embodiment in which the virtual space according to the present invention is a three-dimensional space is mainly described. However, the present invention is not necessarily limited thereto, and the virtual space according to the present invention may be a two-dimensional space.

Next, according to an embodiment of the present invention, the virtual object processing unit 220 may generate a virtual object corresponding to the real world object by recognizing the real world object in the real world image input from the camera.

Specifically, according to an embodiment of the present invention, the virtual object processing unit 220 uses an object recognition technique, an augmented reality scan technique, a SLAM technique, a marker and a QR code recognition technique, etc. A virtual object corresponding to a 3D real world object in real world image by recognizing an object (ie, 3D object recognition) and matching the coordinate and coordinate system of the 3D real world object in the real world image with the 3D coordinate and coordinate system of the virtual object (ie 3D mapping) Can be generated.

In the above, an embodiment in which the virtual object according to the present invention is a three-dimensional object is mainly described, but is not necessarily limited thereto, and the virtual object according to the present invention may be a two-dimensional object.

Meanwhile, in the present invention, the virtual space or the virtual object generated as described above may be visualized separately from the real world image and displayed in a form appended to the real world image on the screen of the user terminal device 300, but in contrast, the virtual space or virtual object The object may not be visualized separately from the real world image. When the virtual space or the virtual object is not visualized separately, only the real world image currently being photographed is displayed on the screen of the user terminal device 300, and the virtual space or the virtual object is dynamically combined with the additional object, which will be described later, to be displayed. It only performs the function of virtually defining a space or real world object.

Next, according to an embodiment of the present invention, the additional object manager 230 dynamically combines the additional object input from the terminal device 300 of the user who wants to write a message with the above virtual space and the above virtual object. By doing so, you can perform the function of generating a message. Here, the additional object may include a visual object including a visual effect that may be combined with a virtual space or a virtual object, an audio object including a sound effect that may be combined with a virtual space or a virtual object, and a user who sends a message. It may include a text object including the text input by the text object, a benefit object including the benefit information available to the user receiving the message, in addition to the schedule, alarm, weather, fortune telling, food information, beauty, fashion, dating, Other information objects, including various other information such as shopping, may be further included.

Hereinafter, additional objects that may be included in a message according to the present invention will be described in detail.

First, according to an embodiment of the present invention, the additional object manager 230 may support editing of a body, motion, movement path, sound effect, etc. of a visual object among additional objects included in a message. A user interface may be provided that allows a user who wants to compose a message to edit a body, a motion, a movement path, a sound effect, etc. of a virtual object. Here, the visual object according to the present invention refers to all kinds of objects that can be visually expressed in combination with the virtual space or the virtual object. For example, the visual object according to the present invention includes a character or an avatar. This may include objects capable of motion or item objects such as clothes, wings, weapons, parts, and equipment.

In detail, the additional object manager 230 according to an embodiment of the present invention may set a path in which a visual object such as a character moves in a virtual space with reference to a user manipulation. Here, the user's operation is an operation capable of specifying a virtual straight line or a curve extending from the first point to the second point in the virtual space, for example, a touch operation extending from the first point to the second point on the display screen, And a motion operation for moving the terminal apparatus including the predetermined motion input means from the first position to the second position or changing the posture of the terminal apparatus from the first posture to the second posture. In addition, the additional object manager 230 according to an embodiment of the present invention may assign a motion (or a property value related to an operation) of a visual object such as a character with reference to the user operation with reference to the user operation. Here, the motion that can be given to the visual object includes various actions such as walking, running, jumping, rolling, crawling, punching, shrugging, shaking, talking, singing, motion expressing the emotional state of the virtual object, and the like. Actions can be included. However, the type of motion in the present invention is not necessarily limited to those listed above, it will be appreciated that it can be added or changed as much as possible within the scope that can achieve the object of the present invention.

Next, according to an embodiment of the present invention, the additional object manager 230 may have a form in which virtual objects existing in the virtual space directly speak text input from the user terminal device 300 (for example, in a speech bubble). Text objects) can be combined into virtual spaces or virtual objects.

Next, according to an embodiment of the present invention, the additional object manager 230 may further include a benefit object related to benefits available to the user who will receive the message in the message generated as described above. For example, the benefit information that may be included in the message may include, but is not limited to, Gifticon to receive the product or service, prepaid card to purchase the product or service, or link information on a product or service already purchased on the online store. The benefit object such as the gift icon, the prepaid card, or the link information may be visualized and displayed in a form associated with the aforementioned virtual space or virtual object in the message.

Meanwhile, according to an embodiment of the present invention, the virtual space, the virtual object, the additional object, or the message generated as described above may be stored in a predetermined database (not shown) in a state that can be used again later, and the stored virtual Spaces, virtual objects, additional objects or messages can be invoked at the user's request and reused to generate other messages.

Next, the communication unit 240 according to an embodiment of the present invention performs a function to enable the service providing system 200 to communicate with the external device. Specifically, the communication unit 240 according to an embodiment of the present invention is interlocked with the message service provided by the messenger server 400 or the SNS server 400 so that the message generated as described above is delivered to at least one other user. To perform the function.

Lastly, the controller 250 according to an embodiment of the present invention controls the flow of data between the virtual space processor 210, the virtual object processor 220, the additional object manager 230, and the communicator 240. To perform. That is, the controller 250 controls the flow of data from the outside or between the respective components of the service providing system, such that the virtual space processor 210, the virtual object processor 220, the additional object manager 230, and the communicator 240 are controlled. Control each to perform its own function.

Referring to FIG. 3, when creating a new message, the virtual space and the virtual object may be generated by recognizing the 3D space and the 3D object in the real world image obtained from the camera and performing 3D mapping (S310, S315, and S320). The user can select a character that is a visual object that can be visually combined with a virtual space or a virtual object according to a user's operation, and set a motion, a movement path, and a sound effect of the character (S325, S330), and text input by the user In operation S335, a text object including the text object may be generated to be displayed in a metabolic form. Meanwhile, when editing an existing message, a virtual space, a virtual object, and a visual object which are stored in advance are called, and a text object including text newly input by the user is combined to be displayed in the form of a dialogue of the character. A message may be generated (S335). Next, the message generated as described above is directly transmitted through the message service provided by the service providing system 200, the known messenger server 400, or the known SNS server 400 according to the present invention (S345), It may be stored in the database for later reuse (S350).

Meanwhile, according to another exemplary embodiment of the present invention, the service providing system 200 performs 3D spatial recognition and 3D mapping on a real world image input from a camera of the terminal device 300 of the first user, thereby real world. A virtual space processor 210 for generating a virtual space corresponding to the 3D real world space in the image, and a virtual object processor for generating a virtual object corresponding to the 3D real world object in the real world image by performing 3D object recognition and 3D mapping on the real world image. 220, and a terminal of the first user by dynamically combining an additional object input in association with the real world object by the second device terminal device 300 with at least one of the virtual space and the virtual object. It may include an additional object manager 230 for generating a message provided on the device 300.

Therefore, according to the service providing system 200 according to another embodiment of the present invention, through a real-world object (for example, a toy car, a doll, etc.) that can be exchanged as a real thing between the first user and the second user. So that the message can be delivered.

Hereinafter, according to another embodiment of the present invention, a process in which the first user delivers a message to the second user through the doll A, which is a real world object, will be described in detail.

First, a first user may photograph a real world image including a doll A, which is a real world object, by using a camera provided in his terminal device 300. In this case, the first user may take a picture captured by the terminal device 300 of the first user. As 3D space recognition, 3D object recognition, and 3D mapping are performed on the real world image, a virtual object corresponding to the doll A in the real world image may be generated. Meanwhile, the virtual object created through the terminal device 300 of the first user may be utilized when the first user inputs an additional object to be delivered to the second user.

Next, the first user may deliver the doll A, which is the real-world object recognized above, to the second user as a real object.

Next, the second user may take a real world image including the doll A, which is a real world object received as a real object from the first user, using a camera provided in the terminal device 300. Accordingly, the second user As 3D space recognition, 3D object recognition, and 3D mapping are performed on the real world image captured by the terminal device 300, the virtual object corresponding to the doll A in the real world image may be generated. Meanwhile, the virtual object created through the terminal device 300 of the second user may be utilized in providing the additional object received by the second user from the first user through the second user terminal device 300.

Next, the first user may input the text to be conveyed to the second user using his terminal device 300 as an additional object to be displayed in combination with the virtual object corresponding to the doll A. In this case, the doll The virtual object corresponding to the doll A and the additional object corresponding to the text input by the first user may be displayed on the screen of the terminal device 300 of the second user photographing A. That is, the second user can receive the message from the first user via the doll A. For another example, suppose that the first user delivers a visual object called doll clothes B to the second user as an additional object to be included in the message while the first user delivers the doll A, which is a real world object, to the second user. In this case, a scene of doll A wearing doll clothes B may be displayed on the screen of the terminal device 300 of the second user photographing doll A, which is a real world object.

Meanwhile, the first user may continuously input, modify or delete various additional objects to be displayed in combination with the virtual object corresponding to the doll A by using the terminal device 300 of the user. The additional object displayed in combination with the virtual object corresponding to the doll A on 300 may be updated.

Embodiments according to the present invention described above may be implemented in the form of program instructions that may be executed by various computer components, and may be recorded in a computer-readable recording medium. The computer-readable recording medium may include program instructions, data files, data structures, etc. alone or in combination. Program instructions recorded on the computer-readable recording medium may be those specially designed and configured for the present invention, or may be known and available to those skilled in the computer software arts. Examples of computer-readable recording media include magnetic media such as hard disks, floppy disks and magnetic tape, optical recording media such as CD-ROMs, DVDs, and magneto-optical media such as floptical disks. media), and hardware devices specifically configured to store and execute program instructions, such as ROM, RAM, flash memory, and the like. Examples of program instructions include not only machine code generated by a compiler, but also high-level language code that can be executed by a computer using an interpreter or the like. The hardware device may be configured to operate as one or more software modules to perform the process according to the invention, and vice versa.

Although the present invention has been described by specific embodiments such as specific components and the like, but the embodiments and the drawings are provided to assist in a more general understanding of the present invention, the present invention is not limited to the above embodiments. For those skilled in the art, various modifications and variations can be made from these descriptions.

Accordingly, the spirit of the present invention should not be limited to the above-described embodiments, and all of the equivalents or equivalents of the claims, as well as the appended claims, fall within the scope of the spirit of the present invention. I will say.

Claims

A method for generating a message comprising a virtual space and a virtual object, the method comprising:

(a) generating a virtual space corresponding to the real world space in the real world image by performing 3D spatial recognition and 3D mapping on a real world image input from a camera;

(b) generating a virtual object corresponding to the real world object in the real world image by performing 3D object recognition and 3D mapping on the real world image, and

(c) generating a message by dynamically combining the additional object input by the user with at least one of the virtual space and the virtual object.

How to include.
The method of claim 1,

In the step (c),

The additional object may include a visual object including a visual effect that may be combined with the virtual space or the virtual object, an audio object that includes a sound effect that may be combined with the virtual space or the virtual object, and the message. And at least one of a text object including text input by a user sending the benefit object and a benefit object including benefit information available to the user receiving the message.
The method of claim 1,

In the step (c),

And when the visual object is included in the additional object, providing a user interface for allowing the user to edit at least one of a body, a motion, a movement path, and a sound effect of the visual object.
The method of claim 2,

In the step (c),

Combining the text object with the virtual object in the form of dialogue spoken by the virtual object when the text object is included in the additional object.
The method of claim 1,

At least one of the virtual space, the virtual object, the additional object, and the message is stored in a reusable state.
The method of claim 1,

(d) causing the generated message to be delivered to at least one other user

How to include more.
A method for generating a message comprising a virtual space and a virtual object, the method comprising:

(a) 3D space recognition and 3D mapping are performed on a real world image input from a camera of a terminal device of a first user to create a virtual space corresponding to the real world space in the real world image. Generating step,

(b) generating a virtual object corresponding to the real world object in the real world image by performing 3D object recognition and 3D mapping on the real world image, and

(c) generating a message provided on the terminal device of the first user by dynamically combining the additional object input in association with the real world object by at least one of the virtual space and the virtual object by the terminal device of the second user; Steps to

How to include.
The method of claim 7, wherein

In the step (c),

The additional object may include a visual object including a visual effect that may be combined with the virtual space or the virtual object, an audio object that includes a sound effect that may be combined with the virtual space or the virtual object, and the second user. And at least one of a text object including text input by the user and a benefit object including the benefit information input by the second user and available to the first user.
A computer-readable recording medium having recorded thereon a computer program for executing the method according to any one of claims 1 to 8.
A system for generating a message comprising a virtual space and a virtual object, the system comprising:

A virtual space processor configured to generate a virtual space corresponding to the real world space in the real world image by performing 3D space recognition and 3D mapping on a real world image input from a camera;

A virtual object processor configured to generate a virtual object corresponding to the real world object in the real world image by performing 3D object recognition and 3D mapping on the real world image;

Additional object management unit for generating a message by dynamically combining the additional object input by the user with at least one of the virtual space and the virtual object

System comprising.
A system for generating a message comprising a virtual space and a virtual object, the system comprising:

Virtual to generate a virtual space corresponding to the real world space in the real world image by performing 3D space recognition and 3D mapping on a real world image input from a camera of the terminal device of the first user Space Processing Unit,

A virtual object processor configured to generate a virtual object corresponding to the real world object in the real world image by performing 3D object recognition and 3D mapping on the real world image;

The additional object generating a message provided on the terminal device of the first user by dynamically combining the additional object input in association with the real world object by at least one of the virtual space and the virtual object by the terminal device of the second user. Management

System comprising.