KR102026416B1 - Method and system for object personification and interaction - Google Patents

Abstract

It provides a method and system for personification and interaction of things. The object interaction method may further include generating a personification character for the object by attaching a face mask to the object specified in the AR (Augmented Reality) space; And providing an interaction with the object by using the personification character as an interface.

Description

METHOD AND SYSTEM FOR OBJECT PERSONIFICATION AND INTERACTION}

The description below relates to techniques for characterizing objects and interacting with them through a characterized interface.

3D models of objects are in increasing demand in various fields such as AR (Augmented Reality), computer games and animation production.

3D models are created using experienced experts and specialized software tools or produced using scanners, but recently, researches are being actively conducted to generate 3D models easily for the general public.

For example, Korean Patent Publication No. 10-1747951 (Registration Date June 9, 2017) discloses a technique for providing a new method of producing 3D human characters using 3D computer graphics based on a multi-view shooting method.

The present invention provides a method and system for interacting with an object by using a character personified as an interface.

It provides a method and system that can create anthropomorphic characters for objects and utilize them as AI digital agents.

An object interaction method performed in a computer system, the method comprising: generating a personification character for an object by attaching a face mask to an object specified in an AR (Augmented Reality) space; And providing an interaction with the thing by using the personification character as an interface.

According to an aspect, the object interaction method may further include generating the face mask from the corresponding image through face modeling of an image input or selected from the electronic device.

According to another aspect, the generating of the face mask may extract the face region from the image to 3D object the extracted face region.

According to another aspect, the generating may include attaching a face mask specialized for a specific company or a product to the thing.

According to another aspect, the generating may include implementing the face mask to track the object based on the 3D information of the object.

According to another aspect, the generating may include blending a boundary of the face mask.

According to another aspect, the generating may include blending a boundary of the face mask using a color value of the object.

According to another aspect, the providing may output information obtained through interworking with an artificial intelligence (AI) dialogue system through the personification character.

According to another aspect, the providing step may output the visual information together using a separate graphical user interface.

According to another aspect, the providing may include changing an expression of the personification character or attaching an effect in response to the given emotion information.

In combination with a computer, a computer program stored in a computer readable recording medium for executing the object interaction method on a computer is provided.

Provided is a computer-readable recording medium having a program recorded thereon for executing the object interaction method on a computer.

A computer system comprising: at least one processor configured to execute computer readable instructions contained in a memory, wherein the at least one processor receives a face mask from the image through face modeling of an input or selected image from an electronic device. A mask generator to generate; A character generator configured to attach the face mask to an object specified in an AR space to generate an anthropomorphic character for the object; And an interaction unit providing an interaction with the object by using the personification character as an interface.

According to embodiments of the present invention, a person who has anthropomorphized a person as an interface may interact with the object, and an anthropomorphic character may be used as various digital agents.

1 is a diagram illustrating an example of a network environment according to an embodiment of the present invention.
2 is a block diagram illustrating an internal configuration of an electronic device and a server according to an embodiment of the present invention.
3 is a block diagram illustrating an example of components that may be included in a processor of a server according to an embodiment of the present invention.
4 is a flowchart illustrating an example of a method that may be performed by a processor of a server according to an embodiment of the present invention.
5 to 6 are exemplary diagrams for explaining a process of generating a face mask according to an embodiment of the present invention.
7 to 8 are exemplary diagrams for explaining a process of generating an anthropomorphic character in an embodiment of the present invention.
9 to 10 are exemplary diagrams for explaining an interaction process using anthropomorphic characters in an embodiment of the present invention.
11 is a view for explaining other examples of anthropomorphic characters in one embodiment of the present invention.

Hereinafter, exemplary embodiments of the present invention will be described in detail with reference to the accompanying drawings.

The object interaction method according to embodiments of the present disclosure may be performed through a computer device, such as an electronic device or a server, to be described later. In this case, a computer program according to an embodiment of the present invention may be installed and driven in the computer device, and the computer device may perform the object interaction method according to the embodiment of the present invention under the control of the driven computer program. . The computer program described above may be stored in a computer-readable recording medium in combination with a computer device to cause the computer to execute the object interaction method. For example, the server serves as an information platform that provides information through AR (Augmented Reality) service, and in particular, may characterize things and support interaction with them through a characterized interface.

1 is a diagram illustrating an example of a network environment according to an embodiment of the present invention. The network environment of FIG. 1 illustrates an example including a plurality of electronic devices 110, 120, 130, and 140, a plurality of servers 150 and 160, and a network 170. 1 is an example for describing the present invention, and the number of electronic devices or the number of servers is not limited as shown in FIG. 1.

The plurality of electronic devices 110, 120, 130, and 140 may be fixed terminals or mobile terminals implemented as computer devices. Examples of the plurality of electronic devices 110, 120, 130, and 140 include smart phones, mobile phones, navigation systems, computers, notebook computers, digital broadcasting terminals, personal digital assistants (PDAs), and portable multimedia players (PMPs). ), Tablet PCs, game consoles, wearable devices, wearable devices, Internet of things (IoT) devices, virtual reality (VR) devices, augmented reality (AR) devices, and the like. For example, although FIG. 1 illustrates the shape of a smart phone as an example of the electronic device 1 110, in the embodiments of the present invention, the electronic device 1 110 may use a wireless or wired communication method to substantially connect the network 170. It may mean one of various physical computer devices that can communicate with other electronic devices 120, 130, 140 and / or servers 150, 160.

The communication method is not limited and includes not only a communication method using a communication network (for example, a mobile communication network, a wired internet, a wireless internet, a broadcasting network, a satellite network, etc.) that the network 170 may include, but also a short range wireless communication between devices. Can be. For example, the network 170 may include a personal area network (PAN), a local area network (LAN), a campus area network (CAN), a metropolitan area network (MAN), a wide area network (WAN), and a broadband network (BBN). And one or more of networks such as the Internet. The network 170 may also include any one or more of network topologies, including bus networks, star networks, ring networks, mesh networks, star-bus networks, trees, or hierarchical networks, but It is not limited.

Each of the servers 150 and 160 communicates with the plurality of electronic devices 110, 120, 130, and 140 through the network 170 to provide a command, code, file, content, service, or the like. It may be implemented in devices. For example, the server 150 may be a system that provides a first service to a plurality of electronic devices 110, 120, 130, and 140 connected through the network 170, and the server 160 may also have a network ( It may be a system that provides a second service to the plurality of electronic devices 110, 120, 130, and 140 connected through the 170. As a more specific example, the server 150 uses an application as a computer program installed and driven in the plurality of electronic devices 110, 120, 130, and 140 to provide a service (for example, an AR service, etc.) for which the corresponding application is intended. The first service may be provided to the plurality of electronic devices 110, 120, 130, and 140. As another example, the server 160 may provide a service for distributing the file for installing and driving the above application to the plurality of electronic devices 110, 120, 130, and 140 as a second service.

2 is a block diagram illustrating an internal configuration of an electronic device and a server according to an embodiment of the present invention. 2 illustrates an internal configuration of the electronic device 1 110 and the server 150 as an example of the electronic device. In addition, the other electronic devices 120, 130, 140, or the server 160 may also have the same or similar internal configuration as the aforementioned electronic device 1 110 or the server 150.

The electronic device 1 110 and the server 150 may include memories 211 and 221, processors 212 and 222, communication modules 213 and 223, and input / output interfaces 214 and 224. The memories 211 and 221 are non-transitory computer readable recording media. The memories 211 and 221 are non-transitory computer-readable recording media. The memories 211 and 221 are non-transitory computer-readable recording media. Permanent mass storage devices. In this case, the non-volatile mass storage device such as a ROM, an SSD, a flash memory, a disk drive, or the like may be included in the electronic device 1 110 or the server 150 as a separate persistent storage device that is distinct from the memories 211 and 221. In addition, the memory 211, 221 includes an operating system and at least one program code (for example, a browser installed and driven in the electronic device 1 110 or an application installed in the electronic device 1 110 to provide a specific service). Code) can be stored. These software components may be loaded from a computer readable recording medium separate from the memories 211 and 221. Such a separate computer-readable recording medium may include a computer-readable recording medium such as a floppy drive, disk, tape, DVD / CD-ROM drive, memory card, and the like. In other embodiments, software components may be loaded into the memory 211, 221 through a communication module 213, 223 that is not a computer readable recording medium. For example, at least one program is a computer program that is installed by files provided by a file distribution system (for example, the server 160 described above) through the network 170 to distribute installation files of developers or applications. It may be loaded into the memories 211 and 221 based on (for example, the above-described application).

Processors 212 and 222 may be configured to process instructions of a computer program by performing basic arithmetic, logic, and input / output operations. Instructions may be provided to the processors 212, 222 by the memory 211, 221 or the communication modules 213, 223. For example, the processors 212 and 222 may be configured to execute a command received according to a program code stored in a recording device such as the memory 211 and 221.

The communication modules 213 and 223 may provide a function for the electronic device 1 110 and the server 150 to communicate with each other through the network 170, and the electronic device 1 110 and / or the server 150 may communicate with each other. May provide a function for communicating with another electronic device (eg, electronic device 2 120) or another server (eg, server 160). For example, a request generated by the processor 212 of the electronic device 1 110 according to a program code stored in a recording device such as the memory 211 may be controlled by the server 170 through the network 170 under the control of the communication module 213. 150). Conversely, control signals, commands, contents, files, and the like provided according to the control of the processor 222 of the server 150 are transmitted to the communication module of the electronic device 1 110 via the communication module 223 and the network 170 ( It may be received by the electronic device 1110 through 213. For example, the control signal, command, content, file, etc. of the server 150 received through the communication module 213 may be transmitted to the processor 212 or the memory 211, and the content, file, etc. may be transferred to the electronic device 1. 110 may be stored as a storage medium (permanent storage described above) that may further include.

The input / output interface 214 may be a means for interfacing with the input / output device 215. For example, an input device may include a device such as a keyboard, a mouse, a microphone, a camera, and the like, and an output device may include a device such as a display, a speaker, a haptic feedback device, and the like. As another example, the input / output interface 214 may be a means for interfacing with a device in which functions for input and output are integrated into one, such as a touch screen. The input / output device 215 may be configured as one device with the electronic device 1110. In addition, the input / output interface 224 of the server 150 may be a means for interfacing with an apparatus (not shown) for input or output that may be connected to or included in the server 150. As a more specific example, the processor 212 of the electronic device 1110 uses data provided by the server 150 or the electronic device 2 120 in processing a command of a computer program loaded in the memory 211. The service screen or the content may be displayed on the display through the input / output interface 214.

Also, in other embodiments, the electronic device 1 110 and the server 150 may include more components than those of FIG. 2. However, it is not necessary to clearly show most of the prior art components. For example, the electronic device 1 110 may be implemented to include at least some of the above-described input / output devices 215 or other components such as a transceiver, a global positioning system (GPS) module, a camera, various sensors, a database, and the like. It may further include elements. More specifically, when the electronic device 1 110 is a smartphone, an acceleration sensor, a gyro sensor, a camera module, various physical buttons, a button using a touch panel, an input / output port, and vibration for a smartphone generally include Various components such as a vibrator may be implemented to be further included in the electronic device 1 110.

Hereinafter, specific embodiments of a method and system for AR-based personification and interaction will be described.

In the AR service, characters can play various roles as not only avatars that replace users in space, but also as digital agents that communicate information and communicate with users, or as main characters or characters in content scenarios. Can be.

Such a character is generated as a 3D model, and may be mainly a human form, or may be embodied by adding a human face to various object templates. However, creating digital characters of real 3D models is very complicated and time consuming.

The present invention proposes a method of modeling a face photograph in 3D, attaching it to an object, characterizing it, and interacting through a characterized interface.

3 is a block diagram illustrating an example of components that a processor of a server according to an embodiment of the present invention may include, and FIG. 4 illustrates a method that may be performed by a processor of the server according to an embodiment of the present invention. A flow chart showing an example of the.

The server 150 according to the present embodiment may provide an information platform service that provides various information as an AR service. In other words, the server 150 serves as a platform for providing AR content to a plurality of electronic devices 110, 120, 130, and 140 that are clients. The server 150 may provide a service that supports AR-based personification and interaction in connection with an application installed on the electronic devices 110, 120, 130, and 140.

In order to perform the object interaction method of FIG. 4, as illustrated in FIG. 3, the processor 222 of the server 150 may include a mask generator 310, a character generator 320, and an interaction unit ( 330). In some embodiments, the components of the processor 222 may be optionally included in or excluded from the processor 222. In addition, according to an embodiment, the components of the processor 222 may be separated or merged to represent the functions of the processor 222.

The processor 222 and the components of the processor 222 may control the server 150 to perform steps S410 to S440 included in the thing interaction method of FIG. 4. For example, the processor 222 and the components of the processor 222 may be implemented to execute instructions according to code of an operating system included in the memory 221 and code of at least one program.

Here, the components of the processor 222 may be representations of different functions of the processor 222 performed by the processor 222 according to an instruction provided by the program code stored in the server 150. . For example, the mask generator 310 may be used as a functional representation of the processor 222 controlling the server 150 in accordance with the above-described command so that the server 150 generates a face mask.

In operation S410, the processor 222 may read a command required from the memory 221 loaded with a command related to the control of the server 150. In this case, the read command may include a command for controlling the processor 222 to execute steps S420 to S440 to be described later.

In operation S420, the mask generator 310 may generate a face mask from the corresponding image through face modeling on an image designated by the user of the electronic device 1110. As the image to be modeled, an image selected from an image received from the electronic device 1 110 or an image on a database related to the server 150 may be used. For example, a face photograph of a user, an acquaintance, or a celebrity, It may include mascots of specific companies or images of specific products. The mask generator 310 may generate a face mask by modeling a face included in an image in 3D. In this case, the modeling method may include shadow-based modeling and a face-based model that generate a face model using a gradual change of shadow information in an image. Known modeling techniques can be used, such as solid modeling that generates face models using vertices, ridges, faces, and masses. The mask generator 310 may extract a mask of a face region including a forehead, eyebrows, eyes, nose, mouth, ears, and the like from the modeled face to 3D object.

In operation S430, the character generator 320 may generate an anthropomorphic character for the object by synthesizing the face mask generated in operation S420 with the object specified in the AR service environment. The character generator 320 attaches a face mask to a specific object pointed to by the camera of the electronic device 1 110 in the AR service environment or to a specific object selected by the user of the electronic device 1 110 through a camera screen. Can be characterized. In this case, the character generator 320 may blend the boundary of the mask area so that the face mask may be naturally placed on a specific object. The character generator 320 may utilize a representative color of a face or an object for the natural rendering of the face mask. For example, the character generator 320 may extract a representative color from an area excluding the eye, eyebrow, nose, and mouth areas, and then reflect the corresponding color to the character skin color. When creating an avatar using a face mask, the avatar skin color may be processed using the extracted representative color to naturally process a portion of the skin color such as a hand or a neck. As another example, the character generator 320 may extract a representative color around the area where the face mask is to be synthesized from a particular object and then blend the boundary of the mask area using the corresponding color. In other words, the character generator 320 may extract a color value of a region synthesized with a face mask from a specific object and perform blending using the color value of the object on the boundary of the mask region. That is, the character generator 320 may reflect the representative color of the object on the boundary of the face mask in order to smooth the boundary between the face mask and the object. In addition, the character generator 320 may implement a face mask attached to a specific object to track the object by using a 3D tracking technology. 2D-based tracking will fail tracking if an object or camera moves over a certain distance. In order to cope with various movements in the X, Y, and Z axes, it is effective to use 3D tracking, and in the case of 3D tracking, 3D model information can be generally obtained using CAD. If you are using CAD, you will most likely need to match the tracking starting point. It is also possible to use 3D model information for various views using deep learning instead of CAD. According to an environment using the AR service of the electronic device 1 110, for example, in a mobile camera based mono camera environment, a target object is based on 3D information of an object previously learned using deep learning. You can add realism to your character in response to changes in motion, rotation, and movement. In other words, the server 150 may construct a standard model, which is 3D information for each object, through the pre-learning of the object, and provide a 3D standard model for a specific object to be characterized. Accordingly, the character generator 320 may obtain a 3D standard model previously learned about a specific object that the user wants to characterize, and implement 3D tracking of the character based on the 3D standard model. In addition, when the electronic device 1 110 uses the AR service through a stereo or 3D sensor environment, the character generator 320 may use the 3D information provided directly by the electronic device 1 110 to track the 3D of the character. Can be implemented.

In operation S440, the interaction unit 330 may provide an interaction with a corresponding object by using a character personified as a face mask as an interface. The interaction unit 330 may interact through a characterized interface by attaching a face mask to a specific object. For example, the interaction unit 330 may communicate with a user through a specific object or provide various information such as weather, news, music, maps, photos, etc. in conjunction with an artificial intelligence (AI) conversation system. The artificial intelligence conversation system is applied to a personal assistant system, a chatbot platform, an artificial intelligence (AI) speaker, and the like to understand the intention of the user's command and provide corresponding information. For example, when the interaction unit 330 receives the voice input “Today's weather” according to the user's speech from the electronic device 1 110, the interaction unit 330 acquires information on today's weather by recognizing and analyzing the received voice input. The obtained information may be output as voice or text through the object characterized in the AR space, such as "the weather of today ...". In this case, the interaction unit 330 may provide visual information together using a separate pop-up, a word bubble, a tooltip, and the like in the information providing process.

In addition, the interaction unit 330 may exchange and express emotions between the object and the user by changing an expression of a face mask implemented as an anthropomorphic character. The interaction unit 330 may change the facial expression of the character through deformation of the facial region of the objectized mask through 3D modeling, and attach various effects to the personified object to maximize the expression of emotion. It is also possible. An effect is a content composed of image objects, and can include all filters, stickers, and emojis that can be synthesized in an AR-based image. This is not only a fixed object but also a moving image object to which a flash or animation is applied. It can be implemented. These effects represent emotion information and can be pre-categorized by emotion, that is, multiple emotions (e.g., joy, sadness, surprise, distress, suffering, anxiety, fear, disgust, anger, etc.) are predefined and each emotion Effects representing the corresponding emotions may be grouped and managed. The interaction unit 330 may extract the emotion information from the sentence of the voice input received from the user to express the emotion. In this case, the emotion information may include an emotion type and an emotion intensity (emotion degree). Emotional terms, that is, emotional terms, are defined in advance and classified into a plurality of emotion types (e.g., joy, sadness, surprise, distress, suffering, anxiety, fear, disgust, anger, etc.) according to predetermined criteria, According to the plurality of strength classes (eg, 1 to 10). Emotional terms may include phrases or sentences including specific words as well as specific words representing emotions. For example, words such as 'I like' or 'tough,' or phrases or sentences such as 'I love you too much' may be included in the category of emotional terms. For example, the interaction unit 330 may extract a morpheme from a sentence according to a user's voice input, and then extract a predetermined emotion term from the extracted morpheme to classify the emotion type and the emotion intensity corresponding to the extracted emotion term. When a sentence of a voice input includes a plurality of emotion terms, weights may be calculated according to the emotion type and emotion intensity to which the emotion term belongs, and through this, an emotion vector for the emotion information of the sentence may be calculated to obtain emotion information representing the sentence. Can be extracted. Techniques for extracting the above emotion information are exemplary and not limited thereto. It is also possible to use other well-known techniques. Accordingly, the interaction unit 330 may change the facial expression of the character or expose the effect of the emotion in response to the given emotion information, for example, the emotion information recognized from the voice input of the user.

As described above, the interaction unit 330 may utilize a character personified as an artificial intelligence digital agent for providing information.

5 to 6 are exemplary diagrams for explaining a process of generating a face mask according to an embodiment of the present invention.

Referring to FIG. 5, when the characterization request for a specific object (eg, macaroon) 501 is received on the AR service screen 500, the mask generator 310 may anthropomorphize the user of the electronic device 1110. You can ask for a portrait photo. Subsequently, as illustrated in FIG. 6, when the portrait photograph 600 is input or selected from the electronic device 1110, the mask generator 310 may use the forehead, eyebrows, eyes, nose, mouth, and ears in the portrait photograph 600. The face region 610 including the back may be extracted (S61), and the face mask 601 may be generated by 3D objectization of the extracted face region 610 (S62).

7 to 8 are exemplary diagrams for explaining a process of generating an anthropomorphic character in an embodiment of the present invention.

The character generator 320 may blend the boundary of the face mask 601 as shown in FIG. 7 for a natural synthesis with a specific object. In this case, the character generator 320 may blend the mask boundary based on the color information of the object to be characterized. For example, if the object 501 requesting characterization is yellow in FIG. 5, the face mask ( The boundary of 601 may be blended in yellow.

Referring to FIG. 8, the character generator 320 may generate a character 801 personifying the object 501 by synthesizing the face mask 601 with a specific object 501 selected on the AR service screen 500. Can be. For example, when selecting a macaron captured on a camera screen in an AR service environment, the macaroon may be made an anthropomorphic character 801 by attaching a face mask 601 to the macaron. In this case, the character generator 320 may attach the face mask 601 at a predetermined ratio and position in the area corresponding to the object 501 or the face mask 601 in the area corresponding to the object 501. It is also possible for the user to adjust the position or size of the area to attach the).

9 to 10 are exemplary diagrams for explaining an interaction process using anthropomorphic characters in an embodiment of the present invention.

The interaction unit 330 may provide an interaction with the object by using the characterized object as an interface, and interoperate with an artificial intelligence conversation system to transmit information to the user of the electronic device 1 110 through an anthropomorphic character. It can be output in an audio or visual way. As illustrated in FIG. 9, the interaction unit 330 may display a graphic user interface 903 such as a speech bubble to transmit information (weather, news, music, map, photo, etc.) to the user through the personification character 801. Can be delivered. In addition, referring to FIG. 10, the interaction unit 330 may expose various effect objects 1003 together around the personification character 801. In this case, the effect object 1003 may be used as an object representing emotion information analyzed in the voice command of the user of the electronic device 1110 or information included in the information to be transmitted to the user. In addition, the interaction unit 330 may also express emotions by changing the expression of the face mask that constitutes the personification character 801 during communication with the user. For example, when the user's emotion is recognized as 'anger', the face mask mesh of the personification character 801 may be changed to an angry expression or a flame effect of anger emotion may be attached to the vicinity of the personification character 801.

In the AR service environment, an anthropomorphic character may be generated by selecting a specific object and attaching a face mask to the object. In addition to the above examples, as shown in FIG. 11, the face mask 1102 is attached to the mobile phone 1101 on the AR space to make the character 1103 anthropomorphizing the mobile phone 1101, or the face mask to the detergent 1106. All objects can be characterized, such as attaching a 1107 to make a character 1108 personifying the detergent 1106, and any object can be synthesized with a mask of a 3D model and characterized to provide this information. interface).

The interaction unit 330 may support characterization and interaction of an object by using a part of a face (eg, eyes, mouth, etc.) or a specific interface graphic instead of a face mask. You can also partner with specific companies or products to provide specialized paces that can be used for promotions or viral marketing. For example, you can specialize in Chicken Face and Chicken-related conversations so that users can anthropomorphize and create and share content. At this time, the interaction or interface may be specialized for promotion or marketing purposes.

As described above, according to the embodiments of the present invention, a person who has anthropomorphized a person as an interface may interact with the object, and an anthropomorphic character may be used as various digital agents. By using a person who characterizes things, they can make things friendly and use them for children's education, etc., and can be useful for viral promotion / publicity of specific products or brands.

The apparatus described above may be implemented as a hardware component, a software component, and / or a combination of hardware components and software components. For example, the devices and components described in the embodiments may include a processor, a controller, an arithmetic logic unit (ALU), a digital signal processor, a microcomputer, a field programmable gate array (FPGA), a programmable PLU (programmable). It can be implemented using one or more general purpose or special purpose computers, such as logic units, microprocessors, or any other device capable of executing and responding to instructions. The processing device may execute an operating system (OS) and one or more software applications running on the operating system. The processing device may also access, store, manipulate, process, and generate data in response to the execution of the software. For convenience of explanation, one processing device may be described as being used, but one of ordinary skill in the art will appreciate that the processing device includes a plurality of processing elements and / or a plurality of types of processing elements. It can be seen that it may include. For example, the processing device may include a plurality of processors or one processor and one controller. In addition, other processing configurations are possible, such as parallel processors.

The software may include a computer program, code, instructions, or a combination of one or more of the above, and configure the processing device to operate as desired, or process it independently or collectively. You can command the device. The software and / or data may be embodied in any type of machine, component, physical device, computer storage medium or device in order to be interpreted by or provided to the processing device or to provide instructions or data. have. The software may be distributed over networked computer systems so that they may be stored or executed in a distributed manner. Software and data may be stored on one or more computer readable recording media.

The method according to the embodiment may be embodied in the form of program instructions that can be executed by various computer means and recorded in a computer readable medium. In this case, the medium may be to continuously store a program executable by the computer, or to temporarily store for execution or download. In addition, the medium may be a variety of recording means or storage means in the form of a single or several hardware combined, not limited to a medium directly connected to any computer system, it may be distributed on the network. Examples of the medium include magnetic media such as hard disks, floppy disks and magnetic tape, optical recording media such as CD-ROMs and DVDs, magneto-optical media such as floptical disks, And ROM, RAM, flash memory, and the like, configured to store program instructions. In addition, examples of another medium may include a recording medium or a storage medium managed by an app store that distributes an application, a site that supplies or distributes various software, a server, or the like.

Although the embodiments have been described by the limited embodiments and the drawings as described above, various modifications and variations are possible to those skilled in the art from the above description. For example, the described techniques may be performed in a different order than the described method, and / or components of the described systems, structures, devices, circuits, etc. may be combined or combined in a different form than the described method, or other components. Or even if replaced or substituted by equivalents, an appropriate result can be achieved.

Therefore, other implementations, other embodiments, and equivalents to the claims are within the scope of the claims that follow.

Claims (20)

In the thing interaction method performed in a computer system,
Generating a personification character for the object by attaching a face mask to the object specified in the AR (Augmented Reality) space; And
Providing an interaction with the object by using the personification character as an interface
Including,
The computer system serves as an information platform for providing information in an AR service environment for electronic devices that are clients.
The generating step,
The object in the AR space by attaching the face mask to the specified object as pointed by the camera of the electronic device in the AR service environment or selected by the user of the electronic device through the camera screen of the electronic device. Characterizing
Object interaction method characterized in that. The method of claim 1,
The thing interaction method,
Generating the face mask from the image through face modeling of an input or selected image from an electronic device
Things interaction method further comprising. The method of claim 2,
Generating the face mask,
Extracting a face region from the image to 3D object the extracted face region
Object interaction method characterized in that. The method of claim 1,
The generating step,
Attaching a face mask specialized for a specific company or product to the thing
Things interaction method comprising a. The method of claim 1,
The generating step,
Implementing the face mask to track the object based on the 3D information of the object
Things interaction method comprising a. The method of claim 1,
The generating step,
Blending a boundary of the face mask;
Things interaction method comprising a. The method of claim 1,
The generating step,
Blending a boundary of the face mask using a color value of the object
Things interaction method comprising a. The method of claim 1,
The providing step,
Outputting information obtained in connection with an artificial intelligence (AI) dialogue system through the personification character;
Object interaction method characterized in that. The method of claim 8,
The providing step,
Outputting visual information together using a separate graphical user interface
Object interaction method characterized in that. The method of claim 8,
The providing step,
Changing the expression of the personification character or attaching an effect in response to a given emotion information
Object interaction method characterized in that. A computer program stored in a computer readable recording medium for causing a computer to execute the object interaction method of any one of claims 1 to 10 in combination with a computer. A computer-readable recording medium having recorded thereon a program for causing a computer to execute the object interaction method of any one of claims 1 to 10. In a computer system,
At least one processor configured to execute computer readable instructions contained in the memory
Including,
The at least one processor,
A mask generator configured to generate a face mask from the image through face modeling of an image input or selected from an electronic device;
A character generator configured to attach the face mask to an object specified in an AR space to generate an anthropomorphic character for the object; And
An interaction unit that provides an interaction with the object by using the personification character as an interface.
Including,
The computer system serves as an information platform for providing information in an AR service environment for electronic devices that are clients.
The character generator,
The object in the AR space by attaching the face mask to the specified object as pointed by the camera of the electronic device in the AR service environment or selected by the user of the electronic device through the camera screen of the electronic device. Characterizing
Computer system characterized in that. The method of claim 13,
The mask generator,
Extracting a face region from the image to 3D object the extracted face region
Computer system characterized in that. The method of claim 13,
The character generator,
Implementing the face mask to track the object based on the 3D information of the object
Computer system characterized in that. The method of claim 13,
The character generator,
Blending the boundary of the face mask to attach to the object
Computer system characterized in that. The method of claim 13,
The character generator,
Blending a boundary of the face mask using a color value of the object
Computer system characterized in that. The method of claim 13,
The interaction unit,
Outputting the information obtained by interworking with the artificial intelligence system through the personification character
Computer system characterized in that. The method of claim 18,
The interaction unit,
Outputting visual information together using a separate graphical user interface
Computer system characterized in that. The method of claim 18,
The interaction unit,
To change the expression of the personification character or to attach an effect in response to a given emotion information
Computer system characterized in that.

Images