WO2009022819A2

WO2009022819A2 - System and method for supporting emotional expression of intelligent robot and intelligent robot system using the same

Info

Publication number: WO2009022819A2
Application number: PCT/KR2008/004618
Authority: WO
Inventors: Hae Wook Jung
Original assignee: Quaterview Co., Ltd.
Priority date: 2007-08-10
Filing date: 2008-08-08
Publication date: 2009-02-19
Also published as: KR100880613B1; WO2009022819A3

Abstract

An emotional expression assistant system and method and intelligent robot system using the emotion expression assistant system and method are provided. In the present invention, E-M mapping information containing mappings between emotion expression codes assigned to respective emotion states and detailed motion codes assigned to detailed motions and control information on the detailed motions are stored in a storage outside of the intelligent robot. An emotion expression RAC server creates, when receiving a request for emotion expression information on an emotion state, the emotion expression information with reference to the E-M mapping information and the control information and sends the emotion expression information to the intelligent robot, whereby the intelligent robot performs an emotion expression according to the emotion expression information received from the emotion expression RAC server. Accordingly, the emotional expression assistant system and method of the present invention enables producing and revising emotional expression-embedded robot content in very simple manner.

Description

Invention Title

SYSTEM AND METHOD FOR SUPPORTING EMOTIONAL EXPRESSION OF INTELLIGENT ROBOT AND INTELLIGENT ROBOT SYSTEM USING THE SAME

Technical Field

The present invention relates to an intelligent robot system and, in particular, to an emotion expression assistant system and method that enables an intelligent robot to express various emotions with sound effects as well as facial expressions and behaviors that are managed in the form of a pre-built database.

Background Art Intelligent robots are characterized by the interaction with the human being. Typically, when using personal computer (PC) based contents, the PC interacts with a user by mans of an input device such as keyboard or mouse. In the case of robot, however, more rich interaction with contents can be expected due to the behaviors and movements of the robot.

Emotion expression of the robot, in comparison with computer, is one of the important factors to enrich the interaction with a user. With such emotion expression ability, the intelligent robot understands the emotion of the human and provides services with appropriate bo emotional expression, thereby becoming a human- friendly companion.

Typically, robot contents are produced with the controls on various behaviors and expressions of the robots for interaction with the human. However, there are many difficulties for the content producers, who have no deep knowledge of possible behaviors and expressions of the robots, to produce robot operation control codes.

Korean Patent Application No. 10-2003-0067691 entitled "Contents Developing Tool for Intelligent Robots and Contents Developing Method using the Tool" discloses a contents developing method for adapting robot behaviors to contents. However, since this method matches behavioral elements to controllable parts of the robot, the content producer have to create a required expression by himself/herself . In this case, if the content producers have no exact knowledge of the ability of robot, they create required robot expressions.

Korean Patent Application No. 10-2005-0059737 entitled "Robot Contents Authoring Method using Script Generator and Commercially-Of-the-Shelf Multimedia Authoring Tool for Control of Various Robot Platforms" discloses a method for producing robot contents containing robot control commands . In order to adopt the content created with control commands for emotional expression of a specific robot to another type of robot, then content should be entirely revised. Also, when new robot control commands are added or the control command associated with the content are required to be modified, the content itself should be recreated. Furthermore, in order to use the modified content in the robot, the entire modified content should be downloaded from a server through a network.

Disclosure Technical Problem

In order to overcome the problems of the above prior arts, the present invention provides an emotional expression assistant system and method that is capable of embedding emotion expressions in a robot content by only selecting Robot Action Control (RAC) data listed in an emotion database. Also, the present invention provides an emotional expression assistant system and method that is capable of simply revising the emotional expression embedded in robot content without decomposing the entire content, thereby reducing network load. Also, the present invention provides an emotional expression assistant system and method that is capable of supporting a remote control service by providing an intelligent robot with emotional expressions appropriate for specific purpose through a network. Also, the present invention provides an intelligent robot system using the emotional expression assistant system and method that enable achieving the above- described objects.

Technical Solution

In accordance with an exemplary embodiment of the present invention, an emotion expression assistant system for an intelligent robot includes an emotion expression robot action control database (RAC DB) which stores emotion-motion (E-M) mapping information between emotion expression codes representing emotional states and at least one detailed motion codes matched with each emotion expression code and control information specifying the detailed motion code; and an emotion expression RAC server which creates, when the robot requests information on a specific emotion expression, the emotion expression information corresponding to the emotion expression with reference to the E-M mapping information and control information and sends the emotion expression information to the robot .

Preferably, the emotion expression RAC server retrieves the detailed motion code and control information with reference to the E-M mapping information and generates the emotion expression information composed of the detailed motion code and control information. Preferably, the control information includes at least one of a performing time, a number of performing times, and repetition option of the detailed motion.

Preferably, the emotion expression assistant system further includes an execution data storage which stores M-P mapping information containing mappings between the detailed motion codes and execution codes assigned for motions corresponding to the detailed motion codes and execution files corresponding to the execution codes, wherein the emotion expression RAC server retrieves the detailed motion code and control intormation corresponding to the emotion expression requested by the robot with reference to the E-M mapping information and control information, extracts execution code mapped to each detailed motion code, and generates the emotion expression information composed of an execution file corresponding to the execution code and the control information . Preferably, the detailed motion code and execution codes are universal codes supported by robots regardless of types of the robots requesting the emotion expression information.

Preferably, the emotion expression RAC DB stores emotion expression description information describing the emotion expression codes, and the emotion expression RAC server sends the emotion expression description information to the robot in response to a request from the robot . Preferably, the detailed motion code indicates^" at least one of eye motion, mouth motion, head motion, arm motion, and swivel motion. In accordance with another exemplary embodiment of the present invention, an intelligent robot system includes an intelligent robot which displays contents "^! and expresses emotion states in association with the respective contents; an emotion expression Robot Action Control database (RAC DB) which stores an E-M mapping information containing mappings between emotion expression codes assigned to respective emotion states and detailed motion codes assigned to detailed motions and control information on the detailed motions; and an emotion expression RAC server which creates, when receiving a request for emotion expression information on an emotion state, the emotion expression information with reference to the E-M mapping information and the control information and sends the emotion expression information to the intelligent robot, wherein the intelligent robot performs an emotion expression according to the emotion expression information received from the emotion expression RAC server. Preferably, the emotion expression RAC server extracts the detailed motion corresponding to the emotion state on which information is requested by the intelligent robot and the control information with reference to the E-M mapping information and the control information and generates the emotion expression information composed of the extracted detailed motion code and control information. Preferably, the control information includes at least one of a performing time, a number of performing times, and repetition option of the detailed motion.

Preferably, the intelligent robot includes an execution data storage for storing M-P mapping information containing mappings between the detailed motion codes and execution codes assigned for motions corresponding to the detailed motion codes and execution files corresponding to the execution codes, wherein the emotion expression RAC server retrieves the detailed motion code and control information corresponding to the emotion expression requested by the robot with reference to the E-M mapping information and control information and sends the emotion expression information composed of the extracted detailed motion code and control information to the intelligent robot; and the intelligent robot extracts the detailed motion code and control information from the emotion expression information, retrieves the execution code mapped to each detailed motion code with reference to the E-M mapping information, and executes an execution file corresponding to the execution code.

Preferably, the detailed motion code and the execution code are universal codes supported by robots regardless of types of the robots requesting the emotion expression information.

Preferably, the detailed motion code indicates at least one of eye motion, mouth motion, head motion, arm motion, and swivel motion.

Preferably, the intelligent robot includes a user interface for displaying the contents and expressing the emotion states of the intelligent robot; a storage for storing the contents and emotion expression synchronization information associated with the emotion states to be expressed in synchronization with the contents; and an RAC client module for controlling the user interface to display the contents stored in the storage and performs the emotion expression on the basis of the emotion expression synchronization information, wherein the RAC client module requests the emotion expression RAC server for the emotion expression information with reference to the emotion expression synchronization information, displays the content corresponding to the emotion expression information received from the emotion expression RAC server, and controls the user interface to present the emotion expression.

Preferably, the emotion expression synchronization information and the content are structured in separated files.

Preferably, the emotion expression synchronization information includes a time point, at which the emotion expression starts in association of the content, and a type of the emotion expression.

Preferably, the intelligent robot system includes an RAC production tool storage for storing at least on RAC production tool; and an RAC production terminal for loading the RAC production tool from the RAC production storage and creating emotion expression synchronization information and the content with the execution of the RAC production tool.

Preferably, the emotion expression RAC DB stores code description information on the emotion expression codes, and the emotion expression RAC server sends^' the code description information read out from the emotion expression RAC DB in response to a request from the RAC production terminal .

Preferably, the RAC production terminal includes a display for displaying the code description information; an input module for receiving an option selected by a user with reference to the code description information; and an RAC data generator for generation the emotion expression synchronization information associated with the emotion expression selected by the user.

Preferably, the RAC production terminal includes a storage for storing the content and the emotion expression synchronization information generated by the RAC data generator; and a simulator for checking whether the content and emotion expression synchronization information stored in the storage are valid.

Preferably, the RAC production terminal generates the E-M mapping information and sends the E-M mapping information to the emotion expression RAC DB, or receives the E-M mapping information from the emotion expression RAC DB, modifies the E-M mapping information, and sending the modified E-M mapping information to the emotion expression RAC DB; and the emotion expression RAC DB stores the E-M mapping information received from the RAC production terminal .

Preferably, the user interface includes a voice output module for outputting voice effect, a display for displaying the content, an input module for receiving user commands, and a physical motion module for driving physical motions of the intelligent robot.

Preferably, the emotion expression RAC server sends the E-M mapping information to the intelligent robot. Preferably, the emotion expression RAC server and the intelligent robot communicate with each other periodically, or when required, for synchronizing the E-M mapping information.

In accordance with another exemplary embodiment of the present invention, an emotion expression assistant method for an intelligent robot includes storing E-M mapping information containing mappings between emotion expression codes assigned to respective emotion states and detailed motion codes assigned to detailed motion codes assigned to detailed motions and control information on the detailed motions outside the intelligent robot; creating, when receiving a request for information on an emotion expression from the intelligent robot, emotion expression information corresponding to the emotion expression with reference to the E-M mapping information; and sending the emotion expression information to the intelligent robot, wherein the intelligent robot performs the emotion expression according to the emotion expression information.

Preferably, creating the emotion expression information includes retrieving the detailed motion^' code corresponding to the emotion expression and the control information on the detailed motion from the E-M mapping information; and generating the emotion expression information composed of the retrieved detailed motion code and control information.

Preferably, the emotion expression assistant method further includes storing M-P mapping information containing mappings between the detailed motion codes and execution codes assigned for motions corresponding to the detailed motion codes and execution files corresponding to the execution codes, wherein creating the emotion expression information includes retrieving the detailed motion code corresponding to the emotion expression and the control information on the detailed motion from the E-M mapping information; retrieving the execution code mapped to the retrieved detailed motion code from the M-P mapping information; and generating the emotion expression information composed of an execution file corresponding to the retrieved execution code and the control information.

Advantageous Effects

The emotion expression assistant system and method of the present invention allows a robot content producer to produce emotion expression-embedded robot content without deep knowledge on robot control mechanism by- selecting pre-produced Robot Action Control objects required for expressing one of listed emotional states. Also, the emotion expression assistant system and method of the present invention defines emotional states to be expressed by a robot with sets of emotion expression elements (such as shapes of eyes and mouth, behaviors of head, arms, and wheels, and LED color) such that a specific emotion expression can be embedded into a robot content by selecting a set of emotion expression elements as a whole rather than selecting the elements independently, resulting in simplification of emotion expression-embedded robot contents. Also, the emotion expression assistant system and method of the present invention stores robot contents and emotion expression-related information in different file formats and manages the contents and emotion expression RAC data files as instructions for controlling the contents and robot separately such that adding a^' new robot control command or modifying the emotion expression embedded in a content can be done by only revising the emotion expression RAC data file without decomposing the entire robot content. Also, the emotion expression assistant system and robot exchange the emotion expression RAC data file which is small in size relative to the content file for revising the emotional expression embedded in the content, resulting in reduction of network traffic load.

Also, the emotion expression assistant system and method of the present invention stores emotion expression RAC data within an emotion expression RAC database separated from a robot such that, when it is required to add an emotional expression to the content presented by the robot data, the robot accesses a server to download the required data. Accordingly, the emotion expression assistant system and method is applicable to a robot - based remote teaching system in which a teacher searches the emotion expression RAC database for an emotional expression appropriate for a progress of a content or study course and sends the emotion expression RAC data to the robot through a network.

Description of Drawings The above and other objects, features and advantages of the present invention will be more apparent from the following detailed description in conjunction with the accompanying drawings, in which: FIG. 1 is a block diagram illustrating a configuration of an intelligent robot system according to an exemplary embodiment of the present invention;

FIG. 2 is a block diagram illustrating a configuration of an intelligent robot system according to another exemplary embodiment of the present invention;

FIG. 3 is a block diagram illustrating a configuration of an intelligent robot system according to another exemplary embodiment of the present invention;

FIG. 4 is a table illustrating an example of emotion expression code description information for use in an intelligent robot system according to an exemplary embodiment of the present invention;

FIG. 5 is a table illustrating an example of E-M mapping information for use in an intelligent robot system according to an exemplary embodiment of the present invention;

FIG. 6 is a table illustrating an example of detailed motion code for use in an intelligent robot system according to an exemplary embodiment of the present invention;

FIG. 7 is a diagram illustrating a format of an RAC data file according to an exemplary embodiment of the present invention;

FIG. 8 is a message flow diagram illustrating steps of generating an emotion expression RAC data file for use in an intelligent robot system according to an exemplary embodiment of the present invention; and

FIG. 9 is a flowchart illustrating an emotion expression assistant method according to an exemplary embodiment of the present invention.

Mode for Invention

In the following, a Robot Action Control defines components such as facial expressions, behaviors, voice effect, and emotional states to be presented with these components by means of robot. The emotional state is represented by an emotion expression RAC object composed of corresponding facial expression, behavior, and voice effect .

In the following, the emotional states include satisfaction, pleasure, disappointment, sadness, and surprise. Each emotional state consists of three detailed sub- states for presenting the emotional state in three different levels.

Exemplary embodiments of the present invention are described with reference to the accompanying drawings in detail.

FIG. 1 is a block diagram illustrating a configuration of an intelligent robot system according to an exemplary embodiment of the present invention. Referring to FIG. 1, the intelligent robot system includes an intelligent robot 10 and an emotion expression assistant system 40. The intelligent robot 10 presents a content' object with an emotional expression in a predetermined manner. The emotion expression assistant system 40 provides the intelligent robot 10 with emotion expression information required for expressing specific emotions. For this purpose, the emotion expression assistant system 40 includes an emotion expression RAC DB 45 and an emotion expression RAC server 43.

The emotion expression RAC DB 45 stores E-M mapping information. The E-M mapping information includes mapping information for mapping an emotion expression code

(ECODE) assigned to each emotional expression presented by the intelligent robot 10 to at least one detailed motion code (MCODE) assigned to at least one detailed motion for achieving the emotional expression and detailed motion control information for controlling detailed motion of the intelligent robot 10.

The emotion expression RAC server 43 generates emotion expression information with reference to the E-M mapping information. In this embodiment, the emotion expression information is composed of a detailed motion code and a detailed motion control information for achieving a predetermined emotional expression. If the intelligent robot 10 requests the emotion expression information, the emotion expression RAC server 43 generates the emotion expression information with the detailed motion code and detailed motion control information corresponding to the emotional expression with reference to the E-M mapping information and transmits the emotion expression information to the intelligent robot 10.

In another exemplary embodiment, the emotion expression RAC server 430 may transmits the entire E-M mapping information to the intelligent robot 10 in response to the emotion expression information request. In this case, since the intelligent robot 10 has all of the emotion expression codes, E-M mapping information, and execution data, it can express a specific emotion independently without assistance of the emotion expression assistant system 40 afterward. Also, the intelligent robot 10 communicates with the emotion expression RAC server 43 periodically, or when required, for updating the E-M mapping information such that the intelligent robot acquires synchronization of the E-M mapping information with the emotion expression RAC server 43.

The intelligent robot 10 includes a user interface 5, contents & RAC data storage 9, an RAC client module 3, and an execution data storage 21.

The user interface 5 is responsible for displaying contents and emotional expression with user recognizable motion.

The execution data storage 21 stores M-P mapping information indicating mappings of detailed motion codes (MCODEs) and execution codes (PCODEs) and execution files corresponding the execution codes. Here, the execution code is assigned for identifying the execution file which is composed of instructions for the intelligent robot 10 to perform emotion expression. The contents & RAC data storage 9 stores (i) contents and (ii) emotion expression synchronization information associated with the emotion to be expressed by the intelligent robot 10. In this embodiment, the emotion expression synchronization information can^' be provided in the form of a file such as the emotion expression RAC data file, e.g. a text file in extended Markup Language (XML) format. The emotion expression RAC data file notifies the start time of the emotion expression and the emotional state to be presented. In this embodiment, it is preferred that the contents and emotion expression synchronization information are stored in the forms of separate files.

FIG. 7 is a diagram illustrating a format of an RAC data file according to an exemplary embodiment of the present invention.

Referring to FIG. 7, the content is entitled "Sing along animation" and supposed to be presented with an emotion expression indicated by the emotion expression codes (ECODE) of HRPOl, HRP03, HRS02, and HRGO2. The code HRPOl a time code indicating that the emotion expression starts at time point of 103 in association with the content "Sing along animation" . The value "103" of the time code is an exemplary one for indicating the start time of the emotion expression time in association with the content "Sing along animation " .

According to the emotion expression RAC data file shown in FIG. 7, the "Sing along animation" is performed by taking the expressions indicated by HRPOl at time 103, HRP03 at time 118, HRS02 at time 121, and HRG02 at time 140, sequentially.

As in the example of FIG. 7, the emotion expression RAC file includes the emotion expression codes supposed to be executed at specific times along with the presentation of the content.

The RAC client module 3 controls the user interface 5 to present the content and emotion expression. The RAC client module 3 displays the content read out from the content & RAC data storage 9 and controls the user interface 5 such that the emotion expression is presented on the basis of the emotion expression synchronization information. The RAC client module 3 can be implemented in the form of a content player for playing the content. In this case, the content player checks the emotional expression synchronized with the content with reference to the emotion expression synchronization information and requests the emotion the emotion expression RAC server 43 for expression information. Upon receipt of the emotion expression information from the emotion expression RAC server 43, the content player synchronizes the content with the emotion expressions using the emotion expression information.

Here, the RAC client module 3 requests the emotion expression RAC server 43 for the required emotion expression information and controls the user interface 5 to perform emotion expressions on the basis of the emotion expression information received from the emotion expression RAC server 43. The emotion expression information transmitted by the emotion expression RAC server 43 may include the detailed motion codes corresponding to the required emotion expressions and control information on the detailed motion codes. The RAC client module 3 analyzes the detailed motion code and detailed motion control information and retrieves execution codes corresponding to the detailed motion codes from the execution data storage 21 with reference to the M-P mapping information. Next, the RAC client module 3 performs the emotional expression by executing the execution file corresponding to the extracted execution code on the basis of the detailed motion control information. The user interface 5 includes a voice output module 17 for outputting voice information, a display module 13- for displaying the content, an input module for receiving instruction from the user, and a physical motion module 19 for generating physical motions. The physical motion module 19 may driving motions of arms, legs, head, etc. of the intelligent robot 10.

The emotions that can be expressed by the intelligent robot 10 may include pleasure, sadness, satisfaction, and disappointment. The pleasure emotion is expressed with more than one detailed motion. For example, the pleasure is expressed with facial expression, head motion, arm gesture, swivel behavior, and voice effect that are performed at the same time or in a sequential order.

In this embodiment, the detailed motions are identified by the detailed motion codes, and the emotional expressions are identified by the emotion expression codes. Also, the execution files are identified by the execution codes for the intelligent robot 10 to execute the detailed motions.

The detailed motion code is preferably a universal code available for various types of robots. That is, it is preferred that the mapping between the detailed motion code and execution code is done in general. This is because it is convenient for the producer can produce the emotion expression RAC data using the detailed motion codes without careful consideration on the the details of the execution codes.

Although the universal codes are preferably, the detailed motion code produced for specific types of robots can be used in this embodiment . Since no standard for the detailed motion code of robots exists yet, each type of robot needs type-specific E-M mapping information. Accordingly, the use of robot type-specific E-M mapping information is not excluded in the embodiment.

As describe above, an emotional expression maps to a unique emotion expression code, and each emotion expression code maps to at least one detailed motion code. Also, each detailed motion code maps to a corresponding execution code. In short, the emotional expression of the intelligent robot 10 is achieved by executing the execution file mapped to the execution code according to the detailed motion control information.

In order to express an emotional state in association with the displayed content display, the intelligent robot 10 sends an emotion expression information request (REQUEST) message, with reference to the emotion expression synchronization information, for requesting the emotion expression assistant system 40 for the emotion expression information. The emotion expression information request message may include the emotion expression code. Upon receipt of the emotion expression information request message, the emotion expression assistant system 40 extracts the detailed motion code and detailed motion control information mapped to the emotion expression code included in the emotion expression information request message from the E-M mapping information and sends the emotion expression information to the intelligent robot 10.

In the exemplary embodiment of FIG. 1, the RAC client module 3 sends the emotion expression information to the emotion expression assistant system 40. The emotion expression RAC DB 45 maintains the E-M mapping information that are previously stored, and the emotion expression RAC server 43 generates the emotion expression information composed of the detailed motion code and detailed motion control information matched with the emotional expression requested by the RAC client module 3 with reference to the E-M mapping information. Upon receipt of the emotion expression information from the emotion expression RAC server 43, the RAC client module 3 extracts the detailed motion code and detailed motion control information from the emotion expression information with reference to the emotion expression information and reads out the execution code corresponding to the detailed motion code from the execution data storage 21 with reference to the M-P mapping information. Consequently, the RAC client module 3 executes the execution file corresponding to the execution code according to the detailed motion control information. In this manner, the intelligent robot 10 expresses its emotional state.

Although the contents and RAC data are stored in the same storage media (i.e. the contents and RAC data storage 9) in the exemplary embodiment of FIG. 1, the intelligent robot 10 can be implemented such that the contents and RAC data are stored within separated storage media. Also, the intelligent robot 10 can be configured such that the execution codes and execution files are stored with the contents and RAC data within the same storage .

Also, although it is depicted that the RAC client module 3 controls the user interface 5 directly in FIG. 1, the intelligent robot 10 can be configured such that the RAC client module 3 controls the user interface 5 indirectly through an intermediate component . Although it is described that the M-P mapping information and execution codes are stored in the same storage, the M-P mapping information and execution codes can be stored in separate storages.

FIG. 2 is a block diagram illustrating a configuration of an intelligent robot system according to another exemplary embodiment of the present invention. Referring to FIG. 2, the intelligent robot system includes an intelligent robot 50 and an emotion expression assistant system 60. The intelligent robot 50 presents content with emotional expression synchronized with the content.

The emotion expression assistant system 60 provides the intelligent robot 50 with emotion expression information required for expressing emotion. For this purpose, the emotion expression assistant system 60 includes an emotion expression RAC DB 65, an emotion expression RAC server 63, and an execution data storage 61. The emotion expression RAC DB 65 stores E-M mapping information. The E-M mapping information includes mappings between emotion expression codes and detailed motion code and detailed motion control information. The execution data storage 61 stores M-P mapping information, execution codes, and execution files corresponding to the execution codes. The M-P mapping information includes mappings between the detailed motion codes and execution codes.

The emotion expression RAC server 63 generates emotion expression information using the E-M mapping information, M-P mapping information, and execution files. In this embodiment, the emotion expression information includes the detailed motion control information and execution file. The intelligent robot 50 includes a user interface 55, contents & RAC data storage 59, and an RAC client module 53. Operations of components of the above structured intelligent robot system are described hereinafter.

The RAC client module 53 controls the user interface 55 to display the contents stored in the contents & RAC data storage 59. In order to request the emotion expression assistant system 60 for the emotion expression information, the RAC client module 53 extracts the emotion expression code of the emotional expression to be expressed in synchronization with the content with reference to the emotion expression synchronization information stored in the content the contents & RAC data storage 59. The RAC client module 53 sends the emotion expression code to the emotion expression assistant system 60 for requesting to send emotion expression information corresponding to the emotion expression code. Upon receipt of the emotion expression information request, the emotion expression RAC server 63 extracts the detailed motion code and detailed motion control information corresponding to the emotion expression code from the E-M mapping information. The emotion expression RAC server 63 refers to the M-P mapping information for retrieving the execution code of the detailed motion code. The emotion expression server 63 extracts the execution code from the execution data storage 61 with reference to the M-P mapping information. Next, the emotion expression RAC server 63 retrieves the execution file matched with the execution code and sends emotion expression information containing the execution file and detailed motion control information to the RAC client module 53. Upon receipt of the emotion expression information, the RAC client module 53 executes the execution file according to the detailed motion control information.

Detailed descriptions of the function and structures referred to as the same reference numerals or similar names are omitted. Although it is depicted that the execution codes are stored in separated storage media in the exemplary embodiment of FIG. 2, the intelligent robot system can be configured such that the execution codes are stored in the emotion expression RAC DB 65. Although the RAC client module 3 is configured to synchronize the emotional expression with the content, execute the execution file, and controls other components in the exemplary embodiment of FIG. 2, the present invention is not limited thereto. For exemplary, the RAC client module can be configured only to synchronize the content with the emotional expression, whereas a controller (not shown) executes the execution file and controls the operations of other components.

Also, although the execution data storage 61 is a component of the emotion expression assistant system 60 in FIG. 2, it can be separately implemented as a server to which the intelligent robot 50 accesses through a network. That is, the execution data are managed by a remote server (hereinafter, called execution data server) such that the intelligent robot to download the execution data through a network. In this case, the emotion expression assistant system 60 sends the emotion expression information including the detailed motion code and its control information to the intelligent robot, and the robot downloads the execution data from the execution data server and expresses its emotional state using the execution data.

FIG. 3 is a block diagram illustrating a configuration of an intelligent robot system according to another exemplary embodiment of the present invention. Referring to FIG. 3, the intelligent robot system further includes an RAC production terminal 90 in addition to the intelligent robot 70 and the emotion expression assistant system 80 in comparison with the intelligent robot system depicted in FIGs. 1 and 2.

The RAC production terminal 90 produces the emotion expression RAC data and sends the emotion expression RAC data to the intelligent robot 70. For example, when it is required to initialize the intelligent robot with RAC data or update the RAC data stored in the intelligent robot 70, the RAC production terminal 90 is required. The RAC production terminal 90 generates the emotion expression synchronization information for acquiring synchronization between the content and the emotional expression to be presented along with. The intelligent robot 70 is provided with the storage media (e.g. contents and RAC data storage 9 and 59) for storing the emotion expression synchronization information created by the RAC production terminal 90.

The RAC production terminal 90 is configured to preferably read and display the previously produced contents. The RAC production terminal 90 can reads the previously produced contents; and creates, modifies, saves, and deletes the emotion expression synchronization for synchronizing the read content with the emotional expression to be performed. The RAC production terminal 90 can be a personal computer (PC) .

In this embodiment, the RAC production terminal 90 creates, modifies, or deletes the data (e.g. E-M mapping information) of the emotion expression RAC DB 85. The RAC production terminal 90 can receive the E-M mapping information from the emotion expression assistant system 80, modify the E-M mapping information, and sends the modified E-M mapping information back to the emotion expression assistant system 80. The RAC production terminal 90 also can perform a simulation using the E-M mapping information received from the emotion expression assistant system 80. The RAC production terminal 90 requests the emotion expression assistant system 80 for an RAC production tool and creates the emotion expression synchronization information using the RAC production tool provided by the emotion assistant system 80. Preferably, the RAC production terminal 90 is provided with a simulator for simulating the emotion expression synchronization information and contents.

Using the RAC production tool, the RAC production terminal 90 reads the previously produced content and inserts an emotional expression of the robot into a specific proceeding step of the contents. Since the emotional expressions are provided by the emotion expression assistant system as described above, the producer can insert an appropriate emotional expression into the content. The emotion expression synchronization information (e.g. emotion expression RAC data) produced using the RAC production tool is stored in a specific file format with an extension (e.g. **.RAC) different form the content file. Next, the RAC client module reads the content file and emotion expression synchronization information at the same time to synchronize with each other.

The RAC production tool can create, modify, save, and delete the emotion expression synchronization information. In this embodiment, the RAC production tool can create, modify, save, and delete the data (e.g. E-M mapping information) in the emotion expression RAC DB. The RAC production tool can download the E-M mapping information from the emotion expression assistant system and modify the downloaded E-M mapping information. Referring to FIG. 3, the emotion expression assistant system 80 includes an emotion expression RAC server 83, an emotion expression RAC DB 85, and an RAC production tool storage 89.

The emotion expression RAC DB 85 stores the emotion expression information, and the RAC production tool storage 89 stores the RAC production tool. In response to an RAC production tool request, the emotion expression RAC server 83 retrieves the RAC production tool from the RAC production tool storage 89 and sends the RAC production tool to the RAC production terminal 90. Preferably, the emotion expression RAC DB 85 stores information describing the emotion expression code (hereinafter, called emotion expression code description information) .

The operation of the RAC production terminal 90 for generation the emotion expression synchronization information is described hereinafter in more detail^'. If an RAC production tool request message is received from the emotion expression assistant system 80, the RAC production terminal 90 retrieves the RAC production tool from the RAC production storage 89 and sends the RAC production tool to the RAC production terminal 90. After receipt of the RAC production tool, the RAC production terminal 70 activates the RAC production tool. The RAC production tool is received in response to a request for the emotion expression code description information transmitted to the emotion expression assistant system 70. The RAC production tool presents the emotion expression code description information to the user such that the user selects an emotional expression to be inserted into the content. The RAC production tool generates the emotion expression synchronization information including the emotion expression code assigned to the emotion expression selected by the user. Although it is described that the RAC production terminal 90 receives the RAC production tool from the emotion expression assistant system 80 in the above description, the present invention is not limited thereto. For example, in a case that the RAC production terminal is a personal computer, the RAC production tool can be an application program installed on an operating system such as Windows .

Referring to FIG. 3, the RAC production terminal 90 includes an RAC data generator 91, a display 93, an input module 95, a storage 97, and a simulator 99.

The RAC data generator 91 generates contents and RAC data as synchronization information for synchronizing the emotional expression selected by the user with the content . In this embodiment, the RAC data generator 91 receives the RAC production tool from the emotion expression assistant system 80 and creates, modifies, saves, or deletes the emotion expression synchronization information using the received RAC production tool . The RAC data generator 91 also can create, modify, save, and delete the data (e.g. E-M mapping information) of the emotion expression RAC DB.

The display 93 displays the emotion expression code description information and the contents. The display 93 also can display a result of simulation performed by the simulator 99. The input module 95 provides a user interface such that the user can select an emotional state to be expressed in association with the content with reference to the emotion expression code description information. The storage 97 stores the RAC data generated by the RAC data generator 91. The storage 97 also stores the E-M mapping information provided by the emotion expression assistant system 80. The storage 97 also stores the execution data for simulation.

The simulator 99 performs simulation for checking whether the content and the emotion expression RAC data synchronized with the content are valid.

The RAC production terminal 90 sends the contents and emotion expression synchronization information associated with the emotional expression to be inserted to the contents to the intelligent robot 70.

The intelligent robot 70 saves the contents and emotion expression synchronization information and requests the emotion expression assistant system 80 for the emotion expression information, when the content is required to be displayed, with reference to the emotion expression synchronization information. When the emotion expression information is requested by the intelligent robot 70, the emotion expression RAC server 83 generates the requested emotion expression information by searching the emotion expression RAC DB 85 and sends the emotion expression information to the intelligent robot 70. The emotion expression information request may include an emotion expression code. In this case, the emotion expression RAC server 83 retrieves the detailed motion code and detailed motion control information mapped to the emotion expression code from the emotion expression RAC DB 85 and sends the detailed motion code and detailed motion control information to the intelligent robot 70.

Although it is assumed that the execution codes and execution files are previously stored in the intelligent robot 70, the execution codes and files can be stored in the emotion expression assistant system 80 as in the exemplary embodiment of FIG. 2. In this case, the emotion expression RAC server 83 sends the execution file to the intelligent robot 70 rather than sending the detailed motion code.

In this embodiment, the function blocks 3, 7, 47, 57, and 87 can be internetworked through peer-to-peer connections, Local Area Network (LAN), Wide Area Network (WAN) , or Integrated Service Digital Network (ISDN) .

FIG. 4 is a table illustrating an example of emotion expression code description information for use in an intelligent robot system according to an exemplary embodiment of the present invention.

Referring to FIG. FIG. 4, the emotion expression code description information can be provided in the form of a table having an emotional expression name, RAC emotion expression code, and RAC information fields.

The emotion expression name defines the emotional state to be expressed, and the RAC information describes the motion be performed.

In this embodiment, the emotion expression code information is stored in the emotion expression RAC DB 85 and, when requested, sent to the RAC production terminal 90. If the emotion expression code description information is received, the RAC production terminal 90 displays the emotion expression code description information such that the user can select one of emotional states provided by the emotion expression code description information. The RAC production terminal 90 requests the emotion expression assistant system 80 for the emotion expression information on the emotional expression selected by the user, and the emotion expression assistant system 80 sends the intelligent robot 70 the detailed motion code and information corresponding to the emotional expression requested by the RAC production terminal 90.

Of course, the execution file corresponding to the detailed motion code can be sent as in the exemplary embodiment of FIG. 2 instead of the detailed motion code. FIG. 5 is a table illustrating an example of E-M mapping information for use in an intelligent robot system according to an exemplary embodiment of the present invention. Referring to FIG. 5, the E-M mapping information can be provided in the form of a table having an RAC emotion expression code field and a detailed motion code field. For example, the emotion expression code "HRPOl" are mapped to seven detailed motion codes of F03, HOl, All, L03, L04, W05, and SOl. This means that the emotional state "pleasure 01" is expressed with a combination of seven detailed motions.

The E-M mapping information table further includes detailed motion control information field. The detailed motion control information field includes sub-fields such as performance time field for indicating performance time of the motion, number-of-motions field for indication a number of motions, and/or repetition field for indicating whether to repeat the motion. In this embodiment, the E-M mapping information is stored in the emotion expression RAC DB 45 such that, when the RAC client module 3 requests the E-M mapping information, the emotion expression RAC server 83 retrieves the detailed motion code and control information from the emotion expression RAC DB 45 with reference to the E-M mapping information and sends the detailed motion code and control information to the RAC client module 3. The RAC client module 3 extracts the execution code from the received detailed motion control information with reference to the previously stored M-P mapping information and acquires the execution file corresponding to the execution code. The RAC client module 3 performs the emotion expression according to the detailed motion control information while displaying the content .

FIG. 6 is a table illustrating an example of detailed motion code for use in an intelligent robot system according to an exemplary embodiment of the present invention.

Referring to FIG. 6, the detailed motion code can be provided in the form of a table. For example, the detailed motion code FOl indicates a normal facial expression, F03 indicates a smile facial expression, and F04 indicates a surprised facial expression. The detailed motion codes and names of motions given in FIG. 6 are only exemplary ones, however, the objects of the present invention can be achieved with other types of codes and names. For example, the table of FIG. 6 can be referred to when producing the E-M mapping information. FIG. 8 is a message flow diagram illustrating steps of generating an emotion expression RAC data file for use in an intelligent robot system according to an exemplary embodiment of the present invention. Referring to FIGs. 3 and 8, the RAC production terminal 90 sends a RAC production tool request message to the emotion expression assistant system 80. If the RAC production tool request message is received, the emotion expression assistant system 80 sends the RAC production tool indicated by the RAC production tool request message to the RAC production terminal 90.

Once the RAC production tool is received, the RAC production terminal 90 activates the RAC production tool. The RAC production tool sends an emotion expression information request message to the emotion expression assistant system 80. Upon receipt of the emotion expression information request message, the emotion expression assistant 80 sends the entire emotion expression information to the RAC production terminal 90. If the emotion expression information is received, the RAC production terminal displays the emotion expression information such that the user selects an emotional state to be expressed at a specific synchronization time point of the content . The RAC production terminal 90 creates a RAC data file informing of the content to be synchronized therewith and the time point at which the selected emotional expression is performed. Here, the RAC data file is generated in separation with the content. If a user input for requesting simulation is detection, the RAC production terminal 90 activates a simulator to execute simulation and displays the simulation result with selection options.

If the user selects a positive option, the RAC production terminal 90 creates the RAC data file and content. After creating the RAC data file successfully, the RAC production terminal 90 sends the RAC data file and content in response to a request from the intelligent robot 90. In the meantime, the RAC data file and content created by the RAC production terminal 90 can be stored in the intelligent robot 70. Although it is explained that the RAC data file and content is sent from the RAC production terminal^' 90 to the intelligent robot 70 through a network such as Internet in the above description, the RAC data file and content can be provided to the intelligent robot 90 in other various manners. For example, the intelligent robot system may further include a server for managing the RAC data files and contents by type of robot or content such that the intelligent robot 90 downloads the required content and RAC data file from the server. Also, the RAC data files and contents produced by the RAC production terminal 90 can be stored in a storage medium by type of robot and content such that the user provides the intelligent robot 90 with the RAC data files and contents by connecting the storage medium to the intelligent robot 90.

Referring to FIGs. 1 and 9, the RAC client module 3 first loads and displays the content read out from the RAC data storage (S901) . Next, the RAC client module 3 refers to the emotion expression synchronization information for checking the emotional expression to be presented with the display of the content (S903) and determines whether an emotion expression code is included in the emotion expression synchronization information (S905) .

If no emotion expression code is included in the emotion expression synchronization information, the RAC client module 3 disables the emotion expression function. Otherwise, if an emotion expression code is included in the emotion expression synchronization information, the RAC client module 3 sends an emotion expression information request message to the emotion expression RAC server 43 (S907) .

Upon receipt of the emotion expression information request message, the emotion expression RAC server 43 creates emotion expression information composed of a detailed motion code and detailed motion control information corresponding to the emotion expression indicated by the emotion expression information request message with reference to the previously stored E-M mapping information and sends the emotion expression information to the RAC client module 3 (S909 and S911) .

If the emotion expression information is received, the RAC client module 3 extracts the detailed motion code and detailed motion control information from the emotion expression information (S913) . Next, the RAC client module 3 retrieves an execution code corresponding to the detailed motion code with reference to the previously stored M-P mapping information (S917) and executes an execution file matched with the execution code according to the detailed motion control information (S919) .

Although exemplary embodiments of the present invention have been described in detail hereinabove, it should be clearly understood that many variations and/or modifications of the basic inventive concepts herein taught which may appear to those skilled in the present art will still fall within the spirit and scope of the present invention, as defined in the appended claims.

Claims

CLAIMS Claim 1

An emotion expression assistant system for an intelligent robot, comprising: an emotion expression robot action control database

(RAC DB) which stores emotion-motion (E-M) mapping information between emotion expression codes representing emotional states and at least one detailed motion codes matched with each emotion expression code and control information specifying the detailed motion code; and an emotion expression RAC server which creates, when the robot requests information on a specific emotion expression, the emotion expression information corresponding to the emotion expression with reference to the E-M mapping information and control information and sends the emotion expression information to the robot.

Claim 2

The emotion expression assistant system of claim 1, wherein the emotion expression RAC server retrieves the detailed motion code and control information with reference to the E-M mapping information and generates the emotion expression information composed of the detailed motion code and control information.

Claim 3

The emotion expression assistant system of claim 1, wherein the control information comprises at least one of a performing time, a number of performing times, and repetition option of the detailed motion.

Claim 4

The emotion expression assistant system of claim 1, further comprising an execution data storage which stores M-P mapping information containing mappings between the detailed motion codes and execution codes assigned for motions corresponding to the detailed motion codes and execution files corresponding to the execution codes, wherein the emotion expression RAC server retrieves the detailed motion code and control information corresponding to the emotion expression requested by the robot with reference to the E-M mapping information and control information, extracts execution code mapped to each detailed motion code, and generates the emotion expression information composed of an execution file corresponding to the execution code and the control information.

Claim 5

The emotion expression assistant system of claim 1, wherein the detailed motion code and execution codes are universal codes supported by robots regardless of types of the robots requesting the emotion expression information. Claim 6

The emotion expression assistant system of claim 1, wherein the emotion expression RAC DB stores emotion expression description information describing the emotion expression codes, and the emotion expression RAC server sends the emotion expression description information to the robot in response to a request from the robot .

Claim 7

The emotion expression assistant system of claim 1, wherein the detailed motion code indicates at least one of eye motion, mouth motion, head motion, arm motion, and swivel motion.

Claim 8

An intelligent robot system comprising: an intelligent robot which displays contents and expresses emotion states in association with the respective contents; an emotion expression Robot Action Control database

(RAC DB) which stores E-M mapping information containing mappings between emotion expression codes assigned to respective emotion states and detailed motion codes assigned to detailed motions and control information on the detailed motions; and an emotion expression RAC server which creates, when receiving a request for emotion expression information on an emotion state, the emotion expression information with reference to the E-M mapping information and the control information and sends the emotion expression information to the intelligent robot, wherein the intelligent robot performs an emotion expression according to the emotion expression information received from the emotion expression RAC server.

Claim 9

The intelligent robot system of claim 8, wherein the emotion expression RAC server extracts the detailed motion corresponding to the emotion state on which information is requested by the intelligent robot and the control information with reference to the E-M mapping information and the control information and generates the emotion expression information composed of the extracted detailed motion code and control information.

Claim 10

The intelligent robot system of claim 8, wherein the control information comprises at least one of a performing time, a number of performing times, and repetition option of the detailed motion.

Claim 11 The intelligent robot system of claim 8, wherein the intelligent robot comprises an execution data storage for storing M-P mapping information containing mappings between the detailed motion codes and execution codes assigned for motions corresponding to the detailed motion codes and execution files corresponding to the execution codes, wherein the emotion expression RAC server retrieves the detailed motion code and control information corresponding to the emotion expression requested by the robot with reference to the E-M mapping information and control information and sends the emotion expression information composed of the extracted detailed motion code and control information to the intelligent robot; and the intelligent robot extracts the detailed motion code and control information from the emotion expression information, retrieves the execution code mapped to each detailed motion code with reference to the E-M mapping information, and executes an execution file corresponding to the execution code.

Claim 12

The intelligent robot system of claim 8, wherein the detailed motion code and the execution code are universal codes supported by robots regardless of types of the robots requesting the emotion expression information. Claim 13

The intelligent robot system of claim 8, wherein the detailed motion code indicates at least one of eye motion, mouth motion, head motion, arm motion, and swivel motion.

Claim 14

The intelligent robot system of claim 8, wherein the intelligent robot comprises: a user interface for displaying the contents and expressing the emotion states of the intelligent robot; a storage for storing the contents and emotion expression synchronization information associated with the emotion states to be expressed in synchronization with the contents; and an RAC client module for controlling the user interface to display the contents stored in the storage and performs the emotion expression on the basis of the emotion expression synchronization information, wherein the RAC client module requests the ^"emotion expression RAC server for the emotion expression information with reference to the emotion expression synchronization information, displays the content corresponding to the emotion expression information received from the emotion expression RAC server, and controls the user interface to present the emotion expression. Claim 15

The intelligent robot system of claim 14, wherein the detailed motion code indicates at least one of eye motion, mouth motion, head motion, arm motion, and swivel motion.

Claim 16

The intelligent robot system of claim 14, wherein the emotion expression .synchronization information and the content are structured in separated files.

Claim 17

The intelligent robot system of claim 14, wherein the emotion expression synchronization information indicates a time point, at which the emotion expression starts in association of the content, and a type of the emotion expression.

Claim 18

The intelligent robot system of claim 13, wherein the intelligent robot system comprises: an RAC production tool storage for storing at least on RAC production tool; and an RAC production terminal for loading the RAC production tool from the RAC production storage and creating emotion expression synchronization information and the content with the execution of the RAC production tool.

Claim 19 The intelligent robot system of claim 18, wherein the emotion expression RAC DB stores code description information on the emotion expression codes, and the emotion expression RAC server sends the code description information read out from the emotion expression RAC DB in response to a request from the RAC production terminal .

Claim 20

The intelligent robot system of claim 19, wherein the RAC production terminal comprises: a display for displaying the code description information; an input module for receiving an option selected by a user with reference to the code description information; and an RAC data generator for generation the emotion expression synchronization information associated with the emotion expression selected by the user.

Claim 21

The intelligent robot system of claim 20, wherein the RAC production terminal comprises: a storage for storing the content and the emotion expression synchronization information generated by the RAC data generator; and a simulator for checking whether the content and emotion expression synchronization information stored in the storage are valid.

Claim 22

The intelligent robot system of claim 18, wherein the RAC production terminal generates the E-M mapping information and sends the E-M mapping information to the emotion expression RAC DB, or receives the E-M mapping information from the emotion expression RAC DB, modifies the E-M mapping information, and sending the modified E-M mapping information to the emotion expression RAC DB; and the emotion expression RAC DB stores the E-M mapping information received from the RAC production terminal.

Claim 23 The intelligent robot system of claim 14, wherein the user interface comprises a voice output module for outputting voice effect, a display for displaying the content, an input module for receiving user commands, and a physical motion module for driving physical motions of the intelligent robot.

Claim 24 The intelligent robot system of claim 8, wherein the emotion expression RAC server sends the E-M mapping information to the intelligent robot.

Claim 25

The intelligent robot system of claim 24, wherein the emotion expression RAC server and the intelligent robot communicates with each other periodically, or when required, for synchronizing the E-M mapping information.

Claim 26

An emotion expression assistant method for an intelligent robot, comprising: storing E-M mapping information containing mappings between emotion expression codes assigned to respective emotion states and detailed motion codes assigned to detailed motion codes assigned to detailed motions and control information on the detailed motions outside the intelligent robot; creating, when receiving a request for information on an emotion expression from the intelligent robot, emotion expression information corresponding to the emotion expression with reference to the E-M mapping information; and sending the emotion expression information to the intelligent robot, wherein the intelligent robot performs the emotion expression according to the emotion expression information.

Claim 27 The emotion expression assistant method of claim 26, wherein creating the emotion expression information comprises : retrieving the detailed motion code corresponding to the emotion expression and the control information on the detailed motion from the E-M mapping information; and generating the emotion expression information composed of the retrieved detailed motion code and control information.

Claim 28

The emotion expression assistant method of claim 26, further comprising storing M-P mapping information containing mappings between the detailed motion codes and execution codes assigned for motions corresponding to the detailed motion codes and execution files corresponding to the execution codes, wherein creating the emotion expression information comprises : retrieving the detailed motion code corresponding to the emotion expression and the control information on the detailed motion from the E-M mapping information; retrieving the execution code mapped to the retrieved detailed motion code from the M-P mapping information; and generating the emotion expression information composed of an execution file corresponding to the retrieved execution code and the control information.