WO2006135209A1

WO2006135209A1 - Interaction between mobile robot and user, system for same

Info

Publication number: WO2006135209A1
Application number: PCT/KR2006/002303
Authority: WO
Inventors: Changsu Lee; Seongho Won; Kyounghee Han; Shinyoung Park; Jongkyu Lim
Original assignee: Sk Telecom Co., Ltd.
Priority date: 2005-06-16
Filing date: 2006-06-15
Publication date: 2006-12-21
Also published as: KR100678728B1; KR20060131458A

Abstract

Disclosed is a method and a system for inviting a user to use various kinds of functions using a mobile robot that recognizes and tracks the user. The method comprises the steps of (a) generating an instruction message inviting the user to perform a function so as to transmit it to the mobile robot; (b) identifying the content of the instruction message, and then storing it in a memory if the mobile robot receives the instruction message; (c) moving the robot so as to search for the user if execution time comes near if the instruction message is of an active mode; (d) outputting a guidance message inviting the user to perform the function if the user has been found before the execution time; (e) executing the function, and transmitting the result to the service server if the user requests execution of the invited function as a result of step (d); (f) identifying whether or not the user is nearby when the instruction message is in a semi-active mode; (g) outputting a guidance message inviting the user to perform the function if the user is nearby in step (f); and (h) executing the function, and transmitting the result to the service server if the user requests execution of the invited function as a result of step (g).

Description

INTERACTION BETWEEN MOBILE ROBOT AND USER, SYSTEM FOR SAME

Technical Field

The present invention relates to a method and a system for interaction between a mobile robot and a user, and more particularly to a method and a system for inviting a user to use various kinds of functions using a mobile robot that recognizes and tracks the user.

Background Art

With the development of computer technologies, mobile robots such as pet robots have been recently developed. Pet robots sold by Sony Corporation have attracted much interest by operating as real pets with considerable intelligence, and cat robots released by Matsushita Electric Industry have displayed an advanced technical ability to some degree in which they can have a conversation with people.

Further, the pet robot may be enabled to dance, and a user can view objects from the viewpoint of the pet robot with a camera built therein. Furthermore, the pet robot is provided with a precise sensor, a remote control device and the like so that it can walk by itself, respond to clap instructions, and perform feats such as hand-standing and

_ -j doing push-ups.

Recently, networked robots, with a variety of functions, together with toys and a mail identification function by connecting to the Internet, have recently been appearing in great numbers. Such robots may be used to link with a household computer and a high-speed Internet network, which are widespead, and provide a function that existing electronic products do not possess among functions required in everyday life. Such a conventional robot is operated whereby a user instructs the robot to execute a function, or its function is executed in the form in which the user remotely controls the robot using a remote control. Since the robot is passively operated by the user, this consequently causes a result in that the robot does not actively provide a new function to the user. However, since the robot is able to move freely, it can invite the user to use the new function, or directly perform the new function.

Disclosure of the Invention

Accordingly, the present invention has been made to solve the above-mentioned problems occurring in the prior art, and an object of the present invention is to provide a method and a system for interaction between a user and a mobile robot that recognizes the user and tracks the user so as to invite and induce the user to use various kinds of new functions .

According to an aspect of the present invention, there is provided a system for interaction between a mobile robot and a user, comprising: a database for storing information for authorization, subscriber's robot ownership information and user's function performance history information; a service server containing a scheduler for generating an instruction message containing a function for inviting the user using the user's function performance history information, other factors and execution time information; and a mobile robot for communicating with the service server through wireless Internet, operating in accordance with the instruction message transmitted from the service server, and transmitting the performance result to the service server, wherein the mobile robot searches for the user before the execution time so as to invite the user to perform the function in a case where the instruction message is an instruction in active mode, and the mobile robot invites the user to perform the function if the user is nearby in a case where the instruction message is an instruction in a semi- active mode.

In accordance with another aspect of the present invention, there is provided a method for interaction between a mobile robot and a user in a system having a service server and the mobile robot, comprising: (a)

_ O _ generating an instruction message inviting the user to perform a function so as to transmit it to the mobile robot;

(b) identifying the content of the instruction message, and then storing it in a memory if the mobile robot receives the instruction message; (c) moving the robot so as to search for the user if execution time comes near if the instruction message is of an active mode; (d) outputting a guidance message inviting the user to perform the function if the user has been found before the execution time; (e) executing the function, and transmitting the result to the service server if the user requests execution of the invited function as a result of step (d) ; (f) identifying whether or not the user is nearby when the instruction message is in a semi-active mode; (g) outputting a guidance message inviting the user to perform the function if the user is nearby in step (f) ; and (h) executing the function, and transmitting the result to the service server if the user requests execution of the invited function as a result of step (g) .

Brief Description of the Drawings

The above and other objects, features and advantages of the present invention will be more apparent from the following detailed description taken in conjunction with the accompanying drawings, in which:

FIG. 1 is a block diagram showing an entire configuration of a system according to the present invention;

FIG. 2 is a block diagram showing a detailed configuration of a mobile robot according to the present invention; and

FIG. 3 is a flowchart illustrating a process of interaction between a user and the mobile robot according to the present invention.

Best Mode for Carrying Out the Invention

Reference will now be made in detail to the preferred embodiment of the present invention. It should be noted that the similar components are designated by similar reference numerals although they are illustrated in different drawings. Also, in the following description, a detailed description of known functions and configurations incorporated herein will be omitted when it may obscure the subject matter of the present invention. FIG. 1 is a block diagram showing an entire configuration of a system according to the present invention.

A service server 11 communicates with a mobile robot

14 through a wireless Internet network. The service server

11 includes a scheduler 12. The scheduler 12 provides an instruction message for control of the mobile robot. That is, in a case where a user intends to operate the mobile robot in an active mode, the scheduler 12 provides execution time to invite the user to use a function and information on a recommendation function and the like. Further, in a case where the user intends to operate the mobile robot in a semi-active mode, the scheduler 12 provides information on a recommendation function. In a user database (DB) 13 are stored the user's personal information, information for authorization and information on the kind of a mobile robot possessed by a receiver and the like. Moreover, in the user DB 13 are stored information frequently used by the user and a user's information utilization form. For example, if the user executes the function of listening to music in the active mode (the mode in which the robot moves to search for the user), such records are stored in the user DB 13. The service server 11 sends an instruction message to control of the mobile robot, which is produced by the scheduler 12, to the mobile robot 14 through the wireless Internet network. The service server 11 provides various required information to the mobile robot, and performs the function of collecting a variety of information from the mobile robot 14 as well as such functions.

There may be three modes for interaction between the mobile robot 14 and the user. One is an active mode in which the robot searches for the user to invite the user to perform a function through an instruction from the service server 11, and another is a semi-active mode in which the robot invites the user to perform a function through an instruction from the service server 11 in a case where the user is detected to be in the vicinity of the robot. The last is an execution mode in which the user executes a function desired by the user themselves through a user interface of the robot. Here, since the user execution mode is a general technique, descriptions thereof will be omitted in the present invention.

FIG. 2 is a block diagram showing a detailed configuration of a mobile robot according to the present invention.

The mobile robot, according to the present invention, includes: a controller 21 for controlling the whole operations; a radio communication module 22 for setting a wireless communication link with the service server 11 under the control of the controller 21; a radio receiver 24 for user recognition, which receives a radio signal containing users' unique information, and calculates a distance from the mobile robot to the user and an angle so as to provide them to the controller 21 for the purpose of recognizing the user; an action and movement controller 23 for receiving a control instruction from the controller 21 so as to move the mobile robot or to control the movement thereof; a media reproducer 25 for reproducing a multimedia message under the control of the controller 21; a voice recognizer 28 for recognizing a user's voice; an input unit 26 for receiving a variety of data from the user; a display unit 27 for displaying a variety of data as characters and images; a memory 29 for arbitrarily storing a variety of data; and a hard disk (HDD) 30 for storing programs and data. The mobile robot, according to the present invention, can identify a user. That is, the user possesses a transmitter for transmitting their unique identifier. It is desirable that the transmitter be attached to a thing that the user always carries on their person. Preferably, the transmitter is a radio transmission device having a transmission output from about 5 to 10m. The mobile robot 14 receives a radio signal transmitted from the transmitter possessed by the user, and then identifies their unique identifier so that the mobile robot recognizes the user to be a possessor of its own. In other words, even if several radio signals are received, the mobile robot identifies a unique identifier obtained through demodulation of the received radio signals so that the mobile robot can recognize a substantial possessor, and estimate the location thereof.

To this end, the mobile robot, according to the present invention, is provided with the radio receiver 24. The radio receiver 24 for user recognition calculates the distance from the robot to the user and the angle from a radio signal received from a radio transmitter possessed by the user, and demodulates the radio signal so as to provide them to the controller 21. The controller 21 provides information on moving direction, velocity and acceleration to the action and movement controller 23 on the basis of information on the distance from the user and the angle transmitted from the radio receiver 24 for user recognition. The action and movement controller 23 controls the mobile robot in accordance with the information on moving direction, velocity and acceleration, which is provided by the controller 21. The radio communication module 22 sets a radio communication link with the service server 11 under the control of the controller 21. Here, the radio communication module 22 may be a general mobile communication modem or wireless Internet modem. In the hard Disk (HDD) 30 are stored programs for operating the mobile robot, various information, information on an indoor map and various kinds of data. The display unit 27 may be a general Liquid Crystal Display (LCD) , and displays information related to operations of the mobile robot and received information (multimedia information) . The input unit 26 may be a touch screen or keypad through which the user inputs data. The media reproducer 25 includes a general video and audio codecs.

If the controller 21 receives an interaction instruction (operation control instruction in an active or semi-active mode) message from the service server 11, it identifies the received instruction message. In the instruction message is contained a mode identifier for identifying whether a transmitted instruction is in active or semi-active mode. The controller 21 directly searches for the user so as to invite the user to use a function in the active mode, and it invites the user to use the function if the user is detected to be nearby in the semi-active mode.

Further, if the user performs a requested function or a specific function, the controller 21 provides information on the function performed by the user to the service server 11.

The service server 11 collects user functional performance information from each mobile robot such that it can grasp the most suitable operation mode for a specific function for each user. For example, the service server 11 estimates that user A prefers performing listening to music in the active mode and taking picture in the semi-active mode through the collected information from each of the mobile robots, while user B prefers performing listening to music in the semi-active mode and taking picture in the active mode. Additionally, the service server 11 estimates the content of a recommendation time and function for the user in the active mode and that of a recommendation function in the semi-active mode through the collected information. The service server 11 sends an instruction for performing the most suitable function for the user to the mobile robot, reflecting the estimation results in such a manner.

The scheduler of the service server sends an operational instruction of the active mode, which contains execution time, a performance function and other factors, to the mobile robot. At this time, the scheduler should send the operational instruction of the active mode prior to performance time. If the scheduler fails to transmit data, it resends the operational instruction before reaching the performance time. Further, the scheduler sends the operational instruction, which contains the performance function and other factors, to the mobile robot. Since the recommendation function of the semi-active mode is not sensitive to transmission time, the robot invites the user to use the function if the mobile robot senses the user after the time when the operational instruction has been sent .

Next, a detailed operation of the controller of the mobile robot will be described with reference to FIG. 3. FIG. 3 is a flowchart illustrating a process of interaction between a user and the mobile robot according to the present invention.

First, if the controller receives an interaction instruction message from the service server in a standby state (101), it identifies the received instruction message, and stores the instruction message in the memory (102). Since a mode identifier is stored in the instruction message, the controller identifies whether the instruction received through the mode identifier is of the active or semi-active mode (103) . In the active mode, the controller identifies whether or not the execution time of the operational instruction stored in the memory is approaching in the standby state (104). If the execution time is approaching (105), the controller causes the action and movement controller to start the movement of the robot for the purpose of searching for a user (106). For example, the controller starts the movement of the robot to search for the user 20 minutes before the execution time.

The process of the mobile robot searching for the user is performed as follows. The controller sends a control instruction to the action and movement controller such that the mobile robot can move along a predetermined pathway. Further, the controller identifies a user' s unique identifier from a radio signal received through the radio receiver for user recognition. In a case where the user's unique identifier is identical with that of a stored possessor, the controller recognizes that it has found the user.

If the controller identifies the user before execution time (107), it outputs a voice message inviting the user to execute a performance function contained in the operational instruction (108). For example, the mobile robot invites the user to perform a function in a form such as "Would you like to listen to music?"

If the user responds by executing the function in response to the function performance invitation (109) , the controller starts the function performance (110). As for the function performance, there may be function performance through communications with the service server and function performance in the mobile robot itself. Meanwhile, if the mobile robot does not find for the user before the execution time, it deletes the sent instruction of the active mode, and informs the service server of instruction execution failure. Further, if the user does not perform the invited function, the mobile robot transmits the fact to the service server.

Meanwhile, if the stored instruction is of the semi- active-mode type (103), in a standby mode (111), the controller identifies whether or not there exists a user nearby (112). If there exists no user, the controller continues to stand by, and if there exists a user nearby, the controller outputs a voice message to invite function execution (113) . For example, the mobile robot invites the user to perform the function of having a picture taken in a form such as "What about having a picture taken?" If the user requests the performance of the function in response to the function execution invitation (114), the controller executes the requested function (115) . Similarly, if the user does not request the performance of the invited function, the controller sends the fact to the service server. Further, in a case where the user performs the invited function or voluntarily performs a specific function, the controller transmits the performed function to the service server.

As described above, the present invention has an advantage in that a mobile robot performs only a function requested by a user, and besides, invites the user to perform the function using user' s function performance history information, so that various and intimate interaction between the user and the robot is possible. Further, there is an advantage in that a variety of interaction modes are provided for one function so that users' preference can be reflected, and various and intimate interaction between the user and the robot is possible.

Although a preferred embodiment of the present invention has been described for illustrative purposes, those skilled in the art will appreciate that various modifications, additions and substitutions are possible, without departing from the scope and spirit of the invention as disclosed in the accompanying claims.

Claims

1. A system for interaction between a mobile robot and a user, comprising: a database for storing information for authorization, subscriber' s robot ownership information and user' s function performance history information; a service server containing a scheduler for generating an instruction message containing a function for inviting the user using the user' s function performance history information, other factors and execution time information; and a mobile robot for communicating with the service server through wireless Internet, operating in accordance with the instruction message transmitted from the service server, and transmitting the performance result to the service server, wherein the mobile robot searches for the user before the execution time so as to invite the user to perform the function in a case where the instruction message is an instruction in active mode, and the mobile robot invites the user to perform the function if the user is nearby in a case where the instruction message is an instruction in a semi- active mode.

2. The system as claimed in claim 1, wherein the mobile robot comprises: a radio communication means for connecting to the service server by radio; a user recognition means for receiving a signal transmitted from a transmitter possessed by the user so as to recognize a possessor, and measuring direction and distance to the user; a voice recognition means for receiving a voice from the user to recognize it; a media reproduction means for reproducing a multimedia message; a memory means for storing programs and various kinds of information; an action and movement control means for controlling action and movement of the robot; and a control means for analyzing an instruction message if the instruction message is received from the service server through the radio communication means so as to transmit a control signal to the action and movement control means for the purpose of searching for the user before the execution time in active mode, and to output a guidance message for inviting the user to perform the function if the user is recognized through the user recognition means.

3. The system as claimed in claim 2, wherein the control means transmits a control signal to the action and movement control means so as to allow the robot to move along a previously stored pathway.

4. The system as claimed in claim 2, wherein the control means transmits result information on whether or not the user has performed the function contained in the instruction message and function performance information executed by the user themselves to the service server.

5. The system as claimed in claim 4, wherein the scheduler of the service server generates an instruction message inviting the user to use the function on the basis of the user' s execution history information transmitted through the control means of the mobile robot.

6. The system as claimed in claim 2, wherein, in a case where the transmission of an instruction message fails, the service server retransmits the instruction message before the execution time.

7. The system as claimed in claim 4, wherein the control means deletes the received instruction message, and transmits the fact to the service server if it does not find the user before the execution time.

8. A method for interaction between a mobile robot and a user in a system having a service server and the mobile robot, comprising: (a) generating an instruction message inviting the user to perform a function so as to transmit it to the mobile robot;

(b) identifying the content of the instruction message, and then storing it in a memory if the mobile robot receives the instruction message;

(c) moving the robot so as to search for the user if execution time comes near if the instruction message is of an active mode;

(d) outputting a guidance message inviting the user to perform the function if the user has been found before the execution time;

(e) executing the function, and transmitting the result to the service server if the user requests execution of the invited function as a result of step (d) ; (f) identifying whether or not the user is nearby when the instruction message is in a semi-active mode; (g) outputting a guidance message inviting the user to perform the function if the user is nearby in step (f) ; and (h) executing the function, and transmitting the result to the service server if the user requests execution of the invited function as a result of step (g) .

9. The method as claimed in claim 8, wherein the instruction message contains an identifier for identifying the mode, and contains execution time, a performance function and other factor information when the instruction message is of the active mode, and contains the performance function and the other factor information in a case where the instruction message is in the semi-active mode.

10. The method as claimed in claim 8 or 9, further comprising a step of deleting the stored instruction message unless the user has been found before the execution time at step (d) .

11. The method as claimed in claim 10, wherein the guidance message for inviting the user to perform the function is a voice message.