JP3318539B2 - Autonomous mobile robot with dialogue system - Google Patents

Description

DETAILED DESCRIPTION OF THE INVENTION

[0001]

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention relates to an autonomous mobile robot, and more particularly, to an instruction word ("Achi", "Achi",
The present invention relates to a new self-contained mobile robot that interprets "over there" and "over here" to determine a moving direction.

[0002]

2. Description of the Related Art Self-standing mobile robots can move independently and freely by, for example, wheels or legs, and are expected to be applied to, for example, pet robots and nursing robots. In this type of application field, a situation is considered in which the user instructs the moving direction of the robot or the like by using a directive, and the robot interprets the directive so that the robot determines its own traveling direction.

[0003]

In such a case, the degree of freedom of the positional relationship between the user and the robot is large, and therefore, there are a plurality of viewpoints that can be selected by the user when speaking. The speaker's or user's perspective greatly affects the meaning or interpretation of the referent. For example, there is a case where an instruction word “here” issued when the user faces the robot has a different meaning from an instruction word “here” issued while the user is running in parallel with the robot.

[0004] Therefore, a main object of the present invention is to provide a self-contained mobile robot capable of properly interpreting a directive from a user.

[0005]

A self-contained mobile robot according to the present invention is a self-contained mobile robot having an interactive system that interprets a command from a user and determines a moving direction, and is uttered by the user. Descriptive word capturing means for capturing a descriptive word, viewpoint determining means for determining at least one of a face-to-face viewpoint, a bird's-eye view viewpoint, or a shared viewpoint based on at least a positional relationship with a user, and interpreting means for interpreting the descriptive word at the viewpoint A self-contained mobile robot.

[0006]

When the user utters a descriptive word such as "this way", "this way" or "that way", the robot takes in the descriptive word and acquires the positional relationship between the robot and the user and the moving speed of the user and the robot. Based on the moving direction and the moving direction, a user viewpoint of one of a face-to-face viewpoint, a bird's-eye view viewpoint, and a shared viewpoint is determined at a predetermined probability determined by a rule. Based on the determined user viewpoint,
Interpret directives. In accordance with the interpreted descriptive word, for example, the target direction is determined from the definition table, and the direction is moved to that direction.

However, when the interpretation of the descriptive word is incorrect, another viewpoint is selected.

[0008]

According to the present invention, the user's viewpoint is inferred in accordance with the positional relationship between the robot and the user, and the descriptive term is interpreted based on the viewpoint. Can be significantly reduced. Therefore, according to the present invention, it is possible to accurately interpret the instruction word from the user.

[0009] Other objects, features and advantages of the present invention will become more apparent from the following detailed description of embodiments taken in conjunction with the accompanying drawings.

[0010]

FIG. 1 is a front view showing an autonomous mobile robot 10 according to an embodiment of the present invention. Referring to FIG. 1, the autonomous mobile robot of this embodiment (hereinafter simply referred to as "robot"). 10 includes a main body or a housing 12,
A wheel 14 is provided at a lower portion of the wheel so as to be rotatable around a shaft 16. The wheels 14 or shaft 16 are driven by a motor 36 (FIG. 2), so that the robot 10 can move in any direction. However, although not shown, a power transmission mechanism capable of independently controlling the left and right wheels, such as a differential gear, is connected to the shaft 16 or the wheel 14.
Incorporated in connection with. Furthermore, the left and right wheels 14
Two motors may be used to independently control. Since such a drive system itself does not have a characteristic, other configurations of the drive system are conceivable.

The housing 12 is formed in a substantially rectangular plane, and an ultrasonic sensor 18 is disposed on four sides thereof. Each ultrasonic sensor 18 is a combination of a transmitter and a receiver, and a microcomputer 26 (FIG. 2), which will be described later, depends on the time at which the ultrasonic wave output from the transmitter is received by the receiver. The position of the robot 10 in the room, the presence of an obstacle, and the like are detected.

An image sensor 20 having, for example, a CCD camera is provided on one side surface of the housing 12, and this image sensor 20 is used as a user tracking sensor.
The user tracking sensor captures an image of the user, for example, and the microcomputer 26 extracts the skin color region, and detects the positional relationship between the user and the robot 10 based on a positional change of the skin color region or a change in the area of the skin color region. It is.

Further, a microphone 22 and a speaker 24 are provided at an appropriate position, for example, the upper surface of the housing 12. The microphone 22 captures the voice generated by the user, and the speaker 24 generates the voice from the robot 10 to the user. Thus, the robot 10 of this embodiment is configured as a robot having a dialogue system.

Referring to FIG. 2, a microcomputer 26 is provided in housing 12 of FIG. Although one microcomputer 26 is illustrated in FIG. 2, a plurality of microcomputers may be provided as necessary, and tasks such as image processing, audio processing, and drive control may be shared among the microcomputers. However, in the following description, one or more microcomputers are represented by the microcomputer 26 for convenience.

The microcomputer 26 includes the ultrasonic sensor 18 and the image sensor 20 described with reference to FIG.
And an input from the microphone 22 and an audio signal to the speaker 24. Although not shown in FIG. 1, the robot 10 is further provided with an encoder 32 and a compass 34. The encoder 32 is the left and right wheels 1
4 and input into the microcomputer 26 pulse signals of a number corresponding to the number of rotations of each wheel 14. The microcomputer 26 counts the pulse signals from the respective enders 32 and calculates the speed at which the robot 10 is moving and the position at which the robot 10 changes every moment. The compass 34 is for knowing the azimuth (moving direction) of the robot 10.

The robot 1 of the embodiment shown in FIGS. 1 and 2
0 interprets a command uttered by the user through the microphone 22, for example, "that", "that", "this", etc.
It moves independently in the direction according to the instruction. At this time, the robot 10 uses the speaker 24 to request the user to confirm whether or not the moving direction is appropriate, for example, “this is it?”, “Is that?”, “Is that?”.
Or "Which is it?"

Here, referring to FIGS. 3 and 4, the usage of the descriptive word will be reconfirmed. The situation in FIG. 3 is a state in which the speaker and the listener face each other. In this state, “here” is the direction of the speaker, “that” is the direction of the listener, and “that” is both the speaker and the listener. It means the direction away. In the situation shown in FIG. 4, the speaker and the listener are running side by side, and in this state, the instruction word indicates the same direction. For example, if one says "here" to indicate the direction we are heading, the other can also understand that "this way" is the direction we are heading.

That is, the examples shown in FIGS. 3 and 4 show that the instruction word changes the instruction method under the influence of the viewpoint in addition to the external situation. The dependency relationship between the meaning of the utterance and the viewpoint shown in this example can be expressed by the following rule 1.

[0019] Rule 1: p ← s ○ _i d here, p is the object pointed to by the directive, s is the situation at the time of the utterance, d represents each spoken directive to. ○ _i represents a viewpoint for interpreting the descriptive word, and the subscript i is a symbol representing who the viewpoint is.

According to this rule 1, the ability required for the robot 10 to understand the instruction word is not only the ability to detect the situation s but also the ability to determine the viewpoint o _i of the speaker.
The situation s is, specifically, a situation that the robot 10 has learned or given in advance, such as the shape and size of the room where the robot 10 is located, the position of the door of the room, the goal (target position), and the obstacle. And so on. Such a situation s is shown in FIG.
Are preset in a ROM 28 of a microcomputer 26 shown in FIG.
As a result of detection at 2, 34, R of the microcomputer 26
Stored in AM30.

In the embodiment shown in FIG. 1, a viewpoint selecting method using physical constraints is adopted in order to cause the robot 10 to select an accurate viewpoint when interpreting or generating a descriptive word. The relationship between the physical constraint and the viewpoint appears in the difference in the usage of the descriptive word in FIGS. 3 and 4, and in the difference in the physical relationship between the speaker and the listener. That is, in the example of FIG. 3, the speaker and the listener face each other, and the instruction word “here” indicates that the speaker is dedicated. In FIG. 4, the two persons are walking together, and the target direction is indicated by the instruction word “here”. The viewpoint selection method of the robot 10 in this embodiment uses such a difference in physical relationship for inferring a viewpoint at the time of a user's utterance.

Here, as shown in FIG. 5, the room used by the inventors for the experiment of the robot 10 is a substantially square room of 4 m square, with the door part starting and the goal ( Target position) is set. The user was given the task of bringing the robot 10 from the start to the goal using a directive.

The room is divided into four blocks, and each block is provided with an IR (infrared) sensor 38. With this IR sensor 38, it is possible to know in which block the robot 10 is located. it can. That is, the output of the IR sensor 38 is sent to the robot 10 by wire or wirelessly. The robot 10, that is, the microcomputer 26, can know which block in the room it is based on the outputs of the four IR sensors 38.

The above-mentioned physical constraint is a constraint that accompanies a positional relationship and an action (collectively, "physical relationship") between the speaker and the listener, and can be defined by the following rule 2.

Rule 2: E (A _u (V _u , D _u ), _Ar (V
_r, at D _r)) → ○ _i Here, A _u is user behavior, A _r represents the behavior of the robot 10, V _i and D _{i (i} = u or r), respectively, the movement of the user or robot The speed and the moving direction are shown.

When this rule 2 is applied, for example, when the user and the robot face each other, E (A
_u (0, robot), _Ar (0, user)). When the user and the robot is moving together in one speed X _{is, E (A u (X,} goal), A r (X, goal)) can be expressed as. That is, it can be understood that Rule 2 is a constraint on the viewpoint ○ _i based on the physical relationship.
Note that the user's action A _u (V _u , D _u ) is determined by the user tracking described above using the image sensor 20 in the RAM.
30. Robot behavior A _r (V _r , D _r )
Are stored in the RAM 30 by robot monitoring using various sensors including the ultrasonic sensor 18, the encoder 32, and the compass 34.

In the room of FIG. 5, the user can take three viewpoints. Viewpoint D when facing the robot
V (Daily View: face-to-face viewpoint), bird's-eye view viewpoint BV (Bird's Eye View) viewed from above the user, and viewpoint SV (Shared View: shared viewpoint) common to the robot. The viewpoints DV and SV are established in the situations of FIGS. 3 and 4, respectively, and are both viewpoints based on the relationship between the user and the robot. On the other hand, the viewpoint BV is a viewpoint based on the relationship (mainly, distance) between the user and the room.
In this case, the speaker would call the direction farther away from him,
An instruction word is selected according to the distance so that the near direction is expressed as “here”.

In the viewpoint selection of the robot 10, that is, the microcomputer 26, a physical constraint established between these three viewpoints and the physical relationship is used. for that reason,
RO of the robot 10, that is, the microcomputer 26
In M28, a definition table shown in the following table, which indicates a relationship between a viewpoint and a designated direction of each designated word, is set. In addition, "-" in the table indicates that the direction does not apply to the instruction direction of another instruction word.

[0029]

[Table 1]

The appearance rates of the respective viewpoints to which the above rule 2 is applied, confirmed by the experiments of the inventors, are as follows.

Rule 3: E (A _u (0, robot), A
_r (0, user)) → DV: 57.9%, BV: 37.7
%, SV: 4.4% Rule _{4: E (A u (X} , goal), A r (X, goal)) →
DV: 48.3%, BV: 22.4%, SV: 29.3
% The terms to the right of the arrows in rules 3 and 4 are DV, B
Which viewpoint of V and SV is the viewpoint ○ _i
Or appear in the This appearance probability is based on a value actually checked in an experiment. For example, rule 3 is
In the situation s in which the user and the robot face each other, _ii indicates that DV is likely to occur. Rule 4
Indicates that the viewpoint SV is likely to appear in the situation s in which the user and the robot are running side by side.

In the embodiment, the following rules are used for physical constraints other than rules 3 and 4. The reason is that the user's viewpoint is almost always DV or BV
It is derived from that.

Rule 5: E (A _u (X, X), _Ar (X,
X)) → DV: 50%, BV: 50%, SV: 0% In the embodiment, the robot 10 acquires robot action data A _r (V _r , D _r ) based on data from various sensors, User behavior data A _u (V _u , D _u ) is obtained based on data from the image sensor. Then, the aforementioned rule 2 physical restriction is selected. Then, at the time of the processing of the interpretation of the directive and the processing of the generation of the directive, the rules 3 to 5 of the physical restriction are applied. In other words, depending on the physical relationship,
Weighting is performed on the viewpoint used in the instruction word processing according to the appearance probability.

As an example, when Rule 3 is applied, the microcomputer 26 generates a random number using a random function, obtains the modulo 1000 of the random number, and selects the viewpoint DV when the result is “0-579”. , "580
In the case of “−957”, the viewpoint BV is selected and “958-9” is selected.
When "99", the viewpoint SV is selected. Similarly, rule 4
Is applied, the microcomputer 26 calculates the modulo 1000 of the random number, selects the viewpoint DV when the result is "0-483", selects the viewpoint BV when the result is "484-707", and selects "708-999". When the viewpoint S
Select V. However, when Rule 5 is applied, the microcomputer 26 generates a random number using a random function, and selects the viewpoint DV or BV depending on whether the random number is even or odd.

In the process of interpreting the descriptive term, the robot 10 or the microcomputer 26 determines the descriptive term d spoken by the user according to s in a situation based on the user's viewpoint _{u u} and the information from various sensors inferred as described above. Interpret Therefore, in inferring the user viewpoint, the viewpoint is stochastically selected in accordance with the ratio of appearance of viewpoints in the rules 3 to 5 of the physical constraint. The interpretation of the instruction word reflecting the physical relationship between the two obtained by the user's 20 moving speed and direction (obtained from the output of the encoder 22 and the like) and the robot behavior data (the moving speed and direction of the robot known from the output of the encoder 22) is performed. In addition,
Rule 2 is a general expression of rules 3-5.

The operation of the robot 10, that is, the microcomputer 26 will now be described with reference to FIG. When the user utters the instruction word by the microcomputer 26, it is sensed by the microphone 22 (FIGS. 2 and 3), and the microcomputer 26 starts executing the routine of FIG.

In the first step S1 in FIG. 6, the microcomputer 26 acquires data of the situation s at the time of the user's utterance according to the rule 1, for example, the positional relationship of the robot position, start, goal and the like. In the subsequent step S2, as described above, based on the outputs from the various sensors, the microcomputer 26 determines the user action data _Au (the moving speed and direction of the user) and the robot action according to Rule 2, that is, Rule 3-5. Data A _r (moving speed and direction of the robot) is acquired, and based on them, the viewpoint of the user is determined according to Rule 3-5 described in detail above.
Determine _u . After the user viewpoint is selectively determined in step S2, in step S3, the microcomputer 26 determines the target p in rule 1.

For example, the user and the robot face each other.
If the user behavior data is A _u(0, robot)
And the robot behavior data is A_r(0, user). I
Therefore, in step S2, according to rule 3, ○_u
= DV becomes easier to select. Therefore, the user's directive
Is “this way”, the robot 10 moves to the viewpoint DV
And according to Table 1 given above,
"P = user", that is, the moving direction is determined.

For example, when the user and the robot are running in parallel at a speed of 10, the user action data is A
_u (10, goal), and the robot behavior data is A _r (1
0, goal). Therefore, in step S2, it is easier to select _{u u} = SV according to rule 4. Therefore, when the user's designated word is “here”, the robot 10 determines “p = goal” as the target p based on the viewpoint SV and according to Table 1 described above, and determines a moving direction for that. Will do.

In step S4, the microcomputer 26 gives a command to the motor 36 and the like in accordance with the target p determined in this way, that is, the moving direction. In response to the command, the robot 10 moves in the moving direction, that is, the direction of the user.

Thereafter, in step S5, the microcomputer 26 waits for a reply from the user for a predetermined time. The user replies within a certain period of time whether the robot 10 is moving in the correct direction in response to the instruction issued by the user. When there is a reply from the user, the microcomputer 26 determines whether or not the reply is "No" in step S6.

When the response from the user is "no", that is, when the robot 10 erroneously interprets the instruction word from the user, the microcomputer 26 executes the previous steps S2 to S5 again. At this time, the microcomputer 26 applies rule 3 again, selects another viewpoint again, and interprets the directive. That is, if the interpretation of the descriptive word is incorrect, the robot 10 reselects the viewpoint, and interprets the descriptive word according to Table 1 according to the changed viewpoint.

As described above, in the embodiment, if the interpretation of the descriptive term in the robot 10 is incorrect, the descriptive term can be immediately reinterpreted from the viewpoint of the next candidate. Therefore, it is advantageous for redetermining the instruction target. In addition, since the re-determined instruction target can be listed as a candidate for the next instruction word interpretation, the instruction word can be interpreted promptly even when the user designates again with another instruction word.

If the interpretation of the instruction in the robot 10, that is, the moving direction is not wrong, the robot 10 waits for the input of the instruction from the user again.

In the embodiment of FIG. 6 described above, the robot 10 interprets the user's instruction and moves as it is in the direction of the determined object. However, in the embodiment of FIG. 7, when interpreting the user's instruction, the robot 10 can generate and speak the instruction and confirm the moving direction to the user.

Referring to FIG. 7, which is another embodiment of the present invention, steps S11 to S13 in FIG. 7 are the same as steps S1 to S3 in FIG. In step S11, similarly to step S1 in FIG.
Acquires the data of the situation s at the time of the user's utterance in rule 1. In the following step S12, the steps in FIG.
S2, in the same manner as, on the basis of the output from the various sensors, the microcomputer 26, the rules 3-5, acquires the user behavior data A _u and the robot behavior data A _r, based on their rules 3- According to 5, the user's viewpoint 視点_u is determined. After the user viewpoint is selectively determined in step S12, in step S13, the microcomputer 26 determines the target p in rule 1 in the same manner as in step S3 in FIG.

Thereafter, in step S14, the robot 1
0 generates a directive at the viewpoint _Or of the robot with respect to the target p determined in this way.

For example, the user and the robot face each other.
If the user behavior data is A _u(0, robot)
And the robot behavior data is A_r(0, user). I
Therefore, in step S14, according to rule 3,
○ As a bot's point of view_r= DV becomes easier to select. did
Therefore, if the user's directive is, for example,
The target p determined in step S13 is, for example, “p = user”
If the microcomputer 26 is
Viewpoint ○_r= Based on DV, the instruction word for confirmation
Right? ”Is generated, and the generated directive is
Through 24, the user speaks.

In the face-to-face viewpoint DV, if the instruction uttered by the user is "that", "p = robot" is determined as the target p in step S13, and the instruction generated by the robot 10 is therefore generated. term, in order to on the basis of the robot viewpoint O _r, is "It is here?".

For example, when the user and the robot are running in parallel at a speed of 10, the user action data is A
_u (10, goal), and the robot behavior data is A _r (1
0, goal). Therefore, in step S14,
According to Rule 4, it becomes easy to select ○ _r = SV as the viewpoint of the robot. Therefore, the user's descriptive word is “here” and “p = goa
When “l” is determined, in this case, the robot 10 generates and speaks the instruction word “this way?” for confirmation in step S14 based on the robot viewpoint o _r = SV.

Thereafter, the response of the user is determined in step S15. If the interpretation of the descriptive term of the robot 10 is incorrect, the process returns to step S12.
It moves in the movement direction determined in step S13 and confirmed in step S14.

[0052] Also in the embodiment of FIG. 7, when the microcomputer 26 determines the robot viewpoint O _r, as in the case where determining the user's view point O _u in the embodiment of FIG. 6 or FIG. 7, Rule 2 namely rules 3 5 and apply the viewpoint DV, S
V or BV is selected and determined stochastically.

[Brief description of the drawings]

FIG. 1 is an illustrative front view showing a robot according to an embodiment of the present invention;

FIG. 2 is a block diagram showing a configuration of the robot of the embodiment in FIG. 1;

FIG. 3 is an illustrative view for explaining a difference in meaning of a designated word due to a difference in viewpoint;

FIG. 4 is an illustrative view explaining a difference in meaning of a descriptive word due to a difference in viewpoint;

FIG. 5 is an illustrative view showing an outline of a room used for the experiment;

FIG. 6 is a flowchart showing an example of the operation of the robot shown in FIGS. 1 and 2.

FIG. 7 is a flowchart showing another example of the operation of the robot shown in FIGS. 1 and 2.

[Explanation of symbols]

 DESCRIPTION OF SYMBOLS 10 ... Robot 14 ... Wheel 18 ... Ultrasonic sensor 20 ... Image sensor 22 ... Microphone 24 ... Speaker 26 ... Microcomputer 28 ... ROM 30 ... RAM 32 ... Encoder 34 ... Compass 36 ... Motor 38 ... IR sensor

────────────────────────────────────────────────── ─── Continuation of the front page (56) References JP-A-60-140405 (JP, A) JP-A-2000-56827 (JP, A) JP-A-2000-137160 (JP, A) (58) Fields investigated ( Int.Cl. ⁷ , DB name) G05D 1/00 G05D 1/02

Claims (5)

(57) [Claims] 1. A self-contained mobile robot having a dialogue system for interpreting a directive from a user and determining a moving direction, comprising: a directive capturing means for capturing a directive uttered by the user; A self-contained mobile robot, comprising: a user viewpoint determining unit that determines a user viewpoint of a face-to-face viewpoint, a bird's-eye view viewpoint, or a shared viewpoint based on the positional relationship described above, and an interpreting unit that interprets the instruction word at the viewpoint. 2. The apparatus according to claim 1, wherein said user viewpoint determining means includes a first selecting means for selecting one of the three viewpoints with a predetermined probability according to a position of the robot and a moving speed and a moving direction of the user and the robot. The autonomous mobile robot according to claim 1, comprising: 3. The autonomous mobile robot according to claim 2, wherein said first selecting means selects another viewpoint when said interpretation is incorrect. 4. A robot viewpoint determining means for determining any one of a robot viewpoint, a face-to-face viewpoint, a bird's-eye view viewpoint, and a shared viewpoint, based on at least a positional relationship with the user. means,
The autonomous mobile robot according to any one of claims 1 to 3, further comprising an utterance unit that utters the generated instruction word. 5. The robot viewpoint determining means includes a second selection means for selecting one of the three viewpoints at a predetermined probability according to a position of the robot and a moving speed and a moving direction of the user and the robot. The autonomous mobile robot according to claim 4, comprising: