JP2002264057A - Robot device, action control method for robot device, program and recording medium - Google Patents

Abstract

PROBLEM TO BE SOLVED: To learn action and to connect an internal state to the learned action. SOLUTION: This robot device is provided with a learning part 1 for learning new action, and a corresponding part 2 for making the internal state such as emotion correspond to the action learned by the learning part 1. When put in a certain emotional state, the robot device can make the corresponding learned action appear or can change the corresponding emotional state according to the action when taking learned action.

Description

DETAILED DESCRIPTION OF THE INVENTION

[0001]

The present invention relates to a robot apparatus that behaves autonomously, a method for controlling the action of such a robot apparatus, a program for controlling the action of such a robot apparatus, and a program for recording such a program. The recorded recording medium.

[0002]

2. Description of the Related Art An autonomous entertainment robot apparatus such as a pet-type robot apparatus which is an application of an autonomous robot apparatus has an emotion model or an instinct model indicating an internal state, and outputs an emotion model or an instinct model. Is reflected in the movement, thereby making it possible to perform a more biological movement. Here, for example, the emotion model is
The state of emotion is modeled as a numerical value or a program, and the instinct model is a model of the state of the instinct as a numerical value or a program.

[0003] The emotion model and the like are predetermined programs, and the robot apparatus executes predetermined actions, elapsed time, sensor input (including user input).
By changing the parameters such as the emotion model as described above, the predetermined operation appears when the changed parameter exceeds a predetermined threshold.

Further, the robot apparatus holds or registers a plurality of action pattern data (or action plan data), and automatically selects one action pattern data according to the emotion model as described above. Then, by reproducing the selected action pattern data, various autonomous actions appear.

On the other hand, there has been proposed a robot apparatus which does not reproduce predetermined action pattern data, but performs an action according to the surrounding environment and situation. That is,
There is a robot device that holds action pattern data and does not act using the action pattern data, but causes an ad hoc action to appear according to an external environment or the like.

Specifically, Japanese Patent Application Laid-Open No. 2000-122992 discloses that a reward (reward) is used as a criterion of motivation to act autonomously so as to select a range of action according to an external environment or the like. There has been proposed a technology of a robot device that performs the following.

Japanese Patent Application Laid-Open No. H11-126198 proposes a technique for learning an action using a recurrent neural network (hereinafter, referred to as RNN). According to this technology, a series of actions can be segmented and acquired by learning actions using an RNN, and a higher-level structure such as a segmented sequence of a series of segmented actions can be obtained. In addition, it is possible to hierarchically acquire a higher-order structure. According to this technology, the robot device is adapted to each learning situation,
For example, "go straight to the exit", "get out of the room", "turn right in the corridor" or "go straight in the corridor"
And the like are segmented, and these actions are combined to act.

[0008] The robot device capable of segmenting and learning an action as described above can acquire various actions by learning according to the user. That is, since the learning operation of the robot apparatus has a different learning environment, various operations can be obtained according to the operating environment. That is, the user (eg, owner)
Therefore, the environment in which the robot apparatus is taught is different, and the robot apparatus can acquire various operations according to such an environment.

Such a robot device behaves in accordance with its own environment, as compared with a robot device that can only act by reproducing the behavior pattern data as described above, so that the user can see Therefore, it can be appreciated as a more autonomous behavior.

[0010]

However, as described above, a robot device that can only act by reproducing the action pattern data cannot be said to be unique. In other words, for example, when you are happy, you can only make an action that will live forever.

In addition, the fact that the behavior pattern is determined in advance means that, in other words, in the case of a robot apparatus that can change the emotion model with respect to its own behavior, the emotion model can be determined only by the behavior prepared in advance. It cannot be changed.

Further, the internal state (for example, emotional state or instinct state) of the robot apparatus and the learning state have no direct relationship. For example, even if an action is learned, the action is not linked to an internal state such as an emotion.

Accordingly, the present invention has been made in view of the above-mentioned circumstances, and has a robot apparatus, a behavior control method of a robot apparatus, and a program capable of learning an action and connecting an internal state to the learned action. And the provision of a recording medium.

[0014]

A robot device according to the present invention is a robot device that changes an internal state according to input information and performs an action based on the internal state. In order to solve the above-described problem, the robot apparatus includes an internal state changing unit that changes an internal state according to input information, a learning unit that learns a new behavior, and an action learned by the learning unit. Associating means for associating an internal state with the
Behavior control means for controlling the behavior in association with the internal state.

[0015] The robot device having such a configuration is as follows.
The internal state is associated with the behavior learned by the learning means for learning a new behavior by the association means, and the behavior control is performed by the behavior control means in association with the internal state. Then, the robot apparatus changes the associated internal state by the internal state changing means.

Thus, the robot device can learn the behavior and associate the emotion with the behavior. For example, the robot apparatus learns the behavior when the emotion is changed by the internal state means and the emotion reaches a predetermined state. Appear, or the emotion corresponding to the learned behavior is changed by the internal state means.

Further, the behavior control method for a robot device according to the present invention is a behavior control method for a robot device that changes an internal state in accordance with input information and controls the behavior of the robot device acting based on the internal state. Is the way. In order to solve the above-described problem, the behavior control method of the robot device includes a learning step in which the robot device learns a new behavior, and the robot device changing an internal state with respect to the behavior learned in the learning process. There is an associating step of associating, and an action controlling step in which the robot apparatus performs action control in association with the internal state.

According to such a behavior control method for a robot device, the robot device can learn the behavior and associate the behavior with the emotion. For example, the robot apparatus changes the emotion by the internal state means and the emotion is changed to a predetermined level. When the state is reached, the learned action is caused to appear, or the emotion corresponding to the learned action is changed by the internal state means.

Further, the program according to the present invention is a program for changing an internal state in accordance with input information and controlling an action of a robot apparatus which acts based on the internal state. In order to solve the above-described problem, the program includes a learning process in which the robot device learns a new behavior, and an association in which the robot device associates an internal state with the behavior learned in the learning process. And a behavior control step in which the robot device performs behavior control in association with the internal state.

The robot device whose behavior is controlled by such a program can learn the behavior and associate the behavior with the emotion. For example, the robot apparatus changes the emotion by the internal state means so that the emotion is in a predetermined state. When it becomes, the learned behavior appears, or the emotion corresponding to the learned behavior is changed by the internal state means.

[0021] The recording medium according to the present invention is a program for changing an internal state according to input information and controlling the behavior of a robot device that acts based on the internal state. In order to solve the above-described problem, the recording medium has a learning process in which the robot device learns a new behavior, and a process in which the robot device associates an internal state with the behavior learned in the learning process. A program is recorded that executes an attaching step and an action control step in which the robot device performs action control in association with an internal state.

A robot apparatus whose behavior is controlled by a program recorded on such a recording medium can learn an action and associate an emotion with the action. Then, when the emotion reaches a predetermined state, the learned behavior appears, or the corresponding emotion is changed by the internal behavior by the learned behavior.

[0023]

Embodiments of the present invention will be described below with reference to the drawings. In this embodiment, the present invention is applied to a robot device that acts autonomously.

The robot device to which the present invention is applied is an autonomous robot device that behaves autonomously according to the surrounding environment and internal state. By applying the present invention, the robot device can learn a new behavior and can link an internal state such as an emotional state or an instinct state to the learned behavior. .

In the description of the embodiment, learning of behavior by a robot device realized by applying the present invention will be described first, and then a specific configuration of the robot device will be described.

(1) Learning of an action and linking of the internal state to the learned action Learning of the action is realized by having a learning unit 1 and an associating unit 2 as shown in FIG. Here, the learning unit 1 is a learning means for learning a new behavior,
The associating unit 2 is an associating unit that associates an internal state such as an emotion with an action learned by the learning unit 1.
For example, the learning unit 1 and the association unit 2 are configured as objects or modules configured by a software program.

The learning unit 1 is configured by a learning model such as a recurrent neural network (hereinafter, referred to as RNN). Here, RNN is
The information of the action to be learned is a neural network that is input toward the input layer, the intermediate layer, and the output layer. The processing at the time of action learning in the RNN will be described later in detail.

The learning unit 1 receives the input,
Output the corresponding output and learn with the input as the learning target. The input is made as time-series data. For example, examples of the input when learning an action include a sensor input and a motor output obtained by the action of the robot apparatus. Specifically, the sensor input includes an image signal. The information input as a learning target to the learning unit 1 is not limited to this, and may be, for example, behavior information generated internally corresponding to a behavior.

The learning section 1 outputs an input to be learned. Here, the output of the learning unit 1 is grasped as a so-called teaching signal. This output shows the same value after learning as long as the same action appears.

The associating unit 2 associates emotions (or links emotions) with the output from the learning unit 1.
Here, the correspondence between emotions is performed on the output after learning. In the embodiment, the emotion is associated with the behavior by associating the emotion with the output from the learning unit 1. However, the present invention is not limited to this. All you have to do is link your emotions. Further, in the embodiment, the case where the emotion is associated with the output is described. However, the present invention is not limited to this, and the target to be associated may be an instinct, or the behavior may be performed in other robot devices. The affected internal state can also be targeted.

Furthermore, various feelings can be considered for the emotions associated with the learned behavior. The learned behavior,
If you can differentiate by emotional attributes,
The most suitable emotion can be associated with the learned behavior. For example, if you are doing a big motion for the first time,
It is also accompanied by fear, so that such a behavior is associated with the feeling of “fear”.
In addition, it is also possible to associate an emotion of "joy" with an action approaching a human, and to associate an emotion leading to curiosity with an action approaching an unknown human. Furthermore, it is also possible to make a randomly selected emotion correspond to the learned behavior, and in this case, it is possible to express a richer emotion by the learned behavior.

With the learning unit 1 and the associating unit 2, a new behavior can be learned, and a predetermined behavior can be associated with the learned behavior. By doing so, the robot device can cause the corresponding learned behavior to appear when a certain emotional state occurs. Thereby, for example, even when the robots are in the same emotional state, different actions appear for each robot device.

Further, the robot device can change the state of the corresponding emotion when the learned behavior appears. For example, by associating the amount of change in emotion with the learned behavior, the state of the corresponding emotion can be changed by the amount of change when the user acts.

Further, in the robot apparatus, since the environment in which the user acts is different, the learning action is also different. For this reason, different behaviors for each robot device are learned by associating them with emotions. Therefore, each user can appreciate the unique behavior of each robot device. That is, since the learning environment of the behavior of the robot device differs depending on the user, even if, for example, the emotion level of “anger” is the same, each robot device causes a different behavior to appear for each user.

As described above, the robot device realizes a more biological expression by learning an action and linking the action to an internal state such as an emotion.

(2) Specific Configuration for Learning Behavior As described above, behavior can be learned by associating emotions. As a technique for learning behavior, for example,
There is a technique disclosed in Japanese Patent Application Laid-Open No. H11-126198. The robot device to which the present invention is applied learns an action, for example, by adopting this technology. Here, an outline of a technique for learning the behavior will be described.

FIG. 2 shows a configuration example of the data processing unit. The configuration shown in FIG. 2 is a specific configuration of the learning unit 1 shown in FIG. As will be described in detail later, the robot device includes a sensor that detects an obstacle and a motor that is driven to move the robot.The information is input to the data processing unit as a learning target. You.

[0038] n-number of RNN1-1～1-n are input _{x t} corresponding to the state of the sensor and the motor is input.
The RNN 1-1 is configured as shown in FIG. Although not shown, the other RNNs 1-2 to 1-n also
The configuration is the same as that of the RNN 1-1 shown in FIG.

As shown in FIG. 3, RNN1-1 is:
Has a neuronal 31 of a predetermined number of the input layer, the neuron 31, an input s _t corresponding to the state of the sensor, the input m _t corresponding to the state of the motor is input. The output of the neuron 31 is output through the neuron 32 in the hidden layer.
The signal is supplied to the neuron 33 in the output layer. From the neuron 33 in the output layer, RNN1
An output _{st + 1} corresponding to the state of the sensor of −1 and an output mt _{+ 1} corresponding to the state of the motor are output. Also, part of the output is context (contex
As t) C _t, it is adapted to be fed back to the neuron 31 in the input layer.

Outputs of the RNNs 1-1 to 1-n are input to the synthesizing circuit 3 via the corresponding gates 2-1 to 2-n.
Here, they are synthesized and the predicted output yt _{+ 1} is output.

At the time of learning, the error between the target value y ^* _{t + 1} as the teacher signal and the output of each of the RNNs 1-1 to ₁ -n controls the state of the corresponding gate 2-1 to 2-n. It has been done.

The same configuration as the above-described lower RNNs 1-1 to 1-n, gates 2-1 to 2-n, and synthesis circuit 3 is also formed in a higher hierarchy. That is, RNNs 11-1 to 11-n and gates 12-1 to 12-1 are located at higher levels.
12-n and a combining circuit 13 are provided. Then, the RNN11-1~11-n, are adapted sequence corresponding to the conduction state of the gate 2-1 to 2-n of the lower layer (closed degree) (gating sequence) _{G t} is input . Then, outputs G ¹ _{T + 1 to} G ⁿ _{T + 1} are output from the RNNs 11-1 to 11 -n, and a prediction output G _{T + 1} is output from the combining circuit 13. At the time of learning, a target value G ^* _{T + 1} is input as a teacher signal. Note that FIG.
Shows only two hierarchies, but higher hierarchies can be provided if necessary.

FIG. 4 is a diagram showing a first RN constituting a higher hierarchy.
The configuration of N11-1 is shown. Note that other RNNs 11
-2 to 11-n have the same configuration as the RNN 11-1 shown in FIG.

As shown in FIG. 4, the upper layer RNN1
1-1 is basically the RNN of the lower hierarchy shown in FIG.
1-1, a plurality of neurons 41 are arranged in an input layer, a plurality of neurons 42 are arranged in an intermediate layer, and a plurality of neurons 43 are arranged in an output layer. Signals g ¹ _{T to} g ⁿ _T corresponding to the conduction states of the gates 2-1 to 2-n are input to the input layer,
Period that the gate conduction (opening) (Time) I _T is input. From the output layer, outputs g ¹ _{T + 1 to} g ⁿ _{T + 1} and _{IT + 1} are output corresponding to these inputs. Also, part of the output of the output layer is fed back to the input layer as a context C _T.

Here, the algorithm of the RNNs 1-1 to 1-n will be described. The conduction state of the gate is expressed as shown in equation (1) using a soft-max activation function.

[0046]

(Equation 1)

Here, gⁱIs the conduction state of the i-th gate
Represents the gate coefficient corresponding to the state, s ⁱIs the i-th game
The value corresponding to the internal state of the conduction state of the switch. Obedience
Thus, the output y of the synthesis circuit 3_{t + 1}Is given by equation (2)
You.

[0048]

(Equation 2)

Here, a likelihood function represented by the equation (3), which becomes the maximum value during prediction learning, is defined.

[0050]

(Equation 3)

Here, σ represents a scaling parameter.

At the time of learning, the weight coefficients and the gate coefficients g of RNNs 1-1 to 1-n are simultaneously updated so that the likelihood function is maximized. At the time of recognition, only the gate coefficient is updated.

In order to establish a rule for updating these weighting coefficients and gate coefficients, the slope of the likelihood function with respect to the internal variable S ⁱ of the exponential function and the slope of the i-th RNN output y ^{i with the} equation (4) And (5).

[0054]

(Equation 4)

[0055]

(Equation 5)

Here, g (i | x _t , y ^* _{t + 1} )
When the i-th RNN is an input _{x t,} the target output ^{y *}
_It means the post-event probability of generating _{t + 1} and is expressed by equation (6).

[0057]

(Equation 6)

[0058] In this _{^{case, || y * t + 1 -y}} j t + 1 || 2
Represents the square error of the current prediction.

[0059] Equation (4) represents a direction to update the ^{s i.} Also, (5) as shown in the formula, the tilt related to y ^{i t} _{+ 1} of the exponential function of the likelihood function, the error condition y
^* _{T +1} contains the error term of -y ⁱ _{t + 1.} This error term is weighted by the i-th RNN post-event probability.

As described above, the weighting coefficients of the RNNs 1-1 to 1-n are adjusted so as to correct the error between the output of the i-th RNN and the target value in proportion to only the post-event probability. Thereby, one expert RNN out of n RNNs
Only the given training pattern (learning pattern) is exclusively learned. The error of each RNN is expressed by equation (7).

[0061]

(Equation 7)

The actual learning of the RNNs 1-1 to 1-n is performed by the back propagation method based on the error obtained by the above equation.

Thus, RNNs 1-1 to 1-n are:
Of the input x _t, so that each the experts can identify predetermined time series pattern which is different from the others, learning is performed.

The above is based on the fact that the RNN in the higher hierarchy
The same applies to 11-1 to 11-n. However, the input in this case is a gate sequence G _T,
The output is G ⁱ _{T + 1} .

With such a configuration, individual operations can be performed by RN
N1-1 to 1-n can learn individually. The RNNs 1-1 to 1-n learn and the manifestation of each operation is managed by the gates 2-1 to 2-n, and various operation sequences of the gates (that is, combinations of various operation orders) are determined. RNNs 11-1 to 11-n are learning. That is, by using such an information learning method, the behavior can be segmented and learned.

By having such a learning unit 1 composed of a plurality of RNNs, the robot device can move in a room forming a passage as shown in FIG. 5 and learn an action during the movement. Can be. For example, the action based on the distance sensor is learned. Specifically, the robot device learns the behavior by moving in the room and self-organizing the layers constituting the learning unit during the movement.

The technology of the learning method outlined above is disclosed in Japanese Patent Application Laid-Open No. H11-126198, and the learning unit 1 of the robot apparatus according to the embodiment of the present invention uses such a learning method. Can be built. The robot device according to the embodiment of the present invention can learn a new behavior by such a learning unit 1. As a result, after learning the k-th action (segmented action) as a new action,
If the action is selected, the k-th RNN
The behavior learned by (RNN11-k) can appear. Here, for example, the selection for causing one action to appear from the possible actions including the learned action is such that the emotion is determined by a condition such as the current posture. Done by The action switching unit will be described later in detail. Also,
Here, the action group to be selected includes actions registered in advance and actions obtained by learning as described above.

(3) Configuration of Robot Apparatus According to the Present Embodiment Next, a specific configuration of the robot apparatus that learns the above-described behavior will be described.

As shown in FIG. 6, a so-called pet robot imitating a “dog” is formed.
Leg units 103A, 103
B, 103C, and 103D are connected, and a head unit 104 and a tail unit 105 are connected to the front end and the rear end of the body unit 102, respectively.

As shown in FIG. 7, a CPU (Central Processing Unit) 110 and a DR
AM (Dynamic Random Access Memory) 111, Flash ROM (Read 0nly Memory) 112, PC (Perso
control unit 116 formed by connecting a card interface circuit 113 and a signal processing circuit 114 to each other via an internal bus 115.
And a battery 117 as a power source of the robot device 100 are stored. The body unit 10
2 includes an angular velocity sensor 118 and an acceleration sensor 11 for detecting the acceleration of the direction and movement of the robot apparatus 100.
9 etc. are also stored.

Also, the head unit 104 receives a charge (CUP) camera 120 for capturing an image of an external situation and a pressure received by a physical action such as “stroke” or “hit” from the user. A touch sensor 121 for detection, a distance sensor 122 for measuring a distance to an object located ahead, a microphone 123 for collecting external sounds, and a speaker 124 for outputting a sound such as a squeal And an LED (Light Emitting Diode) (not shown) corresponding to the “eye” of the robot device 100 are arranged at predetermined positions.

Further, each leg unit 103A-103
D, joint portions of the leg units 103A to 103D and the trunk unit 102, the head unit 10
Actuators 125 _{1 to} 125 _n and potentiometers 126 _{1 to} 126 _n are provided for the number of degrees of freedom, respectively, at a connection portion between the body unit 4 and the body unit 102 and a connection portion at the tail 105 A of the tail unit 105. For example, each of the actuators 125 _{1 to} 125 _n has a servomotor. By driving the servo motor, the leg units 103A to 103D are controlled, and the state shifts to the target posture or operation.

The angular velocity sensor 118, the acceleration sensor 119, the touch sensor 121, and the distance sensor 12
2. Various sensors such as a microphone 123, a speaker 124, and each of the potentiometers 126 _{1 to} 126 _n , an LED, and each of the actuators 125 _{1 to} 125 _n are:
The hubs 127 ₁ to 127 _n are connected to the signal processing circuit 114 of the control unit 116 via the corresponding hubs 127 ₁ to 127 _n , respectively.
The CD camera 120 and the battery 117 are directly connected to the signal processing circuit 114, respectively.

The signal processing circuit 114 sequentially takes in the sensor data, image data, and audio data supplied from each of the sensors described above, and sequentially stores them at predetermined positions in the DRAM 111 via the internal bus 115. In addition, the signal processing circuit 114 sequentially takes in remaining battery power data indicating the remaining battery power supplied from the battery 117 and stores the data in a predetermined position in the DRAM 111.

The sensor data, image data, audio data, and remaining battery data stored in the DRAM 111 as described above are used when the CPU 110 controls the operation of the robot device 100 thereafter.

In practice, the CPU 110
At the initial time when the power supply of the main unit 102 is turned on, the control program stored in the memory card 128 or the flash ROM 112 inserted in the PC card slot (not shown) of the body unit 102 is read out directly or directly through the PC card interface circuit 113. Is stored in the DRAM 111.

The CPU 110 then determines the status of itself and its surroundings and the usage status based on the sensor data, image data, audio data, and remaining battery data sequentially stored in the DRAM 111 from the signal processing circuit 114 as described above. Judge the instruction from the person and the presence or absence of the action.

[0078] Furthermore, CPU 110 is configured to determine a subsequent action based on the control program that is stored in the determination result and DRAM 111, by driving the actuator ₁₂₅ 1 to 125 _n as required based on the determination result, the head Actions such as swinging the unit 104 up and down, left and right, moving the tail 105A of the tail unit 105, and driving and walking each leg unit 103A to 103D are performed.

At this time, the CPU 110 generates audio data as necessary, and supplies the generated audio data to the speaker 124 as an audio signal via the signal processing circuit 114 to output an audio based on the audio signal to the outside. The above-mentioned LED is turned on, turned off or blinked.

In this way, the robot device 100 can behave autonomously in accordance with its own and surrounding conditions, and instructions and actions from the user.

(2) Software Configuration of Control Program Here, the software configuration of the above-described control program in the robot device 100 is as shown in FIG. In FIG. 8, the device driver layer 30 includes:
A device driver set 13 located at the lowest layer of the control program and including a plurality of device drivers
1 is comprised. In this case, each device driver is an object that is allowed to directly access hardware used in a normal computer, such as a CCD camera 120 (FIG. 7) and a timer, and receives an interrupt from the corresponding hardware. Perform processing.

The robotic server object 132 is located at the lowest layer of the device driver layer 130, and is an interface for accessing hardware such as the various sensors and actuators 125 _{1 to} 125 _n described above. Virtual robot 133, which is a software group that provides power, and a power manager 134, which is a software group that manages switching of power supply and the like.
And a device driver manager 135 which is a software group for managing various other device drivers.
And a designed robot 136 which is a software group for managing the mechanism of the robot apparatus 100.

The manager object 137 is composed of an object manager 138 and a service manager 139. The object manager 138 manages the robotic server object 13
2. A software group that manages the activation and termination of each software group included in the middleware layer 140 and the application layer 141. The service manager 139 connects the software stored in the memory card 128 (FIG. 7). A group of software that manages the connection of each object based on the connection information between the objects described in the file.

The middleware layer 140 is located on the upper layer of the robotic server object 132.
It consists of a software group that provides the basic functions of. The application layer 141 is located above the middleware layer 140, and determines the behavior of the robot device 100 based on the processing result processed by each software group constituting the middleware layer 140. It consists of a group of software for performing

FIG. 9 shows specific software configurations of the middleware layer 140 and the application layer 141, respectively.

As shown in FIG. 9, the middle wear layer 140 is used for noise detection, temperature detection, brightness detection,
Signal processing modules 1 for scale recognition, distance detection, posture detection, touch sensor, motion detection, and color recognition
50 to 158, a recognition system 160 having an input semantics converter module 159, etc., and an output semantics converter module 168 and each of posture management, tracking, motion reproduction, walking, fall return, LED lighting and sound reproduction. And an output system 69 having signal processing modules 161 to 167 and the like.

Each signal processing module 15 of the recognition system 160
0 to 158 are robotic server objects 1
DRAM 11 by 32 virtual robots 133
1 (FIG. 7), the corresponding data among the sensor data, the image data, and the audio data read from
A predetermined process is performed based on the data, and a processing result is provided to the input semantics converter module 159. Here, for example, the virtual robot 133
It is configured as a part that exchanges or converts signals according to a predetermined communication protocol.

The input semantics converter module 159 is provided for each of the signal processing modules 150 to 158.
"Noisy" based on the processing result given by
"Hot", "Bright", "Detected ball", "Detected fall", "Stroked", "Slapped", "Heared Domiso scale", "Detected moving object" Or, it recognizes the situation of itself and surroundings such as "detected an obstacle", and commands and actions from the user, and outputs the recognition result to the application layer 141 (FIG. 7).

The application layer 141 is the one shown in FIG.
As shown in FIG. 0, it is composed of five modules: a behavior model library 170, a behavior switching module 171, a learning module 172, an emotion model 173, and an instinct model 174.

The behavior model library 170 includes FIG.
As shown in, "When the battery level is low"
Independently corresponding to several pre-selected condition items such as "return to fall", "when avoiding obstacles", "when expressing emotion", "when ball is detected", etc. Behavior models 170 _{1 to} 170 _n are provided.

The behavior models 170 _{1 to} 17 ₁
0 _n are sent to the emotion model 173 as described later, as necessary, when a recognition result is given from the input semantics converter module 159 or when a certain period of time has passed since the last recognition result was given. The subsequent actions are determined with reference to the parameter values of the corresponding emotions held and the parameter values of the corresponding desires held in the instinct model 174, and the determination result is output to the action switching module 171.

In the case of this embodiment, each of the behavior models 170 _{1 to} 170 _n uses one node (state) NODE as shown in FIG.
_{0 to} NODE _n to any other node NODE _{0 to} NOD
Finite probability automaton for determining probabilistically based on the transition probability _P 1 to P _n which is set respectively arc _ARC 1 _{~ARC n1} connecting between whether a transition to E _n each node NODE ₀ ~NODE _n An algorithm called is used.

Specifically, each of the behavior models 170 _{1 to} 170 _1
n has a state transition table 180 as shown in FIG. 13 for each of the nodes NODE _{0 to} NODE _n corresponding to the nodes NODE ₀ to NODE _n forming their own behavior models 170 _{1 to} 170 _n , respectively. ing.

In this state transition table 180, the node N
Input events (recognition results) as transition conditions in ODE _{0 to} NODE _n are listed in order of priority in the row of “input event name”, and further conditions for the transition conditions are described in the rows of “data name” and “data range”. It is described in the corresponding column.

Therefore, in the node NODE ₁₀₀ represented by the state transition table 80 in FIG.
ALL) ", the size of the ball (SIZ) given together with the recognition result is given.
E) is in the range of “0 to 1000”, or when a recognition result of “obstacle detected (OBSTABLE)” is given, the “distance (DISTANCE)” to the obstacle given together with the recognition result is given. )) Is in the range of “0 to 100”, which is a condition for transitioning to another node.

In the node NODE ₁₀₀ , even when no recognition result is input, the behavior model 170
₁ to 170 _n is out of the parameter values of the emotions and the desire held respectively in the emotion model 173 and the instinct model 74 refers periodically, held in the emotion model 73 "joy (JOY)", "surprise ( When the parameter value of either “SURPRISE” or “Sadness” is in the range of “50 to 100”, transition to another node can be made.

In the state transition table 180, in the column of "transition destination node" in the column of "transition probability to another node", the names of nodes that can transition from the nodes NODE ₀ to NODE _n are listed. Input event name ",
Other nodes NOD that can transition when all the conditions described in the rows of “data value” and “data range” are met
The transition probabilities from E _{0 to} NODE _n are respectively described in corresponding portions in the column of “transition probability to another node”, and the action to be output when transitioning to the node NODE _{0 to} NODE _n is “other It is described in the row of “output action” in the column of “transition probability to node”. Note that the sum of the probabilities of each row in the column of “transition probability to another node” is 100
[%].

Therefore, in the node NODE ₁₀₀ represented by the state transition table 180 in FIG. 13, for example, “ball is detected (BALL)”, and the “SIZE” of the ball is in the range of “0 to 1000”. Is given, the probability of “30 [%]” and “node N
ODE ₁₂₀ (node 120) "and then" A
The action of “CTION1” is output.

Further, the robot apparatus 1 according to the present invention is applied.
00, as described above, can learn the behavior and associate the emotion with the learned behavior. In addition, the learned action (Action) is described in another node, and the value of the corresponding emotion is described, or the action described in advance (the value of the corresponding emotion is already determined). Is made executable (that is, released to be usable), learning of the behavior and association of the emotion with the behavior become possible.

Each of the behavior models 170 _{1 to} 170 _n is constituted by connecting a number of nodes NODE ₀ to NODE _n described as such a state transition table 180, and is recognized from the input semantics converter module 159. When a result is given, the next action is determined stochastically using the state transition table of the corresponding nodes NODE ₀ to NODE _n , and the determined result is output to the action switching module 171. .

The action switching module 171 shown in FIG.
Is the behavior model 170 of the behavior model library 170
Among the behaviors output from _{1 to} 170 _n, behavior models 170 _{1 to} 170 having a predetermined high priority
_n, and outputs a command to execute the action (hereinafter referred to as an action command) to the output semantics converter module 168 of the middleware layer 140. In this embodiment, the priority order is set higher for the behavior models 170 _{1 to} 170 _n shown on the lower side in FIG.

When reproducing the learned behavior, the behavior switching module 171 selects the designated desired behavior and sends a command to execute the behavior to the output semantics converter module 168. By the command from the action switching module 171,
The robot device 100 can output the learned behavior.

Further, the action switching module 171 includes:
After the action is completed, the learning module 172, the emotion model 173, and the instinct model 174 are notified of the completion of the action based on the action completion information provided from the output semantics converter module 168.

On the other hand, the learning module 172 recognizes, among the recognition results given from the input semantics converter module 159, such as “hit” or “stroke”.
The recognition result of the instruction received as an action from the user is input.

Then, based on the recognition result and the notification from the action switching module 171, the learning module 172 lowers the probability of occurrence of the action when “slapped (scolded)”, and “ In some cases, the corresponding behavior model 17 in the behavior model library 170 is increased so as to increase the probability of occurrence of the behavior.
Changing the 0 ₁ to 170 _n corresponding transition probability.

For example, the learning section 1 as described above is configured and realized by such a learning module 172 in the actual robot apparatus 1.

On the other hand, the emotion model 173 indicates “joy (jo
y) "," sadness "," anger ",
For a total of six emotions, “surprise”, “disgust” and “fear”, a parameter indicating the intensity of the emotion is stored for each emotion. Then, the emotion model 173 converts the parameter values of each of these emotions into “strapped” and “stroke” given from the input semantics converter module 159, respectively.
The update is periodically performed based on a specific recognition result such as, for example, an elapsed time and a notification from the action switching module 171.

Specifically, emotion model 173 is based on a recognition result given from input semantics converter module 159, the behavior of robot device 100 at that time, the elapsed time since the last update, and the like. The variation amount of the emotion at that time calculated by the arithmetic expression is ΔE [t], and the current parameter value of the emotion is E
[T], the coefficient representing the sensitivity of the emotion as _{k e,}
The parameter value E [t + 1] of the emotion in the next cycle is calculated by the equation (8), and the parameter value of the emotion is updated by replacing the parameter value E [t] with the parameter value E [t] of the emotion. The emotion model 173 is
Similarly, the parameter values of all emotions are updated.

[0109]

(Equation 8)

Here, by applying the present invention,
The robot apparatus 100 is capable of associating the emotion with the learned behavior, but here, as a processing corresponding thereto, the behavior result of the behavior learned by the robot apparatus 100 is fed back, and the emotion Updating parameter values. For example, the feedback of the action result is performed by the output of the action switching module 171 (the action to which the emotion is added). In addition, it is necessary to change the parameter value of the emotion by a predetermined amount based on the feedback of the action result. For example, an appropriately determined change amount is previously associated with the learned action. By doing so, the robot apparatus 100 performs an action corresponding to the action learned by the RNN,
The corresponding emotion parameter can be changed.

It is determined in advance how much each recognition result and the notification from the output semantics converter module 168 affect the variation ΔE [t] of the parameter value of each emotion. Is the amount of change in the parameter value of the emotion of “anger” △ E
[T] is greatly affected, and the recognition result such as “stroke” is the variation amount of the parameter value of the emotion of “joy” 喜 び E
[T] is greatly affected.

Here, the notification from the output semantics converter module 168 is so-called action feedback information (action completion information), information on the appearance result of the action, and the emotion model 173 also uses such information. Change emotions. This is, for example, a behavior such as "barking" that lowers the emotional level of anger. Note that the notification from the output semantics converter module 168 is also input to the learning module 172 described above, and the learning module 1
72 is an action model 170 _{1 to} 17 based on the notification.
Change the corresponding transition probabilities of 0 _n .

On the other hand, the instinct model 174 indicates that “the desire to exercise (ex
ercise), “affection”, “appet”
ite) "and" curiosity ", each of which has a parameter indicating the strength of the desire for each of the four independent desires. Then, the instinct model 174 periodically updates these parameter values of the desire based on the recognition result given from the input semantics converter module 159, the elapsed time, the notification from the action switching module 171 and the like.

More specifically, the instinct model 174 determines, based on the recognition result, the elapsed time, the notification from the output semantics converter module 168, and the like, for “exercise desire”, “affection desire”, and “curiosity”. The change amount of the desire at that time calculated by the arithmetic expression is ΔI
[K], the current parameter value of the desire I [k], as the coefficient k _i which represents the sensitivity of the desire, in a predetermined cycle (9)
Using the equation, the parameter value I of the desire in the next cycle
[K + 1] is calculated, and the calculation result is replaced with the current parameter value I [k] of the desire to update the parameter value of the desire. Instinct model 174
Updates the parameter values of each desire except “appetite” in the same manner.

[0115]

(Equation 9)

Here, by applying the present invention,
The robot apparatus 100 is capable of associating the instinct with the learned behavior as in the case of the emotion, but here, as the processing corresponding thereto, the behavior result of the behavior learned by the robot apparatus 100 is described. To update the parameter value of desire. For example,
The action result feedback is sent to the action switching module 1
This is performed by the output of 71 (behavior to which an emotion is added).
In addition, it is necessary to change the parameter value of the desire by a predetermined amount based on the feedback of the action result. For example, an appropriately determined change amount is associated with the learned action in advance. By doing so, the robot device can change the desire parameter corresponding to the behavior by causing the behavior corresponding to the behavior learned by the RNN.

It is determined in advance how much the recognition result and the notification from the output semantics converter module 168 affect the variation ΔI [k] of the parameter value of each desire, for example, the output semantics converter. The notification from the module 168 has a large influence on the variation ΔI [k] of the parameter value of “fatigue”.

In the present embodiment, the parameter value of each emotion and each desire (instinct) is regulated to fluctuate in the range of 0 to 100, and the coefficient k
_e, the value of k _i is also set individually for each emotion and each desire.

On the other hand, the output semantics converter module 168 of the middleware layer 40 is, as shown in FIG. 9, "forward" and "pleased" given from the action switching module 171 of the application layer 141 as described above. , "Screaming" or "tracking (following the ball)" is output to the corresponding signal processing module 161-1 of the output system 169.
Give to 67.

The signal processing modules 161 to 161
167, given the behavior command based on the action command, and servo command value to be supplied to the actuator ₁₂₅ 1 to 125 _n (FIG. 7) corresponding to perform that action, the output from the speaker 124 (FIG. 7) The sound data of the sound to be played and / or the driving data to be given to the LED of the "eye" are generated, and these data are sequentially processed via the virtual robot 133 of the robotic server object 132 and the signal processing circuit 114 (FIG. 7). Actuator 125 _{1 to} 125 _n or speaker 124 or LE
D.

As described above, in the robot apparatus 100, based on the control program, it is possible to perform an autonomous action according to its own (internal) and surrounding (external) conditions, and instructions and actions from the user. It has been made possible.

The specific configuration of the robot device 1 has been described above. In the above-described embodiment, the case where the learned behavior is associated with the emotion and the learned behavior and the emotion are changed in conjunction with each other has been described. However, the present invention is not limited to this. For example, an actual action can be compared with a learned action, and based on the comparison result, a predetermined emotion can be expressed in the action or the emotion can be changed. For example, a behavior in a new environment can be learned by learning. When a behavior in an environment where learning is not performed is compared with a result of such learning, a predetermined emotion is set. For example, the emotion of “surprise” or “fear” is changed as the predetermined emotion. This is because, for example, when the user goes to an unknown environment or receives an unknown input, “surprise” or “fear” is accompanied as a general emotion of the animal.

For example, when RNN is used as a learning method, when an input of a new action (specifically, a sensor input) to the RNN after learning is not learned, the output is set to an unknown value. Is output. That is, RN
The state of the sensor that outputs N is very different from the actual state of the sensor. Therefore, it can be determined from the output at that time that the action or environment has not been learned. For this reason, as a result of the discrimination based on the output, when the action or environment is different from the learned action or environment, a predetermined emotion such as “surprise” is expressed in the action or the emotion level is changed. You can also.

[0124]

The robot apparatus according to the present invention comprises: an internal state changing means for changing an internal state according to input information; a learning means for learning a new action; Means for associating the internal state with the internal state, and behavior control means for performing the control of the behavior in association with the internal state, so that the behavior can be learned and the emotion can be associated with the behavior. For example, when the emotion is changed to a predetermined state by the internal state means, the learned behavior can be made to appear, and the emotion corresponding to the learned behavior can be expressed by the internal state means. Can be changed.

Further, in the behavior control method for a robot apparatus according to the present invention, the learning step in which the robot apparatus learns a new behavior and the robot apparatus associates the internal state with the behavior learned in the learning step. The robot device controlled by such a behavior control method of a robot device has an associating step of performing a behavior control process in which the behavior of the robot device is associated with an internal state. At the same time, emotions can be associated with the behavior, for example, when the emotion is changed to a predetermined state by the internal state means, the learned behavior can appear, The emotion corresponding to the learned behavior can be changed by the internal state means.

The program according to the present invention includes a learning step in which the robot device learns a new action, and an association step in which the robot device associates the internal state with the action learned in the learning step. And causing the robot device to execute a behavior control step in which the robot device controls the behavior in association with the internal state, so that the robot device can learn the behavior and associate the emotion with the behavior. For example, when the emotion is changed to a predetermined state by the internal state means, the learned behavior can be made to appear, and the emotion corresponding to the learned behavior can be expressed by the internal state means. Can be changed.

Further, the recording medium according to the present invention provides a learning step in which the robot device learns a new action, and an association in which the robot device associates the internal state with the action learned in the learning step. A program for causing the robot device to execute a process and a behavior control process in which the robot device performs behavior control in association with an internal state is recorded, and the behavior is controlled by the program recorded on such a recording medium. The robot device learns the behavior and can associate the behavior with the emotion. For example, when the emotion is changed to the predetermined state by the internal state means,
The learned behavior can appear, and the emotion corresponding to the learned behavior can be changed by the internal state means.

[Brief description of the drawings]

FIG. 1 is a block diagram showing a main part for realizing the invention in a robot device according to an embodiment.

FIG. 2 is a diagram showing a specific configuration of the above-described learning unit, which is hierarchically configured by a plurality of RNNs.

FIG. 3 is a diagram illustrating a configuration of an RNN in a lower layer of a learning unit configured as the above-described hierarchical structure.

FIG. 4 is a diagram illustrating a configuration of an RNN in an upper layer of a learning unit configured as the above-described hierarchical structure.

FIG. 5 is a diagram used to explain behavior learning by the learning unit described above.

FIG. 6 is a perspective view illustrating an external configuration of the robot device according to the embodiment.

FIG. 7 is a block diagram illustrating a circuit configuration of the robot device described above.

FIG. 8 is a block diagram showing a software configuration of the robot device described above.

FIG. 9 is a block diagram showing a configuration of a middleware layer in the software configuration of the robot device described above.

FIG. 10 is a block diagram showing a configuration of an application layer in the software configuration of the robot device described above.

FIG. 11 is a block diagram showing a configuration of an action model library of the application layer described above.

FIG. 12 is a diagram used for explaining a finite probability automaton that is information for determining an action of the robot apparatus.

FIG. 13 is a diagram showing a state transition table prepared for each node of the finite probability automaton.

[Explanation of symbols]

 1 learning unit, 2 association unit, 100 robot device

Claims (11)

[Claims] 1. An internal state is changed according to input information,
A robot device that acts based on an internal state, comprising: an internal state changing unit that changes the internal state according to input information; a learning unit that learns a new behavior; A robot apparatus, comprising: associating means for associating an internal state with an action performed; and action control means for controlling action in association with the internal state. 2. The robot apparatus according to claim 1, wherein the input information is at least one of external, internal, and action result information. 3. An emotion model in which the internal state is modeled using an emotion state as a parameter, wherein the internal state changing means changes the parameter of the emotion model according to the input information. The robot device according to claim 1, wherein: 4. An instinct model in which the internal state is modeled using an instinct state as a parameter, wherein the internal state changing means changes the parameter of the instinct model in accordance with the input information. The robot device according to claim 1, wherein: 5. A storage device in which the behavior data is registered, wherein the learning device performs the learning of the new behavior in order to make the non-appearing behavior data registered in the storage device appear in advance. The robot apparatus according to claim 1, wherein: 6. The robot apparatus according to claim 1, wherein the action control means causes a corresponding action to appear based on the change in the internal state. 7. The learning means learns to act in a new environment, and the internal state changing means performs an action in an environment not learned by the learning means. The robot apparatus according to claim 1, wherein the target state is changed. 8. The learning method according to claim 1, wherein the learning means learns by using a neural network in which information on an action to be learned is input to an input layer, an intermediate layer, and an output layer. Robotic device. 9. An internal state is changed according to input information,
A behavior control method for a robot device that controls the behavior of a robot device that acts based on an internal state, comprising: a learning step in which the robot device learns a new behavior; and a behavior learned in the learning process. On the other hand, an associating step in which the robot apparatus associates an internal state, and an action control step in which the robot apparatus performs action control in association with the internal state. Behavior control method. 10. A program for changing an internal state in accordance with input information and controlling an action of a robot apparatus that acts based on the internal state, wherein the robot apparatus learns a new action. A learning step of performing, an associating step in which the robot apparatus associates an internal state with the behavior learned in the learning step, and the robot apparatus performs control of the behavior in association with the internal state. A program for causing a robot apparatus to execute an action control step. 11. A program for changing an internal state in accordance with input information and controlling an action of a robot apparatus that performs an action based on the internal state, wherein the robot apparatus learns a new action. A learning step of performing, an associating step in which the robot apparatus associates an internal state with the behavior learned in the learning step, and the robot apparatus performs control of the behavior in association with the internal state. A recording medium on which a program for causing a robot device to execute an action control step is recorded.