JP2003340760A - Robot device and robot control method, recording medium and program - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To enable a robot to act so as not to make a user feel bored. <P>SOLUTION: An action control unit 72 calculates the action value, and determines an action (a) to be performed based on the value. When the action control unit 72 performs the action (a), a reward (r) is given by an environment user 111. The action control unit 72 updates the action value based on the reward (r) acquired from the environment user 111 and the preset learning ratio. The action control unit 72 changes the learning ratio based on input information. The present invention is applicable to an enhancement learning system of the robot. <P>COPYRIGHT: (C)2004,JPO

Description

Detailed Description of the Invention

[0001]

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention relates to a robot apparatus, a robot control method, a recording medium, and a program, and in particular, it allows a robot to act in the same manner as a human being and keep the user from getting bored. And a robot control method, a recording medium, and a program.

[0002]

2. Description of the Related Art When interacting with a robot apparatus simulating a living organism, a user expects the robot apparatus to behave like a human.

In order for the robot apparatus to execute such an action, the robot apparatus may be made to perform reinforcement learning for action acquisition. In reinforcement learning, the robotic device
Learn behavior based on behavior value.

[0004]

However, in such conventional reinforcement learning, the learning rate (update rate) of action value is
Is constant regardless of the external state or internal state of the robot apparatus, and the action value when a similar reward is given is similarly updated.

Therefore, it has been difficult for the robot apparatus to behave in the same manner as a human being and to keep the user from getting tired.

The present invention has been made in view of the above circumstances, and an object thereof is to allow a robot apparatus to act in the same manner as a human being, but to keep the user from getting tired. To do.

[0007]

A robot apparatus according to the present invention is characterized by comprising action management means for dynamically changing the action learning ability.

The behavioral learning ability can be determined by the learning rate.

The learning rate can be changed according to the input information.

The learning rate can be changed according to the time by further comprising a measuring means for measuring the time.

The robot control method of the present invention is characterized by including an action management step of dynamically changing the action learning ability.

The program of the recording medium of the present invention is characterized by including an action management step of dynamically changing the action learning ability.

The program of the present invention is characterized by causing a computer to execute an action management step of dynamically changing the action learning ability.

In the robot apparatus, the robot control method, the recording medium, and the program of the present invention, the behavior learning ability is dynamically changed based on the input information.

[0015]

BEST MODE FOR CARRYING OUT THE INVENTION Embodiments of the present invention will be described below with reference to the drawings. FIG. 1 is a perspective view showing an example of a pet robot 1 to which the present invention is applied.

For example, the pet robot 1 has a four-legged bear shape, and the leg units 3A, 3B, 3C, and 3D are connected to the front, rear, left, and right of the body unit 2, respectively, and the body is The head unit 4 and the tail unit 5 are connected to the front end and the rear end of the sub unit 2, respectively.

FIG. 2 is a block diagram showing an example of the internal configuration of the pet robot 1 of FIG. The body unit 2 includes a controller 1 that controls the entire pet robot 1.
0, a battery 11 for supplying electric power to each part of the pet robot 1, a battery sensor 12, and a heat sensor 13.
An internal sensor 14 consisting of is stored. The controller 10 includes a CPU (Central Processing Unit) 10
A, a memory 10B in which a program for the CPU 10A to control each unit is stored, and a clock 10C that performs a time counting operation and measures the current date and time, elapsed time after startup, and the like.

The CPU 10A is also connected to a communication unit 63 for communicating data via a network typified by the Internet, and a storage unit 62 including a semiconductor memory for storing various data such as programs. Further, a drive 60 for reading / writing data from / to a recording medium such as the removable memory 61 is connected as necessary.

A robot control program for causing the pet robot 1 to perform an operation as a robot apparatus to which the present invention is applied is supplied to the pet robot 1 in a state of being stored in the removable memory 61, read by the drive 60, The hard disk drive installed in the storage unit 62 is installed. The robot control program installed in the storage unit 62 is instructed by the CPU 10A corresponding to the command input by the user.
It is loaded from the storage unit 62 into the memory 10B and executed.

The head unit 4 has a microphone 15 and a CCD (Charge Coupl) corresponding to an "auditory organ like an ear" that senses sound as a sensor that senses an external stimulus.
ed Device), CMOS (Complementary Metal Oxide Semicon)
ductor), an image sensor, and the like, and a video camera 16 corresponding to a “visual organ like an eye” that acquires an external image signal, and a “skin like a skin” that senses pressure or the like caused by contact with a user. Touch sensors 17 corresponding to "tactile organs" are provided at predetermined positions. Further, the head unit 4 is provided with a position detection sensor 18 that measures a distance to an object and a speaker 19 that outputs a predetermined scale and that corresponds to a “mouth-like vocal organ” of the pet robot 1. It is installed in the position.

The respective joint portions of the leg units 3A to 3D, the connecting portions of the leg units 3A to 3D and the body unit 2, the connecting portions of the head unit 4 and the body unit 2, and the tail unit 5 An actuator is installed at a connecting portion between the body unit 2 and the body unit 2. The actuator operates each part based on an instruction from the controller 10.

In the example of FIG. 2, the leg unit 3A is
Is the actuator 3AA₁Through 3AA_KProvided with legs
The unit 3B includes an actuator 3BA₁Through 3B
A_KIs provided. Also, in the leg unit 3C,
Actuator 3CA₁To 3 CA_KIs installed on the leg
Actuator 3DA for knit 3D₁Through 3DA _KBut
It is provided. Furthermore, the head unit 4 is
Player 4A₁Through 4A_LIs provided and the tail uni
The actuator 5A₁And 5A ₂Is that
It is provided.

Below, the leg units 3A to 3D are provided.
Actuator 3AA₁Through 3DA_K, Head uni
Actuator 4A provided on the unit 4₁Through 4
A_L, And the actuator provided in the tail unit
Eta 5A₁And 5A₂It is necessary to distinguish each of
If there is no need, actuator 3AA
₁Through 5A₂Called.

Further, the leg units 3A to 3D include
In addition to the actuators, the switches 3AB to 3DB are installed at places corresponding to the soles of the feet of the pet robot 1. Then, when the pet robot 1 walks, the switches 3AB to 3DB are pressed, and a signal indicating that is input to the controller 10.

The microphone 15 installed in the head unit 4 collects a surrounding voice (sound) including the user's utterance and outputs the obtained voice signal to the controller 10.
The video camera 16 captures an image of the surrounding environment and outputs the obtained image signal to the controller 10. The touch sensor 17 is provided, for example, in the upper part of the head unit 4, detects the pressure received by a physical action such as “stroking” or “striking” from the user, and uses the detection result as a pressure detection signal. Output to the controller 10. The position detection sensor 18 outputs, for example, infrared rays and outputs the detection result at the timing when the reflected light is received to the controller 10.

The controller 10 is the microphone 1
5, based on the audio signal, the image signal, the pressure detection signal, etc. provided from the video camera 16, the touch sensor 17, and the position detection sensor 18, the surrounding conditions, the instruction from the user, the presence or absence of an action from the user, etc. The pet robot 1 determines the operation to be performed next based on the determination result. Then, the controller 10 drives the necessary actuator based on the determination, thereby swinging the head unit 4 vertically and horizontally, moving the tail unit 5, and the leg units 3A to 3D.
To drive the pet robot 1 to walk or the like.

In addition, the controller 10 performs processing such as turning on, off, or blinking an LED (Light Emitting Diode) (not shown) provided on the head unit 4 of the pet robot 1 or the like.

FIG. 3 is a block diagram showing a functional configuration example of the controller 10 shown in FIG. The functions shown in FIG. 3 are realized by the CPU 10A executing the control program stored in the memory 10B.

The controller 10 includes a sensor input processing section 31 for detecting various signals from sensors (microphone 15 to position detection sensor 18 and switches 3AB to 3DB) for detecting an external stimulus, and a sensor input processing section 31. The information processing unit 32 is configured to operate the pet robot 1 based on the detected information and the like.

The angle detection unit 41 constituting the sensor input processing unit 31 is based on the information notified from each of the actuators 3AA _{1 to} 5A ₂ when the motor provided in each of the actuators 3AA _{1 to} 5A ₂ is driven. Then, the angle is detected. The angle information detected by the angle detection unit 41 is the action management unit 72 of the information processing unit 32.
And is output to the sound data generation unit 75.

The volume detecting section 42 detects the volume based on the signal supplied from the microphone 15, and outputs the detected volume information to the behavior management section 72 and the sound data generating section 75.

The voice recognition unit 43 performs voice recognition on the voice signal supplied from the microphone 15. The voice recognition unit 43 uses, as voice recognition information, a command such as “let's talk”, “walk”, “prone”, “follow the ball” and the like as the voice recognition result as the voice recognition information. , The behavior management unit 72 and the sound data generation unit 75.

The image recognition unit 44 performs image recognition using the image signal supplied from the video camera 16. As a result of the processing, the image recognition unit 44 has, for example, "a red round object", "a plane perpendicular to the ground and having a predetermined height or more", "a wide open place", and "a family is present". , ”“ I have a friend of a child in my family, ”etc.,
Instinct / emotion management unit 71, behavior management, using image recognition results such as "there is a ball", "there is a wall", "is a field", "is a house", and "is a school" as image recognition information. Part 7
2, and the sound data generation unit 75 is notified.

The pressure detector 45 processes the pressure detection signal supplied from the touch sensor 17. For example, as a result of the processing, the pressure detection unit 45 is equal to or higher than a predetermined threshold value, and
When a short-time pressure is detected, it is recognized as "struck (exhausted)", and when a long-term pressure that is less than a predetermined threshold value is detected, "stroked (praised)" is detected. , ”And notifies the recognition result to the instinct / emotion management section 71, the behavior management section 72, and the sound data generation section 75 as state recognition information.

The position detecting section 46 measures the distance to a predetermined object on the basis of the signal supplied from the position detecting sensor 18 and notifies the action managing section 72 and the sound data generating section 75 of the distance information. To do. For example, the position detector 46
Detects the distance to the hand and the distance to the ball recognized by the image recognition unit 44 when the user's hand or the like is extended in front of the eyes.

The switch input detection unit 47 is based on the signals supplied from the switches 3AB to 3DB provided in the part corresponding to the sole of the foot of the pet robot 1, for example, when the pet robot 1 is walking. The instinct / emotion management unit 71 and the behavior management unit 72 are notified of the walking timing and the fact that the sole of the foot is touched by the user.

On the other hand, the instinct / emotion management unit 71 which constitutes the information processing unit 32 manages the instinct and emotion of the pet robot 1, and expresses the parameter and the emotion indicating the instinct of the pet robot 1 at predetermined timing. The parameters are output to the behavior management unit 72 and the sound data generation unit 75.

The parameters representing the instinct and the emotions of the pet robot 1 will be described with reference to FIG. As shown in FIG. 4, the instinct / emotion management unit 71 includes an emotion model 101 that expresses the emotion of the pet robot 1,
The instinct model 102 expressing the instinct is stored and managed.

The emotion model 101 includes, for example, "joy", "sadness", "anger", "fun", "fear",
The emotional state (degree) such as “dislike” is represented by emotion parameters in a predetermined range (for example, 0 to 100), and the voice recognition unit 43, the image recognition unit 44, and the pressure detection unit of the sensor input processing unit 31. The value is changed based on the output from the unit 45, the passage of time, and the like.

In this example, the emotion model 101 is
Emotional unit 101A expressing "joy", "sadness"
Emotional unit 101B that represents, Emotional unit 101C that represents "anger", Emotional unit 101 that represents "Enjoyment"
D, an emotion unit 101E indicating “fear”, and an emotion unit 101F indicating “dislike”.

The instinct model 102 is, for example, “motility desire”, “love desire”, “appetite”, “curiosity”, “sleep desire”.
The degree (degree) of desire by the instincts is represented by instinct parameters in a predetermined range (for example, 0 to 100), and output from the voice recognition unit 43, the image recognition unit 44, the pressure detection unit 45, and the like. The value is changed based on the elapsed time or the like. Also, the instinct model 102 is
Based on the action history, the parameter expressing "motivation for exercise" is increased, or based on the remaining amount (voltage) of the battery 11,
Increase the parameter that represents "appetite."

In this example, the instinct model 102 is
An instinct unit 102A representing "motility", an instinct unit 102B representing "love", an instinct unit 102C representing "appetite", an instinct unit 102D representing "curiosity",
And a unit 102E representing "sleep desire".

Such emotion units 101A to 10A
The parameters of 1F and the instinct units 102A to 102E are changed not only by input from the outside but also by mutual influence of each unit as shown by an arrow in the figure.

For example, the emotion unit 101A expressing "joy" and the emotion unit 10 expressing "sadness"
The instinctive / emotional management unit 71 increases the parameter of the emotion unit 101A expressing “joy” when the user compliments, and expresses “sadness” by the mutual inhibition of 1B. The emotion to be expressed is changed by reducing the parameter of the emotion unit 101B.

Further, the parameters of the respective units forming the emotion model 101 and the units forming the instinct model 102 as well as the units forming the instinct model 102 are changed over both models.

For example, as shown in FIG. 4, the instinct unit 102B representing the "love" of the instinct model 102,
The parameters of the emotion unit 101B expressing “sadness” and the emotion unit 101C expressing “anger” of the emotion model 101 are changed according to the change of the parameter of the instinct unit 102C indicating “appetite”.

Specifically, when the parameter of the instinct unit 102B representing "love" or the parameter of the instinct unit 102C representing "appetite" becomes large, the emotion unit expressing "sadness" of the emotion model 101. 10
Emotional unit 1 expressing 1B parameters and "anger"
The parameter of 01C becomes small.

In this way, the emotional parameters or instinct parameters managed by the instinct / emotion management unit 71 are measured in a predetermined cycle and output to the action management unit 72 and the sound data generation unit 75. It

The instinct / emotion management unit 71 is supplied with recognition information from the voice recognition unit 43, the image recognition unit 44, the pressure detection unit 45, etc. Current or past behavior, specifically, behavior information indicating the content of the behavior such as "walking for a long time" is supplied. Then, the instinct / emotion management unit 71 generates different internal information according to the action of the pet robot 1 indicated by the action information even when the same recognition information or the like is given.

For example, when the pet robot 1 greets the user and the user strokes his / her head, the behavior information that the user is greeted and the recognition information that the head is stroked are instinct / emotion. It is supplied to the management unit 71. At this time, in the instinct / emotion management unit 71, the value of the emotion unit 101A representing “joy” is increased.

Returning to the description of FIG. 3, the action management unit 72 has the information supplied from the voice recognition unit 43, the image recognition unit 44, etc., the parameters supplied from the instinct / emotion management unit 71, and the time. The next action is determined based on the progress and the like, and a command instructing execution of the determined action is output to the posture transition management unit 73. The posture transition management unit 73 determines the posture transition based on the behavior instructed by the behavior management unit 72, and outputs the posture transition to the control unit 74. The control unit 74, based on the output from the posture transition management unit 73, the actuator 3
The operation determined by the behavior management unit 72 is performed by controlling AA1 to 5A2.

The sound data generating section 75 is based on the information supplied from the voice recognizing section 43, the image recognizing section 44, etc., the parameters supplied from the instinct / emotion managing section 71, and the passage of time. To generate sound data.
Then, the behavior management unit 72 outputs a command instructing the sound output to the voice synthesis unit 76 when the pet robot 1 speaks or when a sound corresponding to a predetermined motion is output from the speaker 19. The synthesis unit 76 causes the speaker 19 to output a sound based on the sound data output from the sound data generation unit 75.

FIG. 5 shows the principle structure of a reinforcement learning system to which the present invention is applied. In step S1,
The behavior management unit 72 causes the pet robot 1 to execute the behavior a. In step S2, the environment / user 111
The pet robot 1 is given a reward r for the action a. Here, the reward r is, for example, an action of “stroking the head” performed by the environment / user 111 when the action a is correct, or the action a is erroneous (the action was not expected by the user. ) Occasionally, the environment / user 111 performs, for example, an action of “striking the head”.

The action management unit 72 determines the action value Q (a) for this action a based on the acquired reward r according to the following equation (1) and sets a new action value Q (a based on the reward r).
Update to Q ₁ (a).

[Equation 1]

In the equation (1), the reward r is a sensor such as "struck head", "struck", "achieve a given task", "fail a given task", etc. This is a value determined according to the detection signal from the input processing unit 31, and α is a coefficient of a value between 0 and 1, and is a parameter that determines how much the acquired reward is reflected in the action value. (Learning rate).

As is clear from the equation (1), the reward r is reflected as the learning rate α is larger (the learning ability is higher), and the reward r is not reflected as the learning rate α is smaller (the learning ability is lower). .

The action value Q (a) thus obtained
Is updated each time the action a is executed and the reward r is obtained. Therefore, with the newly earned reward r ₂ ,
The updated action value Q ₂ (a) is expressed by using the previous action value Q ₁ (a) and the previous action value Q ₀ (a),

[Equation 2] Since the learning rate α is a value between 0 and 1, the coefficient α of the newly acquired reward r ₂ is the coefficient (1-α) of the previous reward r _1.
It will always be larger than α. Therefore, action value Q ₂ (a)
Is weighted more heavily on the newly received reward r ₂ than on the past reward r ₁ . That is, according to the expression (2), the action value Q (a) reflects the recently received reward rather than the reward in the distant past.

In the present invention, the behavior learning ability is dynamically changed. Therefore, the behavior management unit 72 appropriately changes the learning rate α according to the output of the emotion parameter from the instinct / emotion management unit 71. The processing in this case will be described with reference to FIG.

In step S11, the behavior management unit 72
Reads out emotion parameters from the instinct / emotion management unit 71. In step S12, the behavior management unit 72
Based on the read emotional parameters, the memory 10B
The learning rate α is calculated based on the equation or table stored in (for example, in the case of the example in FIG. 7, the equation or table representing the relationship between “fun” and the learning rate α).

FIG. 7 shows an example of the learning rate α calculated on the basis of the parameter representing "fun". As shown in FIG. 7, when the value of the parameter indicating "fun" is from "0" to m1 ("very boring" area), the learning rate α is
The value of increases gradually and the learning ability improves.
Then, the value of the parameter indicating "fun" is from m1 to m
Up to 2 (“fun” area), the learning rate α is almost constant,
Learning ability does not change. Furthermore, when the value of the parameter indicating "fun" becomes larger than m2 ("very fun" area), the value of the learning rate α gradually increases again,
Learning ability improves. That is, as with the human, if the learning ability of the pet robot 1 is improved, it becomes more enjoyable to take the action, and when the enjoyment is further increased (becomes very enjoyable), the learning ability is further improved.

Further, the learning rate α may be changed according to the output of the instinct parameter from the instinct / emotion management section 71. The processing in this case will be described with reference to FIG.

In step S21, the behavior management unit 72
Reads out the instinct parameter from the instinct / emotion management unit 71. In step S22, the behavior management unit 72
Based on the read instinct parameters, the memory 10B
The learning rate α is calculated based on the equation or table (for example, in the case of the example in FIG. 9, an equation or table representing the relationship between “curiosity” and the learning rate α) stored in the above.

FIG. 9 shows an example of the learning rate α calculated based on the parameter indicating "curiosity". In the example of FIG. 9,
The learning rate α is proportional to the value of the parameter expressing “curiosity”.
The value of becomes large and the learning ability improves. That is, the pet robot 1 can reflect the experience more strongly with less number of times of learning as the human being has more curiosity, like the human.

Further, the behavior management unit 72 is composed of the image recognition unit 4
The learning rate α can be changed in accordance with the information on the location from 4 onward. The processing in this case will be described with reference to FIG.

In step S31, the behavior management unit 72
Detects the place where the pet robot 1 is located from the output of the image recognition unit 44. In step S32, the behavior management unit 72 determines whether or not the place where the pet robot 1 is located is a school. When it is determined in step S32 that the place where the pet robot 1 is located is a school, the behavior management unit 72 advances the process to step S33, and reads the learning rate α at the school from the memory 10B.

When it is determined in step S32 that the place where the pet robot 1 is located is not a school, the behavior management section 72 advances the processing to step S34, and it is determined whether the place where the pet robot 1 is located is a house. To determine. When it is determined in step S34 that the place where the pet robot 1 is located is a house, the behavior management unit 72
Advances the processing to step S35, where the learning rate α at home
Is read from the memory 10B.

When it is determined in step S34 that the place where the pet robot 1 is located is not a house, the behavior management unit 72 advances the processing to step S36, and the place where the pet robot 1 is located is a field. To determine.
When it is determined in step S36 that the place where the pet robot 1 is located is a field, the behavior management unit 72
The process proceeds to step S37, and the learning rate α in the field is read from the memory 10B.

After the processing of step S33, S35, or S37, the behavior management unit 72 sets the read learning rate α as a new learning rate α in step S38. Then, the process ends.

If it is determined in step S36 that the place where the pet robot 1 is located is not in the field, the action management section 72 advances the processing to step S39, performs error processing, and ends the processing.

An example showing the value of the learning rate α depending on the place is shown in FIG.
Shown in 1. When the pet robot 1 is located at the school, the learning rate α has the largest value and the learning ability is high. That is, the pet robot 1 can more strongly reflect the experience with less learning when in school, like a human.

When the pet robot 1 is located at home,
The learning rate α is an average value, and the pet robot 1
Has an average learning ability. Then, when the pet robot 1 is located in the field, the value of the learning rate α becomes the smallest and the learning ability becomes low. That is, the pet robot 1
Like humans, when you're in an open field like a field,
Learning ability is reduced.

Further, the learning rate α can be changed according to the time. The processing in this case will be described with reference to FIG.

In step S51, the behavior management unit 72
Reads the current time from the clock 10C. In step S52, the behavior management unit 72, based on the read time, the formula or table stored in the memory 10B (for example, in the case of the example in FIG. 13, “time in the flow of the day”). The learning rate α is calculated based on an expression or a table indicating the relationship between the learning rate α and the learning rate α.

FIG. 13 shows an example in which the behavior management unit 72 changes the learning rate α based on time. At time t1 when the person just got up in the morning, the learning rate α is set to the smallest value, and the pet robot 1 has a low learning ability. Then, the learning rate α gradually increases with the passage of time, and the learning rate α becomes maximum at time t2 after eating breakfast. Therefore, during the time period after eating breakfast, the pet robot 1
Has improved learning ability and can reflect its experience more strongly with less learning.

After eating breakfast, until sleep time t3,
The learning rate α gradually decreases. Therefore, time t3
The pet robot 1
, The learning ability decreases (however, the learning ability is higher than in the time zone centered on time t1). After that, the learning rate α gradually increases until the time t4 from the evening to the night, and the pet robot 1 further improves the learning ability (a level intermediate between the time zone at the time t2 and the time zone at the time t3). With the learning ability).

Then, at time t5 before going to bed, the learning rate α decreases, and the learning ability of the pet robot 1 gradually decreases. Then, the pet robot 1 goes to bed. In this way, the pet robot 1 performs the same action as a human being in the flow of time during the day.

Further, the learning rate α may be changed according to the elapsed time after the activation (the growing time of the pet robot 1). The processing in this case will be described with reference to FIG.

In step S61, the behavior management unit 72
Reads the time (growth time) since the pet robot 1 was born (started) from the clock 10C. In step S42, the behavior management unit 72, based on the read growth time, the formula or table stored in the memory 10B (for example, in the case of the example of FIG. 15, “growth time” and learning rate α). (Expression or table expressing the relationship of) Learning rate α is calculated.

The action management section 72 changes the learning rate α based on the growth time t, for example, as in the following equation (3).

[Equation 3]

In the equation (3), τ is the reference time,
β represents the rate of change near the reference time. Also,
α _min and α _max represent the minimum and maximum learning rates, respectively.

Learning rate α calculated using this equation (3)
FIG. 15 shows an example showing the change of As shown in FIG. 15, the pet robot 1 has just been born (at the start-up).
The learning rate α at (t = 0) is the maximum value represented by α _max , and the experience can be more strongly reflected with a small number of learning times. After that, when the time (age) has passed,
The learning rate α gradually decreases. Then, the learning rate α finally converges to the minimum value represented by α _min , and the pet robot 1 has a reduced learning ability. That is, like humans,
As you grow older, your learning ability decreases and it becomes difficult to learn what you have learned. At a preset reference time (reference age) τ, the learning rate α is an intermediate value between the maximum value and the minimum value.

The above-described series of processing is performed not only by the animal pet robot shown in FIG. 1 but also by, for example, a humanoid robot capable of bipedal walking or a virtual robot operating in a computer. It may be executed by a robot or the like. In the present specification, the robot also includes an artificial agent.

The series of processes described above can be executed by hardware, but can also be executed by software. When a series of processing is executed by software, various functions are executed by installing the robot device in which the program that constitutes the software is built into the dedicated hardware or various programs. Possible,
For example, it is installed in a general-purpose robot device or the like from a network or a recording medium.

This recording medium, as shown in FIG.
In addition to the device body, the device is not only configured by a package medium such as a removable memory 61 in which the program is recorded, which is distributed to provide the program to the user, but is also installed in the device body in advance. And a hard disk included in the memory 10B in which the program is recorded.

In the present specification, the steps for describing the program recorded on the recording medium are not limited to the processing performed in time series according to the order described, but are not necessarily performed in time series. It also includes processing executed in parallel or individually.

[0086]

As described above, according to the present invention, the robot device can be operated. Moreover, through the action, it is possible to give the user a more life-like pseudo feeling. Therefore, it is possible to realize a robot apparatus that the user does not get tired of.

[Brief description of drawings]

FIG. 1 is a perspective view showing a configuration example of an external appearance of a pet robot to which the present invention has been applied.

FIG. 2 is a block diagram showing an internal configuration example of the pet robot of FIG.

FIG. 3 is a diagram showing a configuration example of a functional module of a pet robot.

4 is a diagram schematically showing an example of functions of an instinct / emotion management unit in FIG.

FIG. 5 is a diagram showing a configuration of a reinforcement learning system to which the present invention has been applied.

FIG. 6 is a flowchart illustrating a process of calculating a learning rate based on emotion parameters.

FIG. 7 is a diagram showing an example of changes in a learning rate based on emotion parameters.

FIG. 8 is a flowchart for explaining a process of calculating a learning rate based on an instinct parameter.

FIG. 9 is a diagram showing an example of changes in a learning rate based on an instinct parameter.

FIG. 10 is a flowchart illustrating a process of determining a learning rate based on a place.

FIG. 11 is a diagram showing an example of a change in learning rate based on a place.

FIG. 12 is a flowchart for explaining a process of calculating a learning rate based on time.

FIG. 13 is a diagram showing an example of changes in the learning rate based on time.

FIG. 14 is a flowchart illustrating a process of calculating a learning rate based on growth time.

FIG. 15 is a diagram showing an example of changes in a learning rate based on growth time.

[Explanation of symbols]

31 sensor input processing unit, 32 information processing unit, 41
Angle detection unit, 42 Volume detection unit, 43 Voice recognition unit, 44 Image recognition unit, 45 Pressure detection unit, 46
Position detector, 47 Switch input detector, 71
Instinct / emotional management unit, 72 behavior management unit, 73 posture transition management unit, 74 control unit, 75 sound data generation unit,
76 Speech synthesizer

   ─────────────────────────────────────────────────── ─── Continued front page    (72) Inventor Osamu Hanagata             6-735 Kita-Shinagawa, Shinagawa-ku, Tokyo Soni             -Inside the corporation (72) Inventor Tsuyoshi Takagi             6-735 Kita-Shinagawa, Shinagawa-ku, Tokyo Soni             -Inside the corporation F-term (reference) 2C150 CA02 DA05 DA24 DA25 DA26                       DA27 DA28 DF03 DF04 DF06                       DF33 ED10 ED39 ED42 ED47                       ED52 EF07 EF09 EF16 EF17                       EF22 EF23 EF29 EF33 EF36                 3C007 AS36 CS08 KS23 KS24 KS31                       KS36 KS39 KT01 LW12 MT14                       WA04 WA14 WB16 WC00                 5H004 GA15 GA26 GB16 JB09 KD31                       KD54 KD56 KD63

Claims (7)

[Claims] 1. A robot apparatus that behaves in accordance with supplied input information, comprising a behavior management unit that dynamically changes a behavior learning ability. 2. The robot apparatus according to claim 1, wherein the behavior learning ability is determined by a learning rate. 3. The robot apparatus according to claim 2, wherein the learning rate is changed according to the input information. 4. The robot apparatus according to claim 2, further comprising a measuring unit that measures time, wherein the learning rate is changed according to the time. 5. A robot control method for a robot apparatus that behaves according to supplied input information, including a behavior management step of dynamically changing behavior learning ability. 6. A computer-readable program for a robot apparatus that behaves in accordance with supplied input information, including a behavior management step for dynamically changing behavior learning ability. Recording medium that is recorded. 7. A program for causing a computer that controls a robot apparatus that performs an action in accordance with supplied input information to execute an action management step of dynamically changing an action learning ability.