JP2006110707A - Robot device - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To realize social interaction of an entertainment robot while harmonizing action controlling softwares made of a plurality of stories with each other. <P>SOLUTION: The status dependent action story selects an action in accordance with a status such as outside stimulation, a change of an inside state, etc. and the reflective action story reflectively acts in accordance with the outside stimulation. Additionally, the status dependent action story restrains practice of an action command manifested from the reflective action story in the case when the reflective action does not match an intention of an action in correspondence with the status by controlling the reflective action of the robot device as the outside stimulation. Alternatively, reversely, it excites the reflective action story in making the reflective action appear. <P>COPYRIGHT: (C)2006,JPO&NCIPI

Description

  The present invention relates to a robot apparatus that performs autonomous operation and realizes realistic communication with a user, and in particular, a robot such as a recognition result of an external environment such as vision or hearing, or an internal state such as instinct or emotion is placed. The present invention relates to a behavior control system, a behavior control method, and a robot apparatus for a robot that selects an appropriate behavior by comprehensively judging the situation.

  A mechanical device that performs a movement resembling human movement using an electrical or magnetic action is called a “robot”. It is said that the word “robot” comes from the Slavic word “ROBOTA (slave machine)”. In Japan, robots started to spread from the end of the 1960s, but many of them are industrial robots such as manipulators and transfer robots for the purpose of automating and unmanned production operations in factories. Met.

  Recently, a pet-type robot that mimics the body mechanism and movement of a quadruped animal such as a dog, cat, or bear, or the body mechanism or movement of a biped upright animal such as a human or monkey. Research and development on the structure of a legged mobile robot and its stable walking control, such as the “humanoid” or “humanoid robot”, has been progressed, and the expectation for practical use is also increasing. These legged mobile robots are unstable compared to crawler robots, making posture control and walking control difficult, but they are superior in that they can realize flexible walking and running operations such as climbing stairs and climbing obstacles. Yes.

  One of the uses of legged mobile robots is to perform various difficult operations in industrial activities and production activities. For example, maintenance work at nuclear power plants, thermal power plants, petrochemical plants, transportation and assembly work of parts at manufacturing plants, cleaning of high-rise buildings, substitution of dangerous work and difficult work such as rescue at fire sites etc. .

  Further, as other uses of the legged mobile robot, rather than the above-described work support, there is a life-contact type, that is, a “symbiosis” or “entertainment” with a human. This type of robot faithfully reproduces the rich emotional expression using the movement mechanism and limbs of relatively intelligent legged walking animals such as humans, dogs (pets), and bears. In addition, it does not simply execute a pre-input motion pattern faithfully, but dynamically responds to words and attitudes received from the user (or other robots) (such as “giving up”, “speaking”, “hitting”). It is also required to realize corresponding and vivid response expressions.

  In the conventional toy machine, the relationship between the user operation and the response operation is fixed, and the operation of the toy cannot be changed according to the user's preference. As a result, the user eventually gets bored with the toy that repeats only the same action. In contrast, an intelligent robot autonomously selects an action consisting of a dialogue, a body motion, and the like, so that realistic communication can be realized at a higher intelligent level. As a result, the user feels deep attachment and familiarity with the robot.

  In a robot or other realistic dialogue system, it is common to select actions sequentially according to changes in the external environment such as vision and hearing. Another example of the action selection mechanism is to model emotions such as instinct and emotion, manage the internal state of the system, and select an action according to a change in the internal state. Of course, the internal state of the system changes depending on changes in the external environment, and also changes depending on the selected behavior.

  However, there are few examples regarding the situation-dependent action control in which the action such as the external environment and the internal state is selected by judging the situation where the robot is placed.

  Here, the internal state includes elements such as instinct that corresponds to access to the limbic system in the living body, and intrinsic and social desires that correspond to access to the cerebral neocortex. It consists of elements that can be captured by an animal behavioral model and elements called emotions such as joy, sadness, anger, and surprise.

  In conventional intelligent robots and other autonomous interactive robots, the internal state consisting of various factors such as instinct and emotions are all collected as “emotional” to manage the internal state in one dimension. That is, the elements constituting the internal state exist in parallel, and the action is selected only in the external environment and the internal state without a clear selection criterion.

  In conventional systems, the selection and expression of the action has all the actions in one dimension and determines which one to select. For this reason, the selection becomes complicated as the number of operations increases, and it becomes more difficult to select an action that reflects the situation and the internal state at that time.

  Recently, a system has been proposed in which emotions such as instincts and emotions are modeled to manage the internal state of the system, and an action is selected according to changes in the internal state (see, for example, Non-Patent Document 1). The behavior selected for the internal state and the external stimulus is often fixed, and it is difficult to change it through interaction with the user or the environment.

  A function that the robot device predicts and performs the next action and action that is optimal for the current situation, and a function that changes the next action and action based on past experience (hereinafter referred to as “situation-dependent action”). Can also be used to provide a greater sense of familiarity and satisfaction, improve amusement as a robotic device, and facilitate interaction with the user. Convenient.

  Also, recently, emotions such as the brain and emotions are modeled and the internal state of the system is managed, and this type of internal state and the recognition results obtained from sensor inputs (such as touch sensor sequence input and color ball An entertainment robot has been proposed that can spontaneously select a situation-dependent action based on two types of inputs, i.e., recognition, face recognition, voice recognition, etc., i.e., external stimuli (see, for example, Patent Document 1). thing).

  On the other hand, in addition to spontaneous situation-dependent behavior, the present inventors think that it is also important for entertainment robots to perform reflexive behavior using sensor input itself as a trigger. For example, it is a reflex action for an external stimulus such as a shoulder touch sensor being pressed, an object suddenly appearing in front of the eyes, or a loud sound. This kind of reflex behavior is basically performed independently of the internal state. In such a case, it is difficult to mediate two types of behaviors, that is, situation dependent behavior and reflex behavior.

Japanese Patent No. 3,558,222 Co-authored by Tetsuya Ogata and Shigeki Kanno, “Robot Behavior Generation Based on Self-Preservation-Realization of Methodology and Machine Model-” The Journal of the Robotics Society of Japan, 1997, Vol. 15, No. 5, p. 710-721

  An object of the present invention is to provide an excellent robot apparatus capable of performing autonomous operation and realizing realistic communication.

  A further object of the present invention is to be able to select an action by comprehensively judging the situation where the robot is placed such as the recognition result of the external environment such as vision and hearing, and the internal state such as instinct and emotion. It is to provide a robot apparatus.

  A further object of the present invention is to provide an excellent robot apparatus that can clarify the significance of the existence of emotions and can appropriately select and execute an action according to an external stimulus or an internal state under a certain order. There is to do.

  A further object of the present invention is to be able to select an action by comprehensively judging the situation where the robot is placed such as the recognition result of the external environment such as vision and hearing, and the internal state such as instinct and emotion. It is to provide a robot apparatus.

  It is a further object of the present invention to provide an excellent robot apparatus capable of realizing social interaction of entertainment robots while harmonizing behavior control software composed of a plurality of layers.

  A further object of the present invention is to provide an excellent robot capable of suitably expressing two different behaviors, a spontaneous situation-dependent behavior according to an external stimulus or an internal state, and a reflex behavior that reacts directly to the external stimulus. To provide an apparatus.

  A further object of the present invention is to provide an excellent robot apparatus that can suitably mediate two different types of behaviors, ie, spontaneous situation-dependent behavior and reflex behavior.

The present invention has been made in consideration of the above problems, and in a robot apparatus that generates an action based on an internal state or an external input,
A plurality of action control layers for controlling actions based on internal states or external inputs;
A resource management unit that manages resources of the robot device and resolves conflicts of operation commands related to driving of the robot device from each behavior control layer;
A robot apparatus comprising:

  Here, each of the behavior control layers is composed of one or more behavior modules that determine the behavior of the robot, and each behavior module outputs an behavior evaluation of the robot device according to an internal state or an external input. Evaluation means and action command output means for outputting an action command of the robot apparatus are provided. In each of the behavior control layers, an operation command for controlling the behavior of the robot apparatus is generated based on the behavior evaluation in the behavior evaluation unit of each behavior module.

  Each behavior control layer can be configured by a plurality of behavior modules in a tree structure according to the realization level of the robot apparatus. This tree structure includes a plurality of branches such as a behavior model obtained by formulating an animal behavioral (ethological) situation-dependent behavior and a branch for executing emotional expression. For example, in the hierarchy immediately below the root behavior module, behavior modules such as “Investigate”, “Estimate”, and “Play” are arranged. Then, below “Investigate”, behavior modules describing more specific search behaviors such as “Investigative Location”, “HeadinAirSniffing”, and “InvestigativeSniffing” are arranged. Similarly, a behavior module describing more specific eating and drinking behavior such as “Eat” and “Drink” is arranged below the behavior module “Ingestive”, and subordinate to the behavior module “Play”. Are arranged with behavior modules describing more specific playing behaviors such as “PlayBowing”, “PlayGreeting”, “PlayPawing” and the like.

  Therefore, in each of the behavior control layers, the behavior module is selected based on the behavior evaluation output from the lower layer behavior module to the upper layer behavior module, and the behavior of the robot apparatus is controlled. be able to.

  The behavior command output means of each behavior module outputs the resource of the robot device used when the behavior command is executed in the robot device. In each of the behavior control layers, the behavior of the robot device is controlled based on the behavior evaluation output from the behavior module of the lower layer of the hierarchical structure to the behavior module of the upper layer and the use resources of the robot device. .

  In such a case, the behavior evaluation unit can evaluate a plurality of behavior modules simultaneously from the top to the bottom of the tree structure. Also, in response to external input or changes in the internal state of the robot apparatus itself, the behavior evaluation unit performs the evaluation of each of the behavior modules, and the execution permission as an evaluation result from the top to the bottom of the tree structure It is possible to selectively execute appropriate actions according to changes in the external environment and internal state. That is, the evaluation and execution of the situation-dependent behavior can be performed on the current.

  Here, the robot apparatus according to the present invention includes, as a plurality of action control layers, for example, a reflex action control hierarchy that directly receives an external input recognition result and directly determines an output action, an external input, and an internal state of the robot apparatus. A situation-dependent action hierarchy that controls actions in response to the situation in which the robot apparatus is placed, and a contemplation action control hierarchy that infers future situations and plans the action of the robot apparatus over a relatively long period of time. It has a hierarchical structure of behavior control consisting of layers.

  Since the action selection based on the reflection action hierarchy and the situation-dependent action hierarchy is performed independently, hardware resources may compete when executing action modules selected from each other on the robot apparatus. The command issued from the reflex behavior hierarchy is a command generated as a reflex action, whereas the command issued from the previous situation-dependent action hierarchy is a command generated as a situation dependence action. Since these behavior layers are generated as separate processes or as separate threads, it is difficult to mediate in advance.

  Therefore, in the present invention, the resource management unit arbitrates competition between operation commands of the reflex behavior hierarchy and the situation-dependent behavior hierarchy.

  The resource management unit includes command conflict solution means for solving a conflict of commands from the reflection action hierarchy and the contemplation action hierarchy, and content management means for managing commands for each hardware resource of the robot apparatus. . The command conflict solution means holds the command, the hardware resource of the robot device used by the command, and the activity level of the command. When a new command is transmitted from any action control layer, It is determined whether or not the hardware resource of the robot device used by the command being executed conflicts with the hardware resource of the robot device used by the new command. Comparing command activity levels can resolve command conflicts.

  Therefore, if the activity level of the currently executing command is lower than the activity level of the new command, the executing command is canceled and the new command is executed. On the other hand, if the activity level of the currently executing command is higher than the activity level of the new command, the new command is executed waiting for the end of the executing command, or the new command is canceled. . When the activity level of a new command is higher than the activity level of a command waiting in the resource management unit, the command waiting in the resource management unit is canceled.

  The behavior evaluation unit in the behavior module includes a behavior induction evaluation value calculation unit that calculates an evaluation value that induces an output of a behavior command by the behavior command unit as an activity level, and a bias calculation that calculates a bias with respect to the activity level as an intention level And a use resource calculation means for specifying the resource of the robot device to be used when the action command by the action command means is executed. The higher behavior control layer instructs the lower behavior control layer to instruct the intention level, thereby suppressing the operation against the intention of the own layer, or conversely exciting it to meet the intention of the own layer. can do.

  For example, the situation-dependent action hierarchy outputs an action intention signal that suppresses or excites a reflection action depending on whether the reflection action matches the intention of the situation-dependent action to the reflection action hierarchy, The behavior intention management means for managing the behavior intention signal is provided, and the intention level in each behavior module is operated based on the behavior intention in the situation-dependent behavior hierarchy. Thereby, reflex behavior contrary to the intention of the situation-dependent behavior can be suppressed, or the reflex behavior that matches the intention of the situation-dependent behavior can be excited.

  Since the reflex behavior control hierarchy operates directly by the input of an external stimulus, the situation-dependent behavior hierarchy cannot sufficiently suppress the reflex behavior by relying on mediation by the resource management unit. On the other hand, the situation-dependent action hierarchy can suitably suppress the reflex action that the situation-dependent action is not intended by notifying the reflection action control hierarchy of the intention level.

  Further, the contemplation behavior layer outputs an action intention signal that suppresses or excites a situation-dependent behavior according to whether or not the situation-dependent behavior matches the intention of the contemplation behavior to the reflex behavior layer, and the situation-dependent behavior layer Comprises an action intention management means for managing an action intention signal, and operates an intention level in each action module based on the action intention in the contemplation action hierarchy. Thereby, it is possible to suppress a behavior module that is contrary to the intention of the contemplation behavior or to excite the behavior module that matches the intention of the contemplation behavior.

  That is, the contemplation action hierarchy can appropriately suppress the situation-dependent action that is not intended by the contemplation action by notifying the intention-dependent action control hierarchy of the intention level.

  In addition, when all the elemental actions that make up the situation-dependent action hierarchy are actions to keep the internal state within a certain range, that is, “homeostasis action”, when all the internal states are sufficiently satisfied, the desire value of each elemental action is Since it becomes smaller, the action value (activity level of the action module for solving the competition of action modules in the action control hierarchy) is also reduced, and the opportunity for the situation-dependent action to occur is reduced. In such a case, since there is no reflex behavior if there is no external stimulus, there is a problem that the robot apparatus does nothing and impairs entertainment. Therefore, “idle behavior” having no homeostasis purpose may be further incorporated as a component of the situation-dependent behavior that expresses spontaneous behavior of the robot apparatus.

  As described above, when the idle behavior is adopted as the self-issued motion, it is necessary to mediate whether the homeostasis behavior or the idle behavior should be output in the situation-dependent behavior hierarchy. In addition, between the situation-dependent action hierarchy and the reflex action hierarchy, it is necessary to mediate which of the self-issued movement and the reflex action output from each should be acted.

  For example, the situation-dependent action hierarchy is composed of a plurality of elemental actions for outputting each homeostasis action and idle action, and the reflex action hierarchy is composed of a plurality of elemental actions for outputting each reflex action. . Elemental behavior corresponds to the behavior module described above. And the action value calculation means which calculates the action value which shows the execution priority of the said action of each element action is each provided. The behavior value calculation means calculates a desire value from the internal state of the robot apparatus, calculates an expected satisfaction value from the internal state and the recognition result, and performs an action of an elementary action of homeostasis behavior from the desire value and the predicted satisfaction value Calculate value. Also, a certain value is given as the action value of the element action of the idle action.

  In such a case, the situation-dependent action hierarchy includes action selection means for selecting an element action to be output as a self-issued action based on the action value, and allows mediation between homeostasis action and idle action. it can.

  While the behavioral value of homeostasis behavior is calculated from the desire value and the expected satisfaction value, a certain behavioral value is given to idle behavior. Therefore, since the behavioral value of homeostasis behavior increases when the desire value increases, homeostasis behavior is selected as the situation-dependent behavior. On the contrary, when all the internal states are sufficiently satisfied, that is, when the action value of homeostasis action decreases and becomes lower than the action value of idle action, the idle action is selected.

  Since the idle behavior has no homeostasis purpose, that is, it has no relation to the internal state, the behavior value of the element behavior is given a constant value. And it is necessary to avoid or alleviate competition with reflex behavior.

  Therefore, the robot apparatus according to the present invention further includes a reflection event management unit that manages a reflection event density at which an external input event that causes a reflection action occurs, and the situation-dependent action hierarchy is idle based on the reflection event density. The probability that each elemental action is selected is obtained, and the elemental action is selected according to the probability. When the reflection event density is high, it is easy to select an idle action or a reflection action with a small amount of movement, and when the reflection event density is low, it is easy to select an idle action with a large amount of movement or it is difficult to select a reflection action. As a result, it is possible to arbitrate the timing for spontaneously outputting idle behavior and the type of motion based on the type and frequency of events that cause reflex behavior.

  The robot apparatus according to the present invention further includes a resource management unit that arbitrates between the elemental motion of the self-issued movement output from the situation-dependent behavioral hierarchy and the elemental motion of the reflective behavior output from the reflective behavior hierarchy. . The situation-dependent action hierarchy and the reflection action hierarchy give command strengths (command activity levels for performing command conflict resolution in the resource manager) to the element actions to be output. The resource management unit preferably mediates between two different behaviors by mediating between homeostasis behavior and reflex behavior or idle behavior and reflex behavior based on the strength of the command. ing.

  Here, the situation-dependent action hierarchy gives command strength based on action value to the elemental action of homeostasis action, and gives unique command strength to each elemental action of idle action. . Also, the reflex behavior hierarchy gives a constant value as the command strength to all element behaviors.

  In such a case, when the action value (desire) of the homeostasis action is low, the robot apparatus easily reacts to the trigger of the reflex action, but when the action value of the homeostasis action is high, the robot apparatus reflects it. It does not respond to the trigger of the action (it seems to concentrate on the action).

  Further, as described above, when the desire is low, the idle behavior that is not homeostasis is selected as the spontaneous behavior, and in this case, the resource management unit mediates between the idle behavior and the reflex behavior. In the idle behavior, the type of motion is determined according to the type and frequency of the event (external input) that causes the reflective behavior. In the idle behavior, when there are many reflection events from the user and the reflection event density is high, the robot apparatus responds to the trigger of the reflection behavior. On the other hand, when the reflection event density is low due to being left by the user, the robot apparatus does not respond to the trigger of the reflection action (it seems to concentrate on the action).

  ADVANTAGE OF THE INVENTION According to this invention, the outstanding robot apparatus which can perform realistic operation | movement and can realize realistic communication with a user can be provided.

  Further, according to the present invention, it is possible to select an action by comprehensively judging the situation where the robot is placed such as the recognition result of the external environment such as vision and hearing and the internal state such as instinct and emotion, An excellent robot apparatus can be provided.

  Further, according to the present invention, it is possible to select an action by comprehensively judging the situation where the robot is placed such as the recognition result of the external environment such as vision and hearing and the internal state such as instinct and emotion, An excellent robot apparatus can be provided.

  Further, according to the present invention, an excellent robot apparatus capable of clarifying the existence significance of emotion and appropriately selecting and executing an action according to an external stimulus or an internal state under a certain order. Can be provided.

  Furthermore, according to the present invention, it is possible to provide an excellent robot apparatus capable of realizing social interaction of entertainment robots while harmonizing behavior control software composed of a plurality of layers.

  In addition, according to the present invention, the reflex action, the situation-dependent action, and the contemplation action are separate processes, and by constructing a control method therefor, the reaction time of the reflex action is delayed by the situation-dependent action and the contemplation action. Because it can be avoided, it is possible to enhance the description of the situation where the robot is placed such as the recognition result of external environment such as vision and hearing and the internal state such as instinct and emotion without worrying about the delay of reaction An apparatus can be provided.

  The robot apparatus according to the present invention can express the degree of concentration of homeostasis action by controlling the output of the reflex action by the action value of the self-issued movement.

  In addition, the robot apparatus according to the present invention can express the concentration degree of the idle action by controlling the output of the reflection action according to the strength of the command of the idle action according to the reflection event density.

  In addition, the robot apparatus according to the present invention can express the concentration level of idle behavior by controlling the type of idle behavior according to the reflection event density.

  In addition, the robot apparatus according to the present invention can easily output the reflex action kinetically by preventing a motion with a large operation amount as much as possible when there are many reflex events.

  Further, the robot apparatus according to the present invention can increase the increase in the reflection event density when there is an explicit reflection event from the user.

  Other objects, features, and advantages of the present invention will become apparent from more detailed description based on embodiments of the present invention described later and the accompanying drawings.

  Hereinafter, embodiments of the present invention will be described in detail with reference to the drawings.

A. Configuration of Robot Device FIG. 1 schematically shows a functional configuration of a robot device 1 used in the present invention. As shown in the figure, the robot apparatus 1 includes a control unit 20 that performs overall control of the entire operation and other data processing, an input / output unit 40, a drive unit 50, and a power supply unit 60. . Hereinafter, each part will be described.

  The input / output unit 40 is disposed as an input unit at a part such as a CCD (Charge Coupled Device) camera 15 corresponding to the eyes of the robot apparatus 1, a microphone 16 corresponding to the ear, a head or a back. It includes a touch sensor 18 that senses a user's contact, or other various sensors corresponding to the five senses. Further, as an output unit, a speaker 17 corresponding to the mouth or an LED indicator (eye lamp) 19 for forming a facial expression by a combination of blinking and lighting timing is provided. These output units can express user feedback from the robot apparatus 1 in a format other than a mechanical motion pattern such as a leg or the like, such as sound or blinking of a lamp.

  The drive unit 50 is a functional block that realizes the body operation of the robot apparatus 1 in accordance with a predetermined motion pattern commanded by the control unit 20, and is a control target by behavior control. The drive unit 50 is a functional module for realizing the degree of freedom in each joint of the robot apparatus 1 and includes a plurality of drive units provided for each axis such as roll, pitch, and yaw in each joint. Each drive unit adaptively controls the rotational position and rotational speed of the motor 51 based on the output of the motor 51 that performs a rotational operation around a predetermined axis, the encoder 52 that detects the rotational position of the motor 51, and the encoder 52. A combination of drivers 53 is used.

  Depending on how the drive units are combined, the robot apparatus 1 can be configured as a legged mobile robot such as a bipedal walking or a quadrupedal walking.

  The power supply unit 60 is a functional module that feeds power to each electrical circuit in the robot apparatus 1 as its meaning. The robot apparatus 1 according to the present embodiment is an autonomous drive type using a battery, and the power supply unit 60 includes a charging battery 61 and a charging / discharging control unit 62 that manages the charging / discharging state of the charging battery 61. . The rechargeable battery 61 is configured, for example, in the form of a “battery pack” in which a plurality of lithium ion secondary battery cells are packaged in a cartridge type. Further, the charge / discharge control unit 62 grasps the remaining capacity of the battery 61 by measuring the terminal voltage of the battery 61, the amount of charge / discharge current, the ambient temperature of the battery 61, etc., and determines the charging start timing and end timing. decide. The charging start / end timing determined by the charge / discharge control unit 62 is notified to the control unit 20 and serves as a trigger for the robot apparatus 1 to start and end the charging operation.

  The control unit 20 corresponds to a “brain”, and is mounted on, for example, the body head or the trunk of the robot apparatus 1.

  FIG. 2 illustrates the configuration of the control unit 20 in more detail. As shown in the figure, the control unit 20 has a configuration in which a CPU (Central Processing Unit) 21 as a main controller is connected to a memory and other circuit components and peripheral devices via a bus. The bus 27 is a common signal transmission path including a data bus, an address bus, a control bus, and the like. Each device on the bus 27 is assigned a unique address (memory address or I / O address). The CPU 21 can communicate with a specific device on the bus 28 by specifying an address.

  A RAM (Random Access Memory) 22 is a writable memory composed of a volatile memory such as a DRAM (Dynamic RAM), and loads a program code executed by the CPU 21 or temporarily stores work data by the execution program. Used to save. A ROM (Read Only Memory) 23 is a read-only memory that permanently stores programs and data. Examples of the program code stored in the ROM 23 include a self-diagnosis test program that is executed when the robot apparatus 1 is powered on, and an operation control program that defines the operation of the robot apparatus 1.

  The control program for the robot apparatus 1 is a “sensor input / recognition processing program” that processes sensor inputs such as the camera 15 and the microphone 16 and recognizes them as symbols, and performs storage operations (described later) such as short-term memory and long-term memory. A “behavior control program” for controlling the behavior of the robot apparatus 1 based on the sensor input and a predetermined behavior control model, and a “drive control program” for controlling the driving of each joint motor and the sound output of the speaker 17 according to the behavior control model. Etc. are included.

  The nonvolatile memory 24 is composed of a memory element that can be electrically erased and rewritten, such as an EEPROM (Electrically Erasable and Programmable ROM), and is used to hold data to be sequentially updated in a nonvolatile manner. Data to be updated sequentially includes an encryption key and other security information, a device control program to be installed after shipment, and the like.

  The interface 25 is a device for interconnecting with devices outside the control unit 20 and enabling data exchange. The interface 25 performs data input / output with the camera 15, the microphone 16, and the speaker 17, for example. The interface 25 inputs and outputs data and commands to and from the drivers 53-1.

  In addition, the interface 25 includes a serial interface such as RS (Recommended Standard) -232C, a parallel interface such as IEEE (Institut of Electrical and Electronics Engineers) 1284, a USB (Universal Serial Bus) I interface 94, an E A general-purpose interface for connecting computer peripherals, such as a small computer system interface (SCSI) interface, a memory card interface (card slot) that accepts PC cards and memory sticks, etc. Between external devices It may be performed to move programs and data.

  As another example of the interface 25, an infrared communication (IrDA) interface may be provided to perform wireless communication with an external device. Furthermore, the control unit 20 includes a wireless communication interface 26, a network interface card (NIC) 27, and the like, and is used for close proximity wireless data communication such as Bluetooth, a wireless network such as IEEE 802.11b, or a wide area such as the Internet. Data communication can be performed with various external host computers via the network.

  By such data communication between the robot apparatus 1 and the host computer, complex operation control of the robot apparatus 1 can be calculated or remotely controlled using remote computer resources.

B. A behavior control system diagram 3 of the robot apparatus has a functional configuration of a behavior control system 100 of the robot apparatus 1 according to an embodiment of the present invention is schematically shown. The robot apparatus 1 can perform behavior control according to the recognition result of the external stimulus and the change in the internal state. Furthermore, by providing a long-term memory function and associatively storing a change in the internal state from an external stimulus, it is possible to perform action control according to the recognition result of the external stimulus and the change in the internal state.

  The illustrated behavior control system 100 can adopt and implement object-oriented programming. In this case, each software is handled in units of modules called “objects” in which data and processing procedures for the data are integrated. In addition, each object can perform data transfer and invoke using message communication and an inter-object communication method using a shared memory.

  The behavior control system 100 includes a visual recognition function unit 101, an auditory recognition function unit 102, and a contact recognition function unit 103 in order to recognize an external environment (Environments).

  The visual recognition function unit (Video) 51 performs image recognition processing such as face recognition and color recognition and feature extraction based on a photographed image input via an image input device such as the CCD camera 15, for example. The visual recognition function unit 51 includes a plurality of objects such as “MultiColorTracker”, “FaceDetector”, and “FaceIdentify” which will be described later.

  The auditory recognition function unit (Audio) 52 performs voice recognition on voice data input through a voice input device such as a microphone, and performs feature extraction or word set (text) recognition. The auditory recognition function unit 52 includes a plurality of objects such as “AudioRecog” and “AuthorDecoder” described later.

  The contact recognition function unit (Tactile) 53 recognizes an external stimulus such as “struck” or “struck” by recognizing a sensor signal from a contact sensor built in the head of the aircraft, for example.

  An internal status management unit (ISM: Internal Status Manager) 104 manages several types of emotions such as instinct and emotion by modeling them, and includes the above-described visual recognition function unit 101, auditory recognition function unit 102, and contact recognition function. The internal state such as instinct and emotion of the robot apparatus 1 is managed according to an external stimulus (ES: External Stimula) recognized by the unit 103.

  The emotion model and the instinct model have the recognition result and the action history as inputs, respectively, and manage the emotion value and the instinct value. The behavior model can refer to these emotion values and instinct values.

  In the present embodiment, the emotion is composed of a plurality of hierarchies depending on the significance of existence, and operates in each hierarchy. From a plurality of determined operations, which operation is to be performed is determined according to the external environment and internal state at that time (described later). In addition, actions are selected at each level, but by expressing actions preferentially from lower-order actions, higher-order actions such as instinct actions such as reflexes and action selection using memory Behavior can be expressed consistently on one individual.

  The robot apparatus 1 according to the present embodiment includes a short-term storage unit 105 that performs short-term storage that is lost over time, and information in order to perform behavior control according to the recognition result of the external stimulus and the change in the internal state. A long-term storage unit 106 for holding for a relatively long time is provided. The classification of memory mechanisms, short-term memory and long-term memory, relies on neuropsychology.

  A short-term memory unit (ShortTermMemory) 105 is a functional module that holds targets and events recognized from the external environment by the visual recognition function unit 101, the auditory recognition function unit 102, and the contact recognition function unit 103 for a short period of time. For example, the input image from the camera 15 is stored for a short period of about 15 seconds.

  A long term memory unit (LongTermMemory) 106 is used to hold information obtained by learning such as the name of an object for a long period of time. For example, the long-term storage unit 106 can associatively store a change in the internal state from an external stimulus in a certain behavior module.

  Also, the behavior control of the robot apparatus 1 according to the present embodiment is performed by the “reflection behavior” realized by the reflection behavior unit 109, the “situation dependence behavior” realized by the situation-dependent behavior hierarchy 108, and the contemplation behavior hierarchy 107. It can be roughly divided into "contemplation behavior" to be realized.

  A reflexive behavior unit (Reflective Situated Behaviors Layer) 109 is a functional module that realizes a reflective body operation in response to an external stimulus recognized by the visual recognition function unit 101, the auditory recognition function unit 102, and the contact recognition function unit 103 described above. It is.

  The reflex behavior is basically a reflex behavior that is triggered by the sensor input itself, that is, it receives the recognition result of the external information input from the sensor directly, classifies it, and outputs the behavior. Decide directly. For example, a behavior such as chasing a human face or nodding is preferably implemented as a reflex behavior. The reflex action unit 109 executes the control cycle at a sufficient speed as compared with the situation-dependent action layer 108.

  The situation-dependent behavior hierarchy (Situated Behaviors Layer) 108 is based on the storage contents of the short-term storage unit 105 and the long-term storage unit 106 and the internal state managed by the internal state management unit 104. Control responsive behavior.

  The situation-dependent action hierarchy 108 prepares a state machine for each action, classifies recognition results of external information input from the sensor, and expresses the action on the aircraft depending on the previous action and situation. . The situation-dependent action hierarchy 108 also realizes an action for keeping the internal state within a certain range (also referred to as “homeostasis action”). When the internal state exceeds the specified range, the internal state is The action is activated so that the action for returning to the range is likely to appear (actually, the action is selected in consideration of both the internal state and the external environment). Situation-dependent behavior has a slower response time than reflex behavior.

  A deliberate action hierarchy (Deliberate Layer) 107 performs a relatively long-term action plan of the robot apparatus 1 based on the storage contents of the short-term storage unit 105 and the long-term storage unit 106. In general, the more the robot device is applied to a behavior that responds to the situation, the easier it is to fall into a set of momentary behavior as a whole. In the present embodiment, not only can the behavior immediately responding to the situation be expressed, but the contemplation behavior hierarchy 107 controls the contemplation behavior of pre-reading (ie, inferring) the situation ahead and planning the behavior. Details of the contemplation action hierarchy 107 will be described later.

  The pondering action mentioned here is an action that is performed based on a given situation or a command from human beings and making a plan for realizing it. For example, searching for a route from the position of the robot and the position of the target corresponds to a contemplation action. Such an inference or plan may require a processing time or a calculation load (that is, a processing time) rather than a reaction time for the robot apparatus 1 to maintain interaction. While responding in real time, the contemplation action makes inferences and plans.

  The contemplation behavior layer 107, the situation-dependent behavior layer 108, and the reflex behavior unit 109 can be described as higher-level application programs that are independent of the hardware configuration of the robot apparatus 1. On the other hand, the hardware dependent layer control unit (ConfigurationDependentActionsAndReactions) 110 performs hardware (external environment) such as joint actuator driving in accordance with commands from these higher-level applications (behavior modules called “schema”). Operate directly.

C. As storage mechanism described above of the robot apparatus, the robot apparatus 1 according to this embodiment is provided with the short-term memory unit 105 and the long-term memory unit 106, such storage mechanism relies on neuropsychological.

  Short-term memory is literally short-term memory and is lost over time. Short-term memory can be used to hold targets and events recognized from the external environment, such as vision, hearing, and touch, for a short period of time.

  In short-term memory, “sensory memory” that holds sensory information (that is, output from the sensor) as it is for about 1 second, and “direct memory” that encodes sensory memory and stores it in a short time with a limited capacity. ”And“ working memory ”that memorizes situation changes and contexts over several hours. Direct memory is said to be 7 ± 2 chunks according to neuropsychological studies. The working memory is also referred to as “intermediate memory” in contrast to short-term memory and long-term memory.

  Long-term memory is used to hold information obtained by learning such as the name of an object for a long period of time. The same pattern can be statistically processed for robust storage.

  Long-term memory is further classified into “declarative knowledge memory” and “procedural knowledge memory”. The declarative knowledge memory is composed of “episode memory” that is a memory related to a scene (for example, a scene when taught) and “semantic memory” that includes memories such as the meaning and common sense of words. The procedural knowledge memory is a procedural memory such as how to use a declarative knowledge memory, and can be used to acquire an operation for an input pattern.

C-1. Short-term memory unit The short-term memory unit 105 is a functional module that expresses and stores objects or events that exist around itself, and is intended to allow a robot to act on the basis of it. The position of an object or event is arranged on the self-centered coordinate system based on sensor information such as vision and hearing, but an object outside the field of view can be stored and an action or the like can be generated.

  For example, when talking to a person A and speaking to another person B, holding a position of A and the conversation contents, holding a conversation with B, and returning to a conversation with A after the end A short-term memory function is required. However, integration based on simple proximity in space and time, such as considering sensor information that is close in time and space as signals from the same object, without performing integration through very complicated processing.

  Moreover, in order to memorize | store the position of objects other than the object which can be discriminate | determined by pattern recognition using techniques, such as stereo vision, it arrange | positions on a self-centered coordinate system. It can be used together with floor surface detection to store the position of an obstacle stochastically.

  In the present embodiment, the short-term storage unit 105 temporally and spatially aligns external stimuli formed by a plurality of recognizers such as the visual recognition function unit 101, the auditory recognition function unit 102, and the contact recognition function unit 103 described above. Integration is performed so as to maintain the sexuality, and a perception regarding each object in the external environment is provided as a short-term memory to a behavior control module such as the situation-dependent behavior hierarchy (SBL) 108.

  Therefore, on the side of the behavior control module configured as a higher-level module, a plurality of recognition results from the outside world can be integrated and handled as meaningful symbol information, and advanced behavior control can be performed. Also, by using more complicated recognition results such as correspondence problems with previously observed recognition results, which skin color area corresponds to which person on the face, which person's voice is this voice, etc. Can be solved.

  In addition, since the short-term storage unit 55 holds information about the recognized observation result as a memory, even if the observation result does not come temporarily during the autonomous action period, the aircraft's behavior control is performed. An upper module such as an application can always make an object appear to be perceived there. For example, information outside the field of view of the sensor is stored without forgetting, so that even if the robot loses sight of an object, it can be searched again later. As a result, it is possible to realize a stable system that is resistant to the error of the recognizer and the noise of the sensor and does not depend on the notification timing of the recognizer. In addition, even if there is not enough information when viewed from a single recognizer, it may be supplemented with other recognition results, so that the recognition performance of the entire system is improved.

  In addition, since related recognition results are linked, it is possible to make a behavior determination using related information in an upper module such as an application. For example, the robot apparatus can extract the name of the person based on the called voice. As a result, it is possible to notes, such as answer such as "Hello, XXX-san." The response of the greeting.

  FIG. 4 illustrates a mechanism of situation-dependent behavior control according to an external stimulus in the behavior control system 100 shown in FIG. The external stimulus is taken into the system by the function modules 101 to 103 of the recognition system and is given to the situation-dependent action hierarchy (SBL) 108 via the short-term memory unit (STM) 105. As illustrated, each function module 101 to 103 of the recognition system, the short-term storage unit (STM) 105, and the situation-dependent action hierarchy (SBL) 108 are configured as objects.

  In the figure, entities represented by circles are entities called “objects” or “processes”. The entire system operates as objects communicate asynchronously. Each object exchanges data and invokes using message communication and an inter-object communication method using a shared memory. The function of each object will be described below.

AudioRecog:
An object that receives voice data from a voice input device such as a microphone and performs feature extraction and voice section detection. Further, when the microphone is a stereo, the sound source direction in the horizontal direction can be estimated. If it is determined that it is a voice section, the feature amount and sound source direction of the voice data in that section are sent to ArterDecoder (described later).

SpeechRecog:
This is an object that performs speech recognition using the speech feature amount, the speech dictionary, and the syntax dictionary received from AudioRecog. The set of recognized words is sent to a short term memory (ShortTerm Memory) 105.

MultiColorTracker:
An object that performs color recognition, receives image data from an image input device such as a camera, extracts color areas based on a plurality of color models that are held in advance, and divides them into continuous areas. Information such as the position, size, and feature amount of each divided area is output and sent to the short-term storage unit (short term memory) 105.

FaceDetector:
An object that detects a face area from an image frame, receives image data from an image input device such as a camera, and reduces and converts it into a nine-stage scale image. A rectangular area corresponding to the face is searched from all the images. The overlapped candidate areas are reduced, and information such as the position, size, and feature amount related to the area finally determined as a face is output and sent to FaceIdentify (described later).

FaceIdentify:
An object for identifying a detected face image, a rectangular area image indicating a face area is received from FaceDetector, and a person is identified by comparing to which person in the person dictionary this face image corresponds. Do. In this case, the face image is received from the face detection, and the ID information of the person is output together with the position and size information of the face image area.

ShortTermMemory (short term memory):
An object that holds information about the external environment of the robot 1 for a relatively short time, receives a speech recognition result (word, sound source direction, certainty factor) from the SpeechRecog, and receives the position, size, and face region position of the skin color region from the MultiColorTracker , Receiving the size and receiving the ID information of the person from FaceIdentify. Further, the direction (joint angle) of the robot's neck is received from each sensor on the body of the robot 1. Then, using these recognition results and sensor output in an integrated manner, it is possible to obtain information about where the person is currently, who the spoken word belongs to, and what kind of dialogue the person has spoken so far. save. The physical information related to such an object, that is, the target and the event (history) seen in the time direction are output to an upper module such as a situation-dependent action hierarchy (SBL).

Situated Behavior layer (situation-dependent behavior hierarchy):
This is an object that determines the behavior of the robot 1 (the behavior depending on the situation) based on the information from the above-mentioned ShortTerm Memory (short-term memory). Multiple actions can be evaluated and executed at the same time. In addition, the action can be switched to put the aircraft in the sleep state and another action can be activated.

ResourceManager:
It is an object that performs resource arbitration of each hardware of the robot 1 in response to an output command. In the example shown in FIG. 4, resource arbitration is performed between an object that controls a speaker for audio output and an object that controls the motion of the neck. In this embodiment, the ResourceManager arbitrates operation commands that are expressed by the Reflexive Situated Behavior Layer (reflex action layer) and the Situated Behavior Layer (situation behavior layer), which will be described in detail later.

SoundPerformer TTS:
It is an object for performing voice output, performs voice synthesis in accordance with a text command given from the Situated BehaviorLayer via ResourceManager, and outputs voice from the speaker on the body of the robot 1.

HeadMotionGenerator:
This is an object that calculates the joint angle of the neck in response to receiving a command for moving the neck from the Situated Behavior Layer via the ResourceManager. When a “track” command is received, based on the position information of the object received from the ShortTermMemory, the joint angle of the neck facing the direction in which the object exists is calculated and output.

  The short-term storage unit 105 includes two types of memory objects, a target memory and an event memory.

  The target memory integrates information from each of the recognition function units 101 to 103, and holds information relating to the currently perceived object, that is, a target. For this reason, when the target object disappears or appears, the target is deleted from the storage area (GarbageCollector) or newly generated. One target can be expressed by a plurality of recognition attributes (Target Associate). For example, an object (human face) that emits a voice with a skin color and facial pattern.

  The position and orientation information of the object (target) held in the target memory is not a sensor coordinate system used in each of the recognition function units 51 to 53, but a specific part on the body such as the trunk of the robot 1 Expression is performed in a world coordinate system fixed at a predetermined place. For this reason, the short-term memory unit (STM) 105 constantly monitors the current values (sensor outputs) of the joints of the robot 1 and performs conversion from the sensor coordinate system to the fixed coordinate system. Thereby, it becomes possible to integrate the information of each recognition function part 101-103. For example, even when the robot 100 moves its neck or the like and the posture of the sensor changes, the position of the object viewed from the behavior control module such as the situation-dependent behavior hierarchy (SBL) remains the same, so that the target can be handled easily. Become.

  The event memory is an object that stores events from the past to the present that occurred in the external environment in time series. Examples of events handled in the event memory include information on changes in the external environment such as the appearance and disappearance of targets, speech recognition words, and changes in self behavior and posture.

  Some events include state changes related to a target. For this reason, by including the ID of the corresponding target as the event information, it is possible to retrieve more detailed information regarding the generated event from the above-described target memory.

  5 and 6 show the flow of information entering the target memory and event memory in the short-term storage unit 105 based on the recognition results in the respective recognition function units 101 to 103, respectively.

  As shown in FIG. 5, a target detector for detecting a target from the external environment is provided in the short-term storage unit 105 (STM object). The target detector adds a new target or reflects an existing target in the recognition result based on the recognition result by each of the recognition function units 101 to 103 such as a voice recognition result, a face recognition result, and a color recognition result. Or update to The detected target is held in the target memory.

  In the target memory, functions such as a garbage collector (GarbageCollector) for searching for and erasing a target that is no longer observed, and a target associate (TargetAssociate) for determining the relevance of a plurality of targets and connecting them to the same target There is. The garbage collector is realized by decrementing the certainty of the target as time passes and deleting a target whose certainty falls below a predetermined value. Further, the target associate can identify the same target by having spatial and temporal closeness between the targets having the same feature (recognition type) feature amount.

  The situation-dependent action hierarchy (SBL) described above is an object that becomes a client (STM client) of the short-term storage unit 105, and periodically receives notification (Notify) of information on each target from the target memory. In this embodiment, the STM proxy class copies the target to a client-local work area independent of the short-term storage unit 105 (STM object), and always holds the latest information. Then, a desired target is read out as an external stimulus from the local target list (Target of Interest), and a schema, that is, an action module is determined (described later).

  Further, as shown in FIG. 6, an event detector that detects an event that occurs in the external environment is provided in the short-term storage unit 105 (STM object). This event detector detects the generation of a target by the target detector and the deletion of the target by the garbage collector as events. Further, when the recognition result by the recognition function units 101 to 103 is voice recognition, the utterance content becomes an event. The events that have occurred are stored as an event list in the event memory in the order in which they occurred.

  The context-dependent action hierarchy (SBL) is an object that becomes a client (STM client) of the short-term storage unit 105, and receives event notification (Notify) from the event memory. In this embodiment, the STM proxy class copies the event list to a client-local work area independent of the short-term storage unit 105 (STM object). Then, a desired event is read from the local event list as an external stimulus, and a schema, that is, a behavior module is determined (described later). The executed behavior module is detected as a new event by the event detector. Further, old events are sequentially discarded from the event list in, for example, a FIFO (Fast In Fast Out) format.

  According to the short-term memory mechanism according to this embodiment, the robot 1 integrates the results of a plurality of recognizers related to external stimuli so as to maintain temporal and spatial consistency, and handles them as meaningful symbol information. It is like that. This makes it possible to use more complex recognition results such as correspondence problems with previously observed recognition results, which skin color area corresponds to which person on the face, and which person's voice this voice is, etc. It is possible to solve.

  Hereinafter, the dialogue processing with the users A and B by the robot 1 will be described with reference to FIGS.

  First, as shown in FIG. 7, when the user A calls “Masahiro (robot name) -kun!”, The recognition function units 51 to 53 perform sound direction detection, voice recognition, and face identification. A situation-dependent action is performed such as tracking the face of the user A or starting a dialogue with the user A.

  Next, as shown in FIG. 8, when the user B calls “Masahiro (robot name) -kun!”, Sound direction detection, voice recognition, and face identification are performed by the respective recognition function units 101 to 103. Depends on the situation in which the conversation with the user A is interrupted (however, the context of the conversation is preserved), the face of the user B is tracked in the called direction, and the conversation with the user B is started. The action is performed. This is a preemption function (described later) of the situation-dependent action hierarchy 108.

  Next, as shown in FIG. 9, when the user A yells “Oh!” And urges the continuation of the conversation, the conversation with the user B is interrupted (however, the context of the conversation is saved). A situation-dependent action is performed in which the face of the user A is tracked in the called direction or the dialogue with the user A is resumed based on the stored context. At this time, the Reentrant function (described later) of the situation-dependent action hierarchy 108 does not destroy the content of the dialogue with the user B due to the dialogue with the user A, and the dialogue can be accurately restarted from the point of interruption.

C-2. Long-term memory section Long-term memory is used to hold information obtained by learning such as the name of an object for a long period of time. The same pattern can be statistically processed for robust storage.

  Long-term memory is further classified into “declarative knowledge memory” and “procedural knowledge memory”. The declarative knowledge memory is composed of “episode memory” that is a memory related to a scene (for example, a scene when taught) and “semantic memory” that includes memories such as the meaning and common sense of words. The procedural knowledge memory is a procedural memory such as how to use a declarative knowledge memory, and can be used to acquire an operation for an input pattern.

  Episodic memory is a kind of declarative knowledge memory (also called statement memory) among long-term memories. For example, when considering riding a bicycle, remembering a scene (time, place, etc.) of riding a bicycle for the first time corresponds to episode memory. Thereafter, the memory about the episode fades with the passage of time, while the meaning memory stores the meaning. In addition, a procedure for riding a bicycle is stored, which corresponds to procedural knowledge storage. In general, it takes time to memorize procedural knowledge. Procedural knowledge memory is latent, while it can be “sayed” by declarative knowledge memory, and appears in the form of performing actions.

  The long-term storage unit 106 according to the present embodiment stores associative memory that stores sensor information related to objects such as visual information and auditory information, and results of changes in internal state as a result of actions performed on the objects, It consists of a frame storage related to the one object, map information constructed from surrounding scenes, or map information given as data, a causal situation, an action for the situation, and a result thereof.

C-2-1. Associative memory Associative memory refers to a mechanism in which an input pattern consisting of a plurality of symbols is stored in advance as a memory pattern, and a pattern similar to one of the patterns is recalled. The associative memory according to the present embodiment is realized by a model using a competitive neural network. According to such an associative memory mechanism, when a pattern having a partial defect is input, the closest stored pattern among a plurality of stored patterns can be output. This is because even when an external stimulus consisting of incomplete data is given, the meaning of an object can be recalled by the firing of the corresponding neuron.

  Associative memory is broadly divided into “self-associative associative memory” and “mutual associative associative memory”. The self-recollection type is a model in which a stored pattern is directly extracted by a key pattern, and the mutual recollection type is a model in which an input pattern and an output pattern are connected by a certain association relationship. In the present embodiment, self-associative associative memory is adopted, but this is capable of statistical storage of input patterns, which is easier to learn than conventional memory models such as Hopfield and Associatron. There are merits such as.

  According to the additional learning, even if a new pattern is newly stored, the past storage is not overwritten and erased. Also, according to statistical learning, if you see many of the same thing, it will remain in memory, and if you repeat the same thing, it will be hard to forget. In this case, even if a complete pattern is not input every time in the storing process, it is converged to a pattern that is often presented by repeated execution.

C-2-2. The pattern memorized by the semantic memory robot device 1 by associative memory is composed of, for example, a combination of an external stimulus to the robot device 1 and an internal state.

Here, the external stimulus is perceptual information obtained by the robot apparatus 1 recognizing sensor input. For example, color information, shape information, and face information processed for an image input from the camera 15. More specifically, it is composed of components such as color, shape, face, 3D general object, hand gesture, movement, voice, contact, smell, and taste.
The

  The internal state refers to emotions such as instinct and emotion based on the robot body. Instinct factors include, for example, fatigue, heat or body temperature, pain, appetite or hunger, dryness, affection, curiosity, excretion ( at least one of elimination or sexual desire. Also, emotional elements are happiness, sadness, anger, surprise, disgust, fear, frustration, boredom, sleepiness. ), Sociality, patience, tense, relaxed, alertness, guilt, spite, honesty, submission or At least one of the heels.

  In the associative memory mechanism to which the competitive neural network according to this embodiment is applied, an input channel is assigned to each element constituting these external stimuli and internal states. In addition, each perceptual function module such as the visual recognition function unit 101 and the auditory recognition function unit 102 does not send a raw signal as a sensor output, but converts the result of recognizing the sensor output into a symbol, and an ID corresponding to the symbol Information (for example, color prototype ID, shape prototype ID, voice prototype ID, etc.) is sent to the corresponding channel.

  For example, each object segmented by the color segmentation module is added to the associative memory system with a color prototype ID. Further, the face ID recognized by the face recognition module is input to the associative memory system. The ID of the object recognized by the object recognition module is input to the associative system. Also, a prototype ID of a word is input from the speech recognition module by the user's utterance. At this time, since a phoneme symbol string of an utterance is also input, it is possible to cause the robot apparatus 1 to utter by a process of storage / association. As for the instinct, an analog value can be handled (described later). For example, if the instinct delta value is stored as 80, an analog value of 80 can be obtained by association.

  Therefore, the associative memory system according to the present embodiment can store external stimuli such as color, shape, voice, and so on and an internal state as an input pattern composed of a combination of symbolized IDs for each channel. In other words, the associative memory system stores

    [Color ID Shape ID Face ID Voice ID ... Instinct ID (Value) Emotion ID]

It is a combination.

  Associative memory has a memory process and a recall process. FIG. 10 shows the concept of the storage process of associative memory.

  A memory pattern input to the associative memory system is composed of a plurality of channels assigned to each element of the external stimulus and the internal state (in the illustrated example, it consists of 8 channels of input 1 to input 8). Each channel receives ID information obtained by symbolizing the recognition result of the corresponding external stimulus and the internal state. In the example shown in the figure, the shading of each channel represents ID information. For example, when the kth column in the storage pattern is assigned to the face channel, the face prototype ID is represented by the color.

  In the example shown in FIG. 10, it is assumed that the associative memory system has already stored a total of n storage patterns 1 to n. Here, the difference in the color of the corresponding channel between the two storage patterns means that the external stimulus or the internal state symbol or ID stored on the same channel is different between the storage patterns.

  FIG. 11 shows the concept of the associative memory recall process. As described above, when a pattern similar to the input pattern stored in the storage process is input, a complete storage pattern is output so as to compensate for the missing information.

  In the example shown in FIG. 11, a pattern in which IDs are given only to the upper three channels among the storage patterns of eight channels is input as a key pattern. In such a case, the associative memory system finds a pattern (stored pattern 1 in the illustrated example) that is closest to these upper three channels among the stored patterns, and outputs the pattern as a recalled pattern. be able to. That is, the closest stored pattern is output so as to compensate for the information of the missing channels 4 to 8.

  Therefore, according to the associative memory system, it is possible to associate a voice ID, that is, a name only from the face ID, or to recall “delicious” or “not delicious” from only the food name. According to the long-term memory architecture based on the competitive neural network, the semantic memory about the meaning of words and common sense can be realized with the same engineering model as other long-term memories.

C-2. Associative Learning by Competitive Neural Network FIG. 12 schematically shows a configuration example of an associative memory system to which a competitive neural network is applied. As shown in the figure, this competitive neural network is a hierarchical neural network composed of two layers: an input layer and a competitive layer.

  This competitive neural network has two modes of operation: memory mode and associative mode. In the memory mode, the input pattern is memorized in a competitive manner, and in the recall mode, the input pattern is partially lost. Recalling a memory pattern.

  The input layer is composed of a plurality of input neurons. Each input neuron receives a symbol corresponding to the recognition result of the external stimulus or the internal state, that is, ID information, from a channel assigned to each element representing the external stimulus or the internal state. In the input layer, it is necessary to prepare a number of neurons corresponding to the number of color IDs + the number of shape IDs + the number of voice IDs + the type of instinct.

  The competitive layer is composed of a plurality of competitive neurons. Each competitive neuron is connected to each input neuron on the input layer side with a certain connection weight. A competitive neuron corresponds to one symbol that each neuron should memorize. In other words, the number of competing neurons corresponds to the number of symbols that can be stored.

  Assume that an input pattern is given to the input layer. At this time, the input pattern is composed of channels representing each element of the external stimulus and the internal state, and the input neuron that receives the corresponding ID from the channel is fired.

  The competing neuron inputs the output from each input neuron with synaptic weighting, and calculates the sum of those input values. Then, learning is performed by selecting a competitive neuron having the maximum sum of input values in the competitive layer and strengthening the binding power between the winning competitive neuron and the input neuron. Also, by selecting a competitive neuron that has won in the competitive layer for a defective input pattern, a symbol corresponding to the input pattern can be recalled.

Memory mode:
The connection weight between the input layer and the competitive layer takes a value between 0 and 1. However, the initial connection weight is determined randomly.

  Storage in the competitive neural network is performed by first selecting a competitive neuron that has won in the competitive layer for the input pattern to be stored, and strengthening the coupling force between the competitive neuron and each input neuron.

Here, the input pattern vector _{[x 1, x 2, ...} , x n] are neurons, corresponding to the color prototype ID1, When ID1 has been recognized, ignite neurons x _1, successively, form, also voice Let it fire like that. A fired neuron takes a value of 1, and a non-fired neuron takes a value of -1.

Further, if the connection force between the i-th input neuron and the j-th competitive neuron is set to w _ij , the value of the competitive neuron y _j for the input x _i is expressed by the following equation.

  Therefore, the neuron that wins the competition can be obtained by the following equation.

  Memorization is performed by strengthening the binding force between the competitive neuron (winner neuron) won in the competitive layer and each input neuron. The connection between the winning neuron and the input neuron is updated as follows according to the Kohonen update rule.

  Here, normalization is performed using L2 Norm.

  This binding force represents the so-called memory strength and becomes the memory power. Here, the learning rate α is a parameter representing the relationship between the number of times of presentation and storage. The greater the learning rate α, the larger the weight is changed with one memory. For example, if α = 0.5 is used, once it is stored, it is not forgotten. If the same pattern is presented next time, the stored pattern can be almost certainly associated.

  In addition, the network connection value (weight) increases as the information is presented and stored. This indicates that the memory becomes stronger as the same pattern is input many times, statistical learning is possible, and long-term memory with little influence of noise in a real environment can be realized.

  In addition, if a new pattern is input and an attempt is made to memorize, a new competitive layer of neurons fires, so that the connection with the new neuron is strengthened and the connection with the neuron from the previous memory is not weakened. In other words, the associative memory by the competitive neural network allows additional learning and is freed from the problem of “forgetting”.

Recall mode:
Assume that an input pattern vector as shown below is presented to the associative memory system shown in FIG. The input pattern is not complete and may be partially missing.

At this time, the input vector may be a prototype ID, or may be a likelihood and a probability for the prototype ID. The value of the output neuron y _j is calculated for the input x _i as

  It can be said that the above expression represents the likelihood of the firing value of the competing neuron according to the likelihood of each channel. What is important here is that it is possible to obtain the overall likelihood by connecting the likelihood inputs from a plurality of channels. In the present embodiment, only one associated with the maximum likelihood, that is, the one with the maximum likelihood is selected, and the neuron that can win the competition can be obtained by the following equation.

  Since the obtained number of the competitive neuron Y corresponds to the stored symbol number, the input pattern X can be recalled by the inverse matrix operation of W as shown in the following equation.

  Further, declarative knowledge storage and procedural knowledge storage can be realized by an associative memory architecture by assigning symbols such as episodes and action IDs to the input layer neurons of the competitive neural network shown in FIG.

D. Situation-dependent action control In this embodiment, a situation-dependent action hierarchy 108 that autonomously selects an action depending on the internal state or its change and an external stimulus, and reflexive / direct airframe action according to the recognized external stimulus And a contemplation action layer 107 that performs a relatively long-term action plan based on the stored contents, and the action of the robot is selected by these hierarchical action control mechanisms.

  The situation-dependent action hierarchy (Situated BehaviorsLayer) 108 is an action control layer that controls a spontaneous action in response to a situation where the robot apparatus 1 is currently placed, and includes a plurality of element actions. Elemental actions are described as action modules or objects called “schema”. Each elemental action periodically calculates the action value from the internal state, the recognition result stored in the short-term memory unit 105 or the long-term memory unit 106, and the action value of each elemental action (hereinafter referred to as “AL value”). Based on the above, it is selected which of the elemental actions should be expressed, and the action is output.

D-1. Configuration of Situation Dependent Action Hierarchy In this embodiment, the situation dependent action hierarchy 108 prepares a state machine for each action module, and the recognition result of the external information input by the sensor depending on the previous action and situation. Classify the behavior on the aircraft. The behavior module is described as a schema having a monitor function that makes a situation determination according to an external stimulus or a change in an internal state, and an action function that realizes a state transition (state machine) associated with behavior execution. The situation-dependent action hierarchy 108 is configured as a tree structure in which a plurality of schemas are hierarchically connected (described later).

  The situation-dependent action hierarchy 108 also realizes an action for keeping the internal state within a certain range (also referred to as “homeostasis action”). When the internal state exceeds the specified range, the internal state is The action is activated so that the action for returning to the range can be easily performed (actually, the action is selected in consideration of both the internal state and the external environment).

  Each functional module in the behavior control system 100 of the robot 1 as shown in FIG. 3 is configured as an object. Each object can perform data transfer and invoke by message communication and an inter-object communication method using a shared memory. FIG. 13 schematically shows the object configuration of the behavior control system 100 according to the present embodiment.

  The visual recognition function unit 101 includes three objects, “FaceDetector”, “MultiColtTracker”, and “FaceIdentify”.

  FaceDetector is an object that detects a face area from an image frame, and outputs the detection result to FaceIdentify. The MultiClotTracker is an object that performs color recognition, and outputs the recognition result to FaceIdentify and ShortTermMemory (an object constituting the short-term memory 105). FaceIdentify also identifies a person by searching the detected face image in a hand-held person dictionary, and outputs the person ID information together with the position and size information of the face image area to ShortTermMemory.

  The auditory recognition function unit 102 includes two objects “AudioRecog” and “SpeechRecog”. AudioRecog is an object that receives audio data from an audio input device such as a microphone and performs feature extraction and audio segment detection, and outputs the feature amount and sound source direction of the audio data in the audio segment to SpeedRecog and ShortTermMemory. The SpeechRecog is an object that performs speech recognition using the speech feature amount, the speech dictionary, and the syntax dictionary received from the AudioRecog, and outputs a set of recognized words to the ShortTermMemory.

  The tactile sensation recognition storage unit 103 includes an object called “TactileSensor” that recognizes a sensor input from a contact sensor, and outputs a recognition result to the ShortTermMemory and an internal state model (ISM) that manages an internal state.

  ShortTermMemory (STM) is an object constituting the short-term storage unit 105, and holds targets and events recognized from the external environment by each object of the recognition system described above (for example, an input image from the camera 15 is about 15 seconds). This is a functional module that stores only for a short period of time, and periodically notifies the Stimated BehaviorsLayer, which is an STM client, of external stimuli (Notify).

  LongTermMemory (LTM) is an object that constitutes the long-term storage unit 106 and is used to hold information obtained by learning such as the name of an object for a long period of time. For example, LongTermMemory can associatively store a change in internal state from an external stimulus in a certain behavior module.

  The Internal Status Manager (ISM) is an object that constitutes the internal state management unit 104, manages several types of emotions such as instinct and emotion by modeling them, and external stimuli (ES that are recognized by each object of the recognition system described above. : Manages the internal state of the robot apparatus 1 such as instinct and emotion according to (ExternalStimula).

  Situated Behaviorslayer (SBL) is an object that constitutes the context-dependent action hierarchy 108. SBL is an object that becomes a client (STM client) of ShorTermMemory. When a notification (Notify) of information related to external stimuli (targets and events) is periodically received from ShorTermMemory, a schema (scheme), that is, an action module to be executed. Is determined (described later).

  The Reflexive Situated BehaviorsLayer is an object that constitutes the reflexive action unit 109, and executes reflexive and direct body motion according to the external stimulus recognized by each object of the recognition system described above. For example, a behavior such as chasing a human face, nodding, or avoiding a trap by detecting an obstacle is performed (described later).

  The Situated Behaviors layer (situation-dependent action hierarchy) selects an action according to a situation such as an external stimulus or a change in an internal state. In contrast, Reflexive Situated BehaviorsLayer acts reflexively in response to external stimuli. In addition, the Situated Behavior layer also manages the reflex behavior of the robot device as an external stimulus, so that when the reflex behavior does not match the intention of the behavior according to the situation, the behavior command expressed from the Reflexive Behavior Layer (reflective behavior layer) Suppress execution (described later). Or conversely, when it is desired to make the reflex behavior appear, the Reflexive BehaviorLayer is excited.

  Since the behavior selection by two objects of Refractive Situated Behaviors Layer and Situated Behavior Layer is performed independently, when the behavior modules (schema) selected from each other are executed on the aircraft, the hardware resources of the robot 1 are in conflict and cannot be realized. Sometimes it is. The ResourceManager object mediates hardware conflicts when selecting actions by the Situated Behaviorslayer and the Reflexive Situated Behaviors Layer (described later). Then, the airframe is driven by notifying each object that realizes the airframe motion based on the arbitration result.

  SoundPerformer, MotionController, and LedController are objects that realize the body operation. The SoundPerformer is an object for performing voice output, performs voice synthesis in accordance with a text command given from the SituatedBehaviorLayer via the ResourceManager, and outputs voice from the speaker on the body of the robot 1. The Motion Controller is an object for performing the operation of each joint actuator on the aircraft, and calculates a corresponding joint angle in response to receiving a command to move a hand, leg, or the like from the Situated BehaviorLayer via the ResourceManager. The LedController is an object for performing the blinking operation of the LED 19, and performs the blinking drive of the LED 19 in response to receiving a command from the Situated BehaviorLayer via the ResourceManager.

  FIG. 14 schematically shows a form of situation-dependent action control by the situation-dependent action hierarchy (SBL) 108 (however, including the reflex action unit 109). The recognition result of the external environment by the recognition systems 101 to 103 is given to the situation-dependent action hierarchy 108 (including the reflex action part 109) as an external stimulus. In addition, a change in the internal state according to the recognition result of the external environment by the recognition system is also given to the situation-dependent action hierarchy 108. In the situation-dependent action hierarchy 108, the situation can be determined according to an external stimulus or a change in the internal state, and action selection can be realized.

  FIG. 15 shows a basic operation example of action control by the situation-dependent action hierarchy 108 shown in FIG. As shown in the figure, in the situation-dependent action hierarchy 108 (SBL), the activity level of each action module (schema) is calculated based on an external stimulus or a change in the internal state, and the schema is determined according to the degree of the activity level. Select and execute an action. For the calculation of the activity level, for example, by using a library, unified calculation processing can be performed for all schemas (the same applies hereinafter). For example, a schema with the highest activity level may be selected, or two or more schemas exceeding a predetermined threshold may be selected and actions may be executed in parallel. (Assuming there are no hardware resource conflicts between schemas).

  FIG. 16 shows an operation example in the case of performing a reflex action by the situation-dependent action hierarchy 108 shown in FIG. In this case, as shown in the figure, the reflexive action unit 109 (ReflexiveSBL) included in the situation-dependent action hierarchy 108 calculates the activity level by directly inputting the external stimulus recognized by each object of the recognition system, Select a schema according to the level of activity level and execute an action. In this case, the change in internal state is not used for the activity level calculation.

  FIG. 17 shows an operation example in the case where emotion expression is performed by the situation-dependent action hierarchy 108 shown in FIG. The internal state management unit 104 manages emotions such as instinct and emotion as a mathematical model, and in response to the state value of the emotion parameter reaching a predetermined value, changes in the internal state are made to the situation-dependent action hierarchy 108. Notify. The situation-dependent behavior hierarchy 108 calculates an activity level using an internal state change as an input, selects a schema according to the level of the activity level, and executes an action. In this case, the external stimulus recognized by each object of the recognition system is used for management / update of the internal state in the internal state management unit 104 (ISM), but is not used for calculating the activity level of the schema.

D-2. The schema situation-dependent action hierarchy 108 prepares a state machine for each action module, and classifies the recognition results of external information input from the sensor depending on the action and situation before that, and the action is displayed on the aircraft. Expressed in The action module describes the action of the action described in the action function according to the external stimulus and the internal state, and describes the situation determination by describing the body motion and realizing the state transition (state machine) accompanying the action execution. It is described as a schema having a Monitor function to be performed.

  FIG. 18 schematically shows that the situation-dependent action hierarchy 108 is composed of a plurality of schemas. FIGS. 13 and 18 show that the operation commands from the Situated Behavior Layer and the Reflexive Situated Behavior Layer are conflict-resolved in the Resource Management module of the robot apparatus 1 (described later).

  The situation-dependent action hierarchy 108 (more strictly speaking, the hierarchy that controls the normal situation-dependent action among the situation-dependent action hierarchy 108) is configured as a tree structure in which a plurality of schemas are hierarchically connected. In accordance with changes in the internal state, a more optimal schema is determined in an integrated manner to perform action control. The tree includes a plurality of subtrees (or branches) such as a behavior model obtained by formulating an ethological situation-dependent behavior and a subtree for executing emotion expression.

  FIG. 19 schematically shows a schema tree structure in the situation-dependent action hierarchy 108. As shown in the figure, the situation-dependent action hierarchy 108 is directed from the abstract action category to the specific action category, starting with the root schema that receives the notification (Notify) of the external stimulus from the short-term storage unit 105. A schema is arranged for each hierarchy. For example, in the hierarchy immediately below the root schema, schemas “Search”, “Insert”, and “Play” are arranged. Then, below “Investigate”, a schema describing more specific search behaviors such as “Investigative Location”, “HeadinAirSniffing”, and “InvestigativeSniffing” is arranged. Similarly, a schema describing more specific eating and drinking behavior such as “Eat” and “Drink” is arranged below the schema “Ingestive”, and “Schema” is placed below the “Play”. Schemas describing more specific playing behaviors such as “PlayBowing”, “PlayGreeting”, and “PlayPawing” are arranged.

  As shown, each schema inputs an external stimulus and an internal state. Each schema has at least a Monitor function and an Action function.

  FIG. 20 schematically shows the internal structure of the schema. As shown in the figure, the schema evaluates each state of the Action function according to the external function and the internal state as an activity level value by describing the action of the body in the form of state transition (state machine). The Monitor function to be returned and a state management unit that stores and manages the state of the schema as the READY (ready), ACTIVE (active), or SLEEP (standby) state machine of the Action function.

  The Monitor function is a function that calculates an activity level (Activation Level: AL value) of the schema in accordance with an external stimulus and an internal state. When the tree structure as shown in FIG. 19 is configured, the upper (parent) schema can call the Monitor function of the lower (child) schema with the external stimulus and the internal state as arguments, and the child schema has an AL value. Is the return value. The schema can also call the child's schema Monitor function to calculate its AL value. Since the AL value from each sub-tree is returned to the root schema, the optimum schema corresponding to the external stimulus and the change of the internal state, that is, the behavior can be determined in an integrated manner.

  For example, a schema having the highest AL value may be selected, or two or more schemas having an AL value exceeding a predetermined threshold value may be selected and executed in parallel (however, when executing in parallel) (Assuming there is no hardware resource conflict between schemas).

  FIG. 21 schematically shows the internal configuration of the Monitor function. As shown in the figure, the Monitor function includes an action induction evaluation value calculator that calculates an evaluation value that induces the action described up to the ski as an activity level, and an intention level (Intention Level: (IV value) and a used resource calculator for specifying an expected resource to be used.

  The IV value calculated by the bias calculator is, for example, suitable for the case where the robot apparatus 1 is prevented from operating against the intention of the situation-dependent action during the execution of the situation-dependent action or the situation-dependent intention. Used for excitation. This IV value is also used when the contemplation action hierarchy (DeliverativeLayer) makes a situation-dependent action expression plan and manages the order in which the situation-dependent action occurs.

  In the example shown in FIG. 20, when the Monitor function is called from a behavior state control unit (tentative name) that manages a schema, that is, a behavior module, it virtually executes the state machine of the Action function, and the activity level and the resource used Is calculated and returned.

  Further, the Action function includes a state machine (described later) that describes the behavior of the schema itself. When the tree structure shown in FIG. 19 is configured, the parent schema can call the Action function to start or interrupt the execution of the child schema. In the present embodiment, the action state machine is not initialized unless it becomes Ready. In other words, even if it is interrupted, the state is not reset, and the work data being executed by the schema is saved, so that it can be interrupted and reexecuted (described later).

  In the example shown in FIG. 20, a behavior state control unit (tentative name) that manages a schema, that is, a behavior module, selects an action to be executed based on a return value from the Monitor function, and calls an action function of the corresponding schema. Alternatively, the transition of the schema state stored in the state management unit is instructed. For example, the schema having the highest activity level as the action induction evaluation value is selected, or a plurality of schemas are selected according to the priority order so that resources do not compete. In addition, when a schema having a higher priority is activated and resource conflict occurs, the behavior state control unit saves the state of the schema having a lower priority from ACTIVE to SLEEP, and when the conflict state is solved, the behavior state control unit changes to ACTIVE. Control schema state, such as recovery.

  As shown in FIG. 22, only one behavior state control unit may be provided in the situation-dependent behavior hierarchy 108 so that all schemas constituting the hierarchy 108 are centrally managed.

  In the illustrated example, the behavior state control unit includes a behavior evaluation unit, a behavior selection unit, and a behavior execution unit. For example, the behavior evaluation unit calls the Monitor function of each schema at a predetermined control cycle, and acquires each activity level and resource used. The action selection unit performs action control and aircraft resource management by each schema. For example, schemas are selected in descending order of activity level, and two or more schemas are simultaneously selected so that resources used do not conflict. The behavior execution unit issues a behavior execution command to the Action function of the selected schema, manages the schema status (READY, ACTIVE, SLEEP), and controls the execution of the schema. For example, when a schema with a higher priority is activated and a resource conflict occurs, the state of the schema with a lower priority is saved from ACTIVE to SLEEP, and when the conflict state is solved, it is restored to ACTIVE.

  Or you may make it arrange | position the function of such an action state control part for every schema in the situation dependence action hierarchy 108. FIG. For example, as shown in FIG. 19, when the schema forms a tree structure (see FIG. 23), the behavioral state control of the higher-level (parent) schema is the lower-level with external stimulus and internal state as arguments. The Monitor function of the (child) schema is called, and the activity level and the used resource are received as return values from the child schema. In addition, the child schema further calls the Monitor function of the child schema in order to calculate its activity level and resource used. Then, the activity level control unit and the resources used from each sub-tree are returned to the action state control unit of the root schema, so that the optimum schema corresponding to the external stimulus and the change of the internal state, that is, the action is integrally determined. , Call the Action function to start or interrupt the execution of the child schema.

  FIG. 24 schematically shows a mechanism for controlling a normal situation-dependent action in the situation-dependent action hierarchy 108.

  As shown in the figure, an external stimulus is input (Notify) from the short-term storage unit 105 and a change in the internal state is input from the internal state management unit 109 to the situation-dependent action hierarchy 108. The situation-dependent action hierarchy 108 is composed of a plurality of subtrees such as an action model obtained by formulating an animal behavioral (ethological) situation-dependent action, and a subtree for executing emotion expression. In response to the notification (Notify) of the external stimulus, the monitor function of each sub-tree is called, and the activity level (AL value) as the return value is referred to, and the integrated action selection is performed and the selected function is selected. Call the action function on the subtree that realizes the action. In addition, the situation-dependent action determined in the situation-dependent action hierarchy 108 is applied to the aircraft operation (Motion Controller) through the mediation of hardware resource competition with the reflex action by the reflex action unit 109 by the resource manager. .

  Further, in the situation-dependent action layer 108, the reflexive action unit 109 executes reflexive and direct aircraft actions according to the external stimulus recognized by each object of the recognition system described above (for example, an obstacle Avoid by detection) For this reason, unlike the case where normal situation-dependent behavior is controlled (FIG. 19), a plurality of schemas for directly inputting signals from each object of the recognition system are arranged in parallel without being hierarchized. .

  FIG. 25 schematically shows a schema configuration in the reflex action unit 109. As shown in the figure, the reflex action unit 109 operates in response to the recognition result of the visual system, “AvoidBigSound”, “FacetoBigSound” and “NodingSound” as schemas that operate in response to the recognition result of the auditory system. “FacetoMovingObject” and “AvoidMovingObject” as schemas, and “retract hand” as schemas operating in response to the recognition result of the tactile system are arranged in an equal position (in parallel).

  As shown, each schema that performs reflex behavior has an external stimulus as input. Each schema has at least a monitor function and an action function. The monitor function calculates the AL value of the schema in accordance with the external stimulus, and determines whether or not the corresponding reflex behavior should be expressed according to the calculated AL value. The action function includes a state machine (described later) describing the reflexive behavior of the schema itself. When called, the action function expresses the corresponding reflexive behavior and changes the state of the action.

  FIG. 26 schematically shows a mechanism for controlling the reflex behavior in the reflex behavior unit 109.

  As shown also in FIG. 25, in the reflex action part 109, the schema which described the reaction action and the schema which described the immediate response action exist in parallel. When a recognition result is input from a recognition system object, the corresponding reflex behavior schema calculates an AL value by the monitor function, and it is determined whether or not the action should be trajected according to the value. The reflex behavior determined to be activated by the reflex behavior unit 109 is applied to the aircraft operation (Motion Controller) through the mediation of hardware resource competition with the reflex behavior by the reflex behavior unit 109 by the resource manager. The

The schema constituting the situation-dependent action hierarchy 108 (including the reflex action part 109) can be described as, for example, a “class object” described in a C ⁺⁺ language base. FIG. 27 schematically shows a schema class definition used in the situation-dependent behavior hierarchy 108. Each block shown in the figure corresponds to one class object.

  As shown in the figure, the situation-dependent behavior hierarchy (SBL) 108 includes one or more schemas, an EventDataHandler (EDH) that allocates an ID to an SBL input / output event, a SchemaHandler (SH) that manages the schema in the SBL, 1 or more ReceiveDataHandler (RDH) which receives data from external objects (STM, LTM, resource manager, each object of recognition system, etc.) and 1 or more SendDataHandler (SDH) which transmits data to an external object Yes.

  EventDataHandler (EDH) is a class object for allocating IDs for SBL input / output events, and receives notification of input / output events from RDH and SDH.

  SchemaHandler stores information (SBL configuration information) such as each schema and tree structure constituting the situation-dependent action hierarchy (SBL) 108 and the reflex action part 109. For example, when the system is started up, SchemaHandler reads this configuration information file, constructs (reproduces) the schema structure of the situation-dependent action hierarchy 108 as shown in FIG. 19, and stores each schema in the memory space. Map entities.

  Each schema has OpenR_Guest positioned as the base of the schema. OpenR_Guest includes at least one class object called Dsub for the schema to transmit data to the outside and DO object for the schema to receive data from the outside. For example, when the schema sends data to an external object of SBL (STM, LTM, recognition system objects, etc.), Dsubject writes transmission data to SendDataHandler. In addition, DOObject can read data received from an external object of SBL from ReceiveDataHandler.

  SchemaManager and SchemaBase are both class objects that inherit OpenR_Guest. Class inheritance is inheriting the definition of the original class. In this case, it means that SchemaManager and SchemaBase are also provided with class objects such as Dsubject and DOObject defined in OpenR_Guest (the same applies hereinafter). For example, as shown in FIG. 19, when a plurality of schemas have a tree structure, SchemaManager has a class object SchemaList (which has a pointer to a child schema) that manages a list of child schemas, and has a child schema. A function can be called. SchemaBase has a pointer to the parent schema and can return a return value of a function called from the parent schema.

  SchemaBase has two class objects, StateMachine and Pronome. StateMachine manages a state machine for behavior (Action function) of the schema. FIG. 28 illustrates a state machine for schema behavior (Action function). Actions are associated with the transitions between the states of the state machine.

  The parent schema can switch (change state) the state machine of the action function of the child schema. Further, a target to which the schema executes or applies an action (Action function) is substituted for Proname. As will be described later, the schema is occupied by the target assigned to Pronome, and the schema is not released until the action is completed (completed, abnormally terminated, etc.). In order to perform the same action for a new target, a schema with the same class definition is generated in the memory space. As a result, the same schema can be executed independently for each target (the work data of the individual schemas do not interfere with each other), and behavioral reentrance (described later) is ensured.

  ParentSchemaBase is a class object that inherits multiple of SchemaManager and SchemaBase, and manages a parent schema and a child schema, that is, a parent-child relationship for the schema itself in the tree structure of the schema.

  IntermediaParentSchemaBase is a class object that inherits ParentSchemaBase and implements interface conversion for each class. In addition, IntermediaParentSchemaBase has SchemaStatusInfo. This SchemaStatusInfo is a class object that manages the state machine of the schema itself.

  The parent schema can switch the state of its state machine by calling the action function of the child schema. Also, it is possible to ask the AL value corresponding to the state of the state machine by calling the Aonitor function of the child schema. However, it should be noted that the schema state machine is different from the action function state machine described above.

  FIG. 29 illustrates a state machine for actions described by a schema itself, that is, an Action function. As already mentioned, the state machine of the schema itself defines three states of READY (ready), ACTIVE (active), and SLEEP (standby) for actions described by the Action function. Yes. When a schema with a higher priority is activated and resource conflict occurs, the state of the schema with the lower priority is saved from ACTIVE to SLEEP, and when the conflict state is solved, it is restored to ACTIVE.

  As shown in FIG. 29, ACTIVE_TO_SLEEP is defined for the state transition from ACTIVE to SLEEP, and SLEEP_TO_ACTIVE is defined for the state transition from SLEEP to ACTIVE. What is characteristic in this embodiment is

(1) A process for saving data (context) necessary for resuming the transition to ACTIVE later and an action necessary for SLEEP are associated with ACTIVE_TO_SLEEP.
(2) SLEEP_TO_ACTIVE is associated with a process for restoring saved data (context) and an action necessary for returning to ACTIVE.

That is the point. The action necessary for the SLEEP is, for example, an action of saying a line such as “Please wait for a while” to tell the other party to pause (other gesture gestures may be added). The action required to return to ACTIVE is, for example, an action that says a line such as “send me a wait” that expresses gratitude to the other party (others may be gesture gestures added).

  AndParentSchema, NumOrParentSchema, and OrParentSchema are class objects that inherit from IntermediateMediaSchemaBase. AndAndParentSchema has pointers to multiple child schemas that are executed simultaneously. OrParentSchema has pointers to multiple child schemas to be executed alternatively. NumOrParentSchema has pointers to a plurality of child schemas that execute only a predetermined number at the same time.

  ParentSchema is a class object that inherits these AndParentSchema, NumOrParentSchema, and OrParentSchema multiple times.

  FIG. 30 schematically shows a functional configuration of classes in the situation-dependent action hierarchy (SBL) 108.

  The Situation Dependent Action Hierarchy (SBL) 108 includes one or more Receive Data Handlers (RDH) that receive data from external objects such as STM, LTM, resource manager, and recognition system objects, and one or more that transmit data to external objects. SendDataHandler (SDH).

  EventDataHandler (EDH) is a class object for allocating IDs for SBL input / output events, and receives notification of input / output events from RDH and SDH.

  The SchemaHandler is a class object for managing the schema, and stores configuration information of the schema constituting the SBL as a file. For example, when the system is started up, SchemaHandler reads this configuration information file and constructs a schema configuration in the SBL.

  Each schema is generated according to the class definition shown in FIG. 27, and entities are mapped on the memory space. Each schema uses OpenR_Guest as a base class object, and includes class objects such as DSubject and DOObject for data access to the outside.

  The functions and state machines that the schema has are shown below.

ActivationMonitor (): an evaluation function for becoming active when the schema is ready.
Actions (): State machine for execution at the time of Active.
Goal (): A function that evaluates whether the schema has reached Goal at the time of Active.
Goal (): A function that determines whether the schema is in a fail state at the time of Active.
SleepActions (): State machine executed before Sleep.
SleepMonitor (): an evaluation function for Resume at the time of Sleep.
ResumeActions (): State machine for Resume before Resume.
DestroyMonitor (): an evaluation function that determines whether the schema is in a fail state during Sleep.
MakePronome (): A function that determines the target of the entire tree.

  These functions are described in SchemaBase.

  FIG. 31 shows a processing procedure for executing the MakePronome function in the form of a flowchart.

  When the MakePronome function of a schema is called, it is first determined whether or not a child schema exists in the schema confidence (step S1).

  If there is a child schema, the MakePronom function of all child schemas is similarly called recursively (step S2).

  Then, MakeProname of the schema itself is executed, and the target is assigned to the Pronome object (step S3).

  As a result, the same target is assigned to Pronom of all the schemas below itself, and the schema is not released until the action ends (complete, abnormal end, etc.). In order to perform the same action for a new target, a schema with the same class definition is generated in the memory space.

  FIG. 32 shows a processing procedure for executing the Monitor function in the form of a flowchart.

  First, the evaluation flag (AssessmentFlag) is set to ON (step S11), and the action of the schema itself is executed (step S12). At this time, the child schema is also selected. Then, the evaluation flag is turned off (step S13).

  If a child schema exists (step S14), the Monitor function of the child schema selected in step S12 is recursively called (step S15).

  Next, the Monitor function of the schema itself is executed (step S16), the activity level and the resource used for action execution are calculated (step S17), and set as the return value of the function.

  FIG. 33 and FIG. 34 show the processing procedure for executing the Actions function in the form of a flowchart.

  First, it is checked whether or not the schema is in a STOPPING state (step S21), and then it is checked whether or not it is in a STOPPING state (step S22).

  If it is a state to be stopped, it is further checked whether or not there is a child schema (step S23). If there is a child schema, it is shifted to the GO_TO_STOP state (step S24), and then the HaveToStopFlag is turned on (step S25).

  If the state is not to be stopped, it is checked whether or not it is in the RUNNING state (step S26).

  If it is not in the RUNNING state, it is further checked whether or not there is a child schema (step S27). If there is a child schema, the HaveToStopFlag is turned on (step S28).

  Next, the next own state is determined from the current system state, HaveToRunFlag, HaveToStopFlag, and the operation state of the child schema (step S29).

  Next, the Action function of the schema itself is executed (step S30).

  Thereafter, it is checked whether or not the schema itself is in a GO_TO_STOP state (step S31). If it is not in the GO_TO_STOP state, it is further checked whether there is a child schema (step S32). If there is a child schema, it is checked whether there is a child schema in the GO_TO_STOP state (step S33).

  If there are child schemas in the GO_TO_STOP state, the Action function of these schemas is executed (step S34).

  Next, it is checked whether or not there is a child schema in RUNNING (step S35). If there is no child schema in RUNNING, it is checked whether there is a child schema that is stopped (step S36), and the action function of the stopped child schema is executed (step S37).

  Next, it is checked whether there is a child schema in the GO_TO_RUN state (step S38). If there is no child schema in the GO_TO_RUN state, it is checked whether there is a child schema in the GO_TO_STOP state (step S39). If so, the action function of this child schema is executed (step S40).

  Finally, the next state of itself is determined from the current system state, HaveToRunFlag, HaveToStopFlag, and the child's operating state, and the entire processing routine is terminated (step S41).

D-3. The function- dependent behavior hierarchy (situated behaviors layer) 108 of the situation-dependent behavior hierarchy is based on the storage contents of the short-term storage unit 105 and the long-term storage unit 106 and the internal state managed by the internal state management unit 104. Control behaviors that respond quickly to your situation. Further, in consideration of the IV value, control is performed so that a situation-dependent behavior suitable for the intention of the contemplation behavior can be expressed.

  As described in the previous section, the situation-dependent action hierarchy 108 according to the present embodiment is configured by a schema tree structure (see FIG. 19). Each schema is independent with knowledge of its child and parent information. With such a schema configuration, the situation-dependent behavior hierarchy 108 has the main characteristics of current evaluation, current execution, preemption, and reentrant. Hereinafter, these features will be described in detail.

(1) Current evaluation:
It has already been described that the schema as the behavior module has a Monitor function for judging the situation according to the external stimulus and the change of the internal state. The Monitor function is implemented by providing the Monitor function in the schema with the class object SchemaBase. The Monitor function is a function that calculates an activity level (Activation Level: AL value) of the schema in accordance with an external stimulus, an internal state, and an IV value.

  When the tree structure as shown in FIG. 19 is configured, the upper (parent) schema can call the Monitor function of the lower (child) schema with the external stimulus and the internal state as arguments, and the child schema has an AL value. Is the return value. The schema can also call the child's schema Monitor function to calculate its AL value. Since the AL value from each sub-tree is returned to the root schema, the optimum schema corresponding to the external stimulus and the change of the internal state, that is, the behavior can be determined in an integrated manner.

  Since the tree structure is formed in this way, the evaluation of each schema based on the external stimulus and the change in the internal state is first performed to the current from the bottom to the top of the tree structure. As shown in the flowchart of FIG. 32, if the schema has a child schema (step S14), the selected child's Monitor function is called (step S15), and then the own Monitor function is executed.

  Next, the execution permission as the evaluation result is passed from the top to the bottom of the tree structure. Evaluation and execution are performed while solving the competition of resources used by the action.

  Since the situation-dependent action hierarchy 108 according to the present embodiment can evaluate actions in parallel using the schema tree structure, it is adaptable to situations such as external stimuli and internal states. . Further, at the time of evaluation, the entire tree is evaluated, and the tree is changed by the activity level (AL) value calculated at this time, so that the schema, that is, the action to be executed can be dynamically prioritized.

(2) Current execution:
Since the AL value from each sub-tree is returned to the root schema, it is possible to integrally determine the optimal schema, that is, the behavior corresponding to the external stimulus and the change in the internal state. For example, a schema having the highest AL value may be selected, or two or more schemas having an AL value exceeding a predetermined threshold value may be selected and be executed in parallel (however, each schema is executed in parallel execution) (Assuming there is no hardware resource conflict between each other).

  Schemas that have permission to execute are executed. In other words, the schema actually executes the command by observing more detailed external stimuli and internal state changes. Regarding execution, it is performed sequentially from the top to the bottom of the tree structure, that is, to the current. As shown in the flowcharts of FIGS. 33 and 34, if the schema includes a child schema, the child Actions function is executed.

  The Action function includes a state machine (described later) that describes the behavior of the schema itself. When the tree structure shown in FIG. 19 is configured, the parent schema can call the Action function to start or interrupt the execution of the child schema.

  The context-dependent behavior hierarchy 108 according to the present embodiment can simultaneously execute other schemas that use surplus resources when resources do not compete using the schema tree structure. However, if there are no restrictions on the resources used before Goal, there is a possibility that a stupid behavior will occur. The situation-dependent action determined in the situation-dependent action hierarchy 108 is applied to the aircraft operation (Motion Controller) through the mediation of hardware resource competition with the reflex action by the reflex action unit 109 by the resource manager.

(3) Preemption:
Even if the schema has been moved to once, if there is a more important (higher priority) action, the schema must be interrupted and the right to execute must be passed to it. It is also necessary to resume the original schema and continue execution when more important actions are completed (completion or execution stop, etc.).

  The execution of tasks according to such priorities is similar to a function called Preemption of OS (Operating System) in the computer world. The OS has a policy of sequentially executing tasks with higher priorities at a timing considering the schedule.

  On the other hand, since the behavior control system 100 of the robot 1 according to the present embodiment spans a plurality of objects, arbitration between the objects is necessary. For example, Reflexive SBL, which is an object that controls reflex behavior, needs to avoid or balance things without worrying about the behavior evaluation of SBL, which is an object that controls higher-level situation-dependent behavior. This means that the execution right is actually taken and executed, but the higher-level action module (SBL) is notified that the execution right has been taken, and the higher-level preemptive ability is obtained by performing the processing. Hold.

  In the situation-dependent behavior layer 108, it is assumed that execution of a certain schema is permitted as a result of the evaluation of the AL value based on the external stimulus and the change in the internal state. Furthermore, it is assumed that the importance of another schema becomes higher due to the evaluation of the AL value based on the subsequent external stimulus and the change in the internal state. In such a case, it is possible to switch the preemptive behavior by using the Actions function of the schema being executed and suspending the sleep state.

  The state of Actions () of the schema being executed is saved, and Actions () of a different schema is executed. In addition, after the Actions () of the different schema ends, the Actions () of the interrupted schema can be executed again.

  Further, the Actions () of the schema being executed is interrupted, and the SleepActions () is executed before the execution right is transferred to a different schema. For example, when the robot 1 finds a soccer ball during the conversation, it can say “Please wait a moment” and play soccer.

(4) Reentrant:
Each schema constituting the situation-dependent action hierarchy 108 is a kind of subroutine. When a schema is called from a plurality of parents, it is necessary to have a storage space corresponding to each parent in order to store the internal state.

  This is similar to the Reentrant property of the OS in the computer world, and is referred to as schema Reentrant property in this specification. As described with reference to FIG. 30, the schema is composed of class objects, and the Reentrant property is realized by generating an entity, that is, an instance of the class object for each target (Pronome).

  The Reentrant property of the schema will be described more specifically with reference to FIG.

  The SchemaHandler is a class object for managing the schema, and stores configuration information of the schema constituting the SBL as a file. At system startup, the SchemaHandler reads this configuration information file and builds a schema configuration in the SBL. In the example shown in FIG. 31, it is assumed that an entity of a schema that defines an action such as Eat or Dialog is mapped on the memory space.

  Here, it is assumed that the target (Pronom) A is set for the schema Dialog by the evaluation of the activity level based on the external stimulus and the change in the internal state, and the Dialog starts to execute the dialogue with the person A. .

  After that, the person B interrupts the conversation between the robot 1 and the person A and evaluates the activity level based on the external stimulus and the change in the internal state. As a result, the schema for the conversation with B has a higher priority. Suppose that

  In such a case, SchemaHandler maps another Dialog entity (instance) that inherits the class for performing the interaction with B on the memory space. Since another Dialog entity is used and the dialogue with B is performed independently of the previous Dialog entity, the content of the dialogue with A is not destroyed. Therefore, Dialog A can maintain data consistency, and when the dialogue with B is finished, the dialogue with A can be resumed from the point at which it was interrupted.

  The schema in the Ready list is evaluated, that is, the AL value is calculated according to the object (external stimulus), and the execution right is handed over. Thereafter, an instance of the schema that has been moved into the Ready list is generated, and evaluation is performed on other objects. Thereby, the same schema can be set in the active or sleep state.

E. In the robot behavior control system 100 according to the present embodiment, the situation-dependent behavior hierarchy 108 determines the behavior according to the internal state and the external environment.

  The internal state of the robot apparatus 1 is composed of several types of emotions such as instinct and emotion, and is handled as a mathematical model. An internal state management unit (ISM: Internal Status Manager) 104 manages an internal state based on an external stimulus (ES: External Stimula) recognized by each of the recognition function units 101 to 103 described above and a time course.

E-1. Emotional Hierarchization In this embodiment, emotions are composed of a plurality of hierarchies depending on their significance and operate in each hierarchy. From a plurality of determined operations, which operation is to be performed is determined according to the external environment and internal state at that time (described later). In addition, actions are selected at each level, but by expressing actions preferentially from lower-order actions, higher-order actions such as instinct actions such as reflexes and action selection using memory Behavior can be expressed consistently on one individual.

  FIG. 36 schematically shows a hierarchical configuration of the internal state management unit 104 according to the present embodiment.

  As shown in the figure, the internal state management unit 104 changes the internal information such as emotions according to primary emotions necessary for individual survival such as instinct and desire, and satisfaction (over or shortage) of the primary emotions 2 Broadly divided into the following emotions. Further, primary emotions are hierarchically subdivided from those that are more physiological to those that are associated with the survival of the individual.

  In the example shown in the figure, the primary emotion is divided into a lower primary emotion, an upper primary emotion, and an associated primary emotion from the lower order to the higher order. The lower primary emotion corresponds to access to the limbic system, and the emotion is generated so that homeostasis (individual maintenance) is maintained, and is given priority when homeostasis is threatened. The upper primary emotions correspond to access to the cerebral neocortex and are related to race maintenance such as intrinsic desires and social needs. The degree of satisfaction of the upper primary emotion varies depending on learning and environment (satisfied by learning and communication).

  Each level of the primary emotion outputs a change amount ΔI of a temporary emotion (instinct) level by executing the action-selected schema.

  The secondary emotion corresponds to so-called emotion, and includes elements such as joy, sadness, anger, surprise, anxiety, and fear. A change amount (satisfaction) ΔE of the secondary emotion is determined according to the change amount ΔI of the primary emotion.

  In the situation-dependent action hierarchy 108, action selection is performed mainly based on the primary emotion, but when the secondary emotion is strong, action selection based on the secondary emotion can also be performed. Furthermore, it is also possible to modulate the action selected based on the primary emotion using the parameter generated by the secondary emotion.

  The emotional hierarchy for the survival of an individual is first selected for behavior by innate reflexes. Next, an action that satisfies the lower primary emotion is selected. Then, it is realized by the action generation satisfying the upper primary emotion, the action generation satisfying the primary emotion by the association, and the more primitive individual retention.

  At this time, the primary emotion of each layer can apply pressure to the nearest layer. When the index for selecting the action determined by itself is strong, it is possible to suppress the action determined in the latest hierarchy and express its own action.

  As described in the previous section D, the situation-dependent action hierarchy 108 is composed of a plurality of schemas having target actions (see FIGS. 18 and 19). In the situation-dependent action hierarchy 108, a schema, that is, an action is selected using an activity level of each schema as an index. The activity level of the entire schema is determined by the activity level of the internal state and the activity level of the external situation. The schema maintains an activity level for each halfway progress to execute a target action. The occurrence of an action satisfying OO corresponds to executing a schema in which an action satisfying OO is the final goal.

The activity level of the internal state is determined by the sum of the change ΔE of the satisfaction level of the secondary emotion based on the change amount ΔI for each hierarchy in the primary emotion when the schema is executed. Here, when the primary emotion is composed of three layers L1, L2, and L3, and changes in the secondary emotion derived from each layer of the primary emotion at the time of schema selection are ΔE _L1 , ΔE _L2 , and ΔE _L3 , respectively. Is multiplied by weight factors w ₁ , w ₂ , and w ₃ to calculate the activity level. By increasing the weighting factor for the lower primary emotion, it becomes easier to select an action that satisfies the lower primary emotion. Further, by adjusting these weight factors, it is possible to obtain an effect that the primary emotion of each layer applies pressure to the nearest layer (concentration).

  Here, an example of action selection using a hierarchical structure of emotion will be described. However, in the following, Sleep (drowsiness) is treated as the lower primary emotion, and Curiosity (curiosity) is treated as the upper primary emotion.

(1) Assume that the level of activity of schemas that satisfy Sleep has increased as Sleep, which is a lower-level primary emotion, has become insufficient. At this time, if the activity level of the other schema does not rise, the schema satisfying Sleep executes itself until Sleep is satisfied.

(2) Suppose that the upper primary emotion, Curiosity, is insufficient before Sleep is satisfied. However, since Sleep is directly linked to individual maintenance, a schema that satisfies Sleep continues to be executed until the activity level of Sleep is below a certain value. When Sleep is satisfied to some extent, a schema that satisfies the Curiosity can be executed.

(3) It is assumed that the hand is moved close to the face of the robot while executing the schema satisfying the Curiosity. In response to this, the robot knows that the skin color has suddenly approached due to color recognition and size recognition, and as an innate reflex action it avoids the face from the hand, that is, pulls the head behind it reflexively. . This reflexive movement corresponds to the animal's spinal reflex. Since reflection is the lowest schema, the reflection schema is executed first.

  After the spinal cord reflex, the accompanying emotional change occurs, and it is determined from the change width and the activity level of other schemas whether or not the emotional expression schema is subsequently performed. If the emotion expression schema has not been performed, the schema that satisfies the Curiosity is continued.

(4) A schema below a certain schema itself is usually more likely to be selected than itself, but only when the activity level of the schema itself is extremely high, the lower schema is suppressed (Concentration) and a constant value. It is possible to execute itself. When the sleep shortage is significant, even if it is time to take action of the reflex behavior schema, the schema that satisfies the sleep is preferentially executed until it is restored to a constant value.

E-2. Cooperation with Other Functional Modules FIG. 37 schematically shows communication paths between the internal state management unit 104 and other functional modules.

  The short-term storage unit 105 outputs recognition results from the recognition function units 101 to 103 that recognize changes in the external environment to the internal state management unit 104 and the situation-dependent action hierarchy 108.

  The internal state management unit 104 notifies the internal state to the situation-dependent action hierarchy 108. On the other hand, the situation-dependent action hierarchy 108 returns information on the instinct or emotion that is associated or determined.

  In addition, the situation-dependent action hierarchy 108 selects an action based on the activity level calculated from the internal state and the external environment, and executes and completes the selected action to the internal state management unit 104 via the short-term storage unit 105. Notice.

  The internal state management unit 104 outputs the internal state to the long-term storage unit 106 for each action. In response to this, the long-term storage unit 106 returns stored information.

  The biorhythm management unit supplies biorhythm information to the internal state management unit 104.

E-3. Changes in internal state over time The internal state index changes over time. For example, primary emotions, that is, instincts such as Hunger (fatigue), Fatigue (fatigue), and Sleep (sleepiness) change as follows according to the passage of time.

Hunger: Reduced stomach (virtual value or battery level)
Fatigue: Sleepiness: Sleepiness

  Further, in this embodiment, Pleasantness (satisfaction), Activation (activity), and Certality (definition) are defined as elements of secondary emotions of the robot, that is, emotions. To change.

Pleasantness: Neutral changes: Activation: Biorhythms and Sleep (sleepiness) depend on Certification: Attentions

  FIG. 38 shows a mechanism for the internal state management unit 104 to change the internal state with time.

  As shown in the figure, the biorhythm management section notifies the biorhythm information at a constant cycle. On the other hand, the internal state management unit 104 changes the value of each element of the primary emotion by biorhythm and changes the activation (activity) that is the secondary emotion. The situation-dependent behavior hierarchy 108 receives the index value of the internal state such as instinct and emotion from the internal state management unit 104 every time there is a notification from the biorhythm management unit, so the activity level of each schema based on the internal state By calculating, an action (schema) depending on the situation can be selected.

E-4. Change in internal state due to operation execution The internal state also changes when the robot executes an operation.

  For example, in a schema that performs an action of “sleeping”, an action that satisfies Sleep (sleepiness) as a lower primary emotion is a final goal. In the situation-dependent action hierarchy 108, the activity level of each schema is calculated and compared based on the sleep as the primary emotion and the activation as the secondary emotion, and the “sleep” schema is selected. Is realized.

  On the other hand, the situation-dependent behavior hierarchy 108 transmits the completion of execution of the action of sleeping to the internal state management unit 104 via the short-term storage unit 105. On the other hand, the internal state management unit 104 changes the index value of Sleep, which is the primary emotion, by executing the “sleep” action.

  Then, in the situation-dependent action hierarchy 108, the activity level of each schema is calculated and compared anew based on the degree to which Sleep is satisfied and the activation as the secondary emotion. As a result, another schema having a higher priority is selected and the system leaves the schema of sleeping.

  FIG. 39 shows a mechanism for the internal state management unit 104 to change the internal state by executing the robot operation.

  The situation-dependent action hierarchy 108 notifies the internal state management unit 104 of the execution start and execution end of the action selected in the context-dependent manner and the Attention information via the short-term storage unit 105.

  When the execution completion information of the selected action is notified, the internal state management unit 104 confirms the external environment obtained from the short-term storage unit 105 according to the Attention information, and instinct (Sleep) as the primary emotion And the emotion as the secondary emotion are also changed accordingly. Then, the update data of the internal state is output to the situation dependent action hierarchy 108 and the long-term storage unit 106.

  In the situation-dependent action hierarchy 108, the activity level of each schema is calculated based on the newly received index value of the internal state, and the next action (schema) depending on the situation is selected.

  The long-term storage unit 106 updates the storage information based on the internal state update data, and notifies the internal state management unit 104 of the updated content. The internal state management unit 104 determines the certainty factor (Certainty) as the secondary emotion based on the certainty factor for the external environment and the certainty factor of the long-term storage unit 106.

E-5. Change of internal state based on sensor information The degree of movement of the robot when the robot executes the movement is recognized by each of the recognition function units 101 to 103 and notified to the internal state management unit 104 via the short-term storage unit 105. The internal state management unit 104 can reflect this degree of operation as, for example, fatigue (fatigue) in the change of the primary emotion. Further, the secondary emotion can be changed in response to the change of the primary emotion.

  FIG. 40 shows a mechanism for the internal state management unit 104 to change the internal state based on the recognition result of the external environment.

  When the internal state management unit 104 receives the recognition results by the recognition function units 101 to 103 via the short-term storage unit 105, the internal state management unit 104 changes the index value of the primary emotion and changes the emotion as the secondary emotion accordingly. To do. Then, the update data of the internal state is output to the situation dependent action hierarchy 108.

  In the situation dependent action hierarchy 108, the activity level of each schema can be calculated based on the newly received index value of the internal state, and the next action (schema) depending on the situation can be selected.

E-6. Changes in Internal State due to Association As described above, the robot according to this embodiment has an associative memory function in the long-term storage unit 106. This associative memory is a mechanism in which an input pattern consisting of a plurality of symbols is stored in advance as a memory pattern, and a pattern similar to one pattern is recalled. Associative memory.

  For example, consider the case where an emotional change of “happy” occurs when an apple is seen.

  When the apple is recognized by the visual recognition function unit 101, the situation-dependent behavior hierarchy 108 is notified as a change in the external environment via the short-term storage unit 105.

  The long-term memory unit 106 recalls an action of “eating (apple)” and an internal state change in which the primary emotion (hunger) is satisfied by an index value of 30 by eating with an associative memory related to “apple”. can do.

  When receiving the storage information from the long-term storage unit 106, the situation-dependent behavior hierarchy 108 notifies the internal state management unit 104 of the change ΔI = 30 in the internal state.

  The internal state management unit 104 can calculate the change amount ΔE of the secondary emotion based on the notified ΔI, and obtain an index value of the secondary emotion E by eating an apple.

  FIG. 41 shows a mechanism for the internal state management unit 104 to change the internal state by associative memory.

  The external environment is notified to the situation dependent action hierarchy 108 via the short-term storage unit 105. With the associative memory function of the long-term storage unit 106, it is possible to recall an action according to the external environment and a change ΔI of the primary emotion.

  The situation-dependent action hierarchy 108 selects an action based on the storage information obtained by the associative memory, and notifies the internal state management unit 104 of the change ΔI of the primary emotion.

  The internal state management unit 104 calculates the secondary emotion change ΔE based on the primary emotion change ΔI received and the primary emotion index value managed by the internal state management unit 104. To change. Then, the newly generated primary emotion and secondary emotion are output to the situation dependent behavior hierarchy 108 as internal state update data.

  In the situation dependent action hierarchy 108, the activity level of each schema can be calculated based on the newly received index value of the internal state, and the next action (schema) depending on the situation can be selected.

E-7. Changes in Internal State due to Intrinsic Behavior As described above, the robot according to the present embodiment changes the internal state by executing an action (see FIG. 39). In this case, the action is selected based on the index value of the internal state composed of the primary emotion and the secondary emotion, and the emotion is satisfied when the execution of the action is completed. On the other hand, the robot according to the present embodiment also defines an innate reflex behavior that does not depend on emotion. In this case, the reflex behavior is directly selected according to the change in the external environment, and the mechanism is different from the internal change caused by normal operation execution.

  For example, consider the case of taking an innate reflex behavior when a large thing suddenly appears.

  In such a case, for example, the recognition result (sensor information) of “large” by the visual recognition function unit 101 is directly input to the situation-dependent action hierarchy 108 without passing through the short-term storage unit 105.

  In the situation-dependent action hierarchy 108, an activity level of each schema is calculated by an external stimulus “large”, and an appropriate action is selected (see FIGS. 15, 25, and 26). In this case, in the situation-dependent action hierarchy 108, the spinal reflex action “OK” is selected, the secondary emotion “surprise” is determined, and this is notified to the internal state management unit 104.

  The internal state management unit 104 outputs the secondary emotion sent from the situation-dependent action hierarchy 108 as its own emotion.

  FIG. 42 shows a mechanism for the internal state management unit 104 to change the internal state by innate reflection behavior.

  When performing an innate reflex action, the fighting information by each recognition function part 101-103 is directly input into the situation dependence action hierarchy 108 not via the short-term memory | storage part 105. FIG.

  In the situation-dependent action hierarchy 108, the activity level of each schema is calculated by an external stimulus obtained as sensor information, an appropriate action is selected, a secondary emotion is determined, and this is sent to the internal state management unit 104. Notice.

  The internal state management unit 104 outputs the secondary emotion sent from the situation-dependent action hierarchy 108 as its own emotion. Further, the final activation is determined based on the level of biorhythm with respect to the activation from the situation-dependent behavior hierarchy 108.

  In the situation dependent action hierarchy 108, the activity level of each schema can be calculated based on the newly received index value of the internal state, and the next action (schema) depending on the situation can be selected.

E-8. Relationship between schema and internal state management unit The situation-dependent behavior hierarchy 108 is composed of a plurality of schemas. For each schema, an activity level is calculated based on an external stimulus or a change in an internal state, and the activity level is determined according to the level of the activity level. The schema is selected and the action is executed (see FIGS. 18, 19, and 25).

  FIG. 43 schematically shows the relationship between the schema and the internal state management unit. The schema can communicate with external objects such as the short-term storage unit 105, the long-term storage unit 106, and the internal state management unit 104 via a proxy such as DSubject or DOObject (see FIG. 30).

  The schema includes a class object that calculates an activity level according to an external stimulus or a change in an internal state. An RM (Resource Management) object communicates with the short-term storage unit 105 via a proxy, acquires an external environment, and calculates an activity level based on the external environment. In addition, the Motivation calculation class object communicates with the long-term storage unit 106 and the internal state management unit 104 via the proxy, acquires the amount of change in the internal state, and calculates the activity level based on the internal state, that is, the Motivation. . The calculation method of Motivation will be described in detail later.

  As described above, the internal state management unit 104 is layered in stages into primary emotions and secondary emotions. The primary emotions are dimensionally hierarchized into primary emotion layers based on innate reactions, primary emotions based on homeostasis, and primary emotions based on association (see FIG. 36). The emotion as the secondary emotion is mapped to three elements of P (Pleasantness), A (Activity), and C (Concentration).

  All changes ΔI in each layer of the primary emotion are input to the secondary emotion and used to calculate the Pleasantness change ΔP.

  Activity is comprehensively determined from information such as sensor input, operation time, and biorhythm.

  Further, the certainty factor of the selected schema is used as the certainty factor in the actual secondary emotion hierarchy.

  FIG. 44 schematically shows the Motivation calculation path by the Motivation calculation class object.

  The RM class object accesses the short-term storage unit 105 via a proxy, acquires sensor information, and determines the activity level by an external stimulus based on the strength of the stimulus such as the distance and size of the recognized object. evaluate.

  On the other hand, the Motivation calculation class / object accesses the short-term storage unit 105 via a proxy, acquires characteristics relating to the object, and further inquires about the characteristics of the object in the long-term storage unit 106 via the proxy to change the internal state. To get. Then, the internal state management unit 104 is accessed via a proxy to calculate an internal evaluation value inside the robot. Therefore, the calculation of Motivation is independent of the strength of the external stimulus.

  As described above, the behavior control system of the robot according to the present embodiment uses the associative memory to recall the change of the internal state from the external stimulus, thereby calculating the secondary emotion and selecting the behavior (FIG. 41). checking). Furthermore, by using associative memory, it is possible to recall a change in the internal state that is different for each object. Thereby, even in the same situation, it is possible to vary the ease of expression of the action. That is, in addition to external stimuli, physical conditions, and the current internal state, an action can be selected in consideration of memory for each object of the robot, and more various and diversified responses can be realized.

  For example, instead of taking action determined by the external environment or internal state, such as “I do XX because I can see XX” or “I do XX because I do n’t have enough XX” , Even if you can see ○○, □□ because it is △△, or “you can see it because it is XX, but ■■■”, you can use it to change the internal state of the object. You can turn on.

  FIG. 45 schematically shows the mechanism of the Motivation calculation process when an object is present.

  First, the short-term storage unit 105 is accessed via a proxy, and the target features recognized by the recognition function units 101 to 103 are inquired.

  Next, using the extracted feature, this time, the long-term storage unit 106 is accessed via the proxy, and how the feature object changes the desire related to the schema, that is, the primary emotion change ΔI is changed. To win.

  Next, the internal state management unit 104 is accessed via the proxy to extract how the pleasantness / discomfort value changes due to a change in desire, that is, a secondary emotion change ΔPleaant.

Then, the i-th motivation is calculated by the following motivation calculation function g _target-i that uses the secondary emotion change ΔPleasant and the certainty of the object as arguments.

  FIG. 46 schematically shows the mechanism of the Motivation calculation process when there is no object.

  In this case, first, a change ΔI in desire due to the action is asked with respect to the memory for the action.

Next, using the acquired ΔI, the internal state management unit 104 derives a change ΔPleasant of the secondary emotion when the primary emotion changes by ΔI. In this case, the i-th _{motivation is} calculated by the following _motivation calculation function g _nottarget-i using the secondary emotion change ΔPleasant as an argument.

E-9 . Method for Changing Each Element of Secondary Emotion FIG. 47 illustrates a mechanism for changing the Pleasantness of the secondary emotion.

  The long-term storage unit 106 inputs changes in the primary emotion according to the amount of storage to the internal state management unit 104. In addition, the short-term storage unit 105 inputs changes in the primary emotion caused by sensor inputs from the recognition function units 101 to 103 to the internal state management unit 104.

  In addition, the schema inputs the primary emotion change (Nourishment, Moisture, Sleep) due to the schema execution and the primary emotion change (Affection) depending on the schema contents to the internal state management unit 104.

  Pleasantness is determined according to changes in excess or deficiency of the primary emotion.

  FIG. 48 illustrates a mechanism for changing the activity of the secondary emotion.

  Activity is determined in an integrated manner based on the sum of times other than Sleep in the schema, biorhythm, and sensor input.

  Further, FIG. 49 illustrates a mechanism for changing the Certificate of the secondary emotion.

  When the long-term storage unit 106 is inquired about an object, Certificate is returned. Which primary emotion is focused depends on the target behavior of the schema. Then, the extracted Certificate becomes the Certificate in the secondary emotion of the internal state management unit 104 as it is.

  FIG. 50 schematically shows a mechanism for obtaining the certificate.

  The long-term storage unit 106 stores the probability of each item such as the recognition result and emotion related to the object for each schema.

  The schema asks the long-term storage unit 106 for a certainty value for the storage related to the schema. In contrast, the long-term storage unit 106 gives the certainty of the storage related to the schema as the certainty of the object.

F. Command management in ResourceManager A command for operating the robot apparatus 1 is transmitted to the resource management (ResourceManager) module of the robot apparatus 1 from the Situated Behaviorslayer and the Reflexive Situated Behaviors Layer. (See FIGS. 13 and 18)

  A command issued from Reflexive Situated BehaviorsLayer is a command generated as a reflex action. A command issued from Situated Behaviorslayer is a command generated as a situation-dependent action. Since these are generated as separate processes or as separate threads, it is difficult to mediate in advance, and it is difficult to control by the intention level, which is a mechanism for preventing them from being expressed in advance (described later).

  For this reason, ResourceManager arbitrates these two commands at the activity level associated with the command. FIG. 51 shows the configuration of the ResourceManager. The illustrated ResourceManager is composed of a command contention resolver that performs contention resolution of commands transmitted from a Situated Behaviorslayer or Refractive Situated BehaviorsLayer, and a content manager that manages commands for each hardware resource.

  The command conflict resolution unit holds a command, information on hardware resources of the robot apparatus 1 used by the command, and an activity level of the command. When a new command is transmitted from Situated Behaviors or Reflexive Situation Behaviors Layer, whether the currently executed command conflicts with the hardware resource information of the robot device 1 being used and the hardware resource information of the new command. judge. If there is a conflict, the activity levels associated with the command are compared to resolve the command conflict.

  That is, when the activity level of the command currently being executed is lower than the activity level of the new command, the command being executed is canceled and the new command is executed.

  On the other hand, if the activity level of the currently executing command is higher than the activity level of the new command, the new command is executed waiting for the end of the executing command, or the new command is canceled. .

  If the activity level of a new command is higher than the activity level of a command waiting in ResourceManager, the command waiting in ResourceManager is cancelled.

G. Control of subordinate actions using intention level in SituatedBehaviorlayer and DeliverableLayer
G-1. The intention level control SituatedBehaviorlayer from SituatedBehaviorlayer to ReflexiveSituatedBehaviorsLayer may not allow reflex behavior to occur during the behavior. For example, it is assumed that the interactive action existing in the Situated Behavior layer causes the robot apparatus 1 to track the face.

  At this time, if the human motion is quicker than the operation speed of the robot apparatus 1, the face may be suddenly observed larger. At this time, it is considered that a large thing suddenly appeared, and as a natural behavior, the behavior of Reflexive Behavior is manifested, and the robot apparatus 1 is dragged (surprised). As a result, the face tracking command is overwritten, and the robot tracking stops. This is an inappropriate behavior during a dialogue.

  In such a case, the dialogue behavior (schema) of the Situated Behavior is transmitted to the Reflexive Behavior as a command with the schema and the intention level information for reflexive Behaviors Layer that reflects the intention level in the Action function.

  FIG. 52 schematically shows control from the Situated Behavior Layer to the Reflexive Behavior Layer. The behavior intention manager of the behavioral state control unit of the Reflexive Behavior layer manages the intention level information command and transmits the command to a bias calculator of a schema (behavior to lift the neck) that reflects the intention level (for the bias calculator, see FIG. 21). checking). The bias calculator reflects the intention level of Reflexive Behavior. The schema is reflected in the activity level of the Monitor function as shown in FIG.

G-2. Intent level control DeliverableLayer from DeliverableLayer to SituatedBehaviorlayer is implemented as a schema in the present invention. Use long-term and short-term memory to plan an action. The planned action at this time is configured as a sequence of the Situated Behavior layer schema. Therefore, it controls by the same mechanism as the intention level of G-1.

  FIG. 53 schematically shows control from the Deliverable Layer to the Situated Behavior Layer. In other words, the DeliverativeLayerLayer transmits the planned sequence schema as the target schema and the intention level information as a command to the SituatedBehaviorLayer.

  The behavior intention management unit of the behavioral state control unit of the Situated Behavior Layer manages the intention level information command and transmits it to a bias calculator having a schema that reflects the intention level. (Refer to FIG. 21 for the bias calculator.) The schema reflecting the intention level of the Situated Behavior Layer is reflected in the activity level in the Monitor function as shown in FIG. 21 and excited, so it is not a situation dependent but a Deliverable Layer. Expressed with support from When the expression ends, the behavior intention management unit of the behavior state control unit transmits the termination information to the Deliverable Layer and returns the intention level information as a command to the schema that has been transmitted.

H. In the section D of the other configuration example of the behavior control system, the case has been described on the assumption that all the element behaviors constituting the situation-dependent behavior hierarchy are behaviors for keeping the internal state within a certain range, that is, “homeostasis behavior”. . In such a case, when all the internal states are sufficiently satisfied, the desire value of each element action becomes small, so the action value is also small, and the opportunity for the situation-dependent action to occur is reduced. In addition, since there will be no reflex behavior without external stimuli, the robotic device will do nothing. However, robots that do nothing autonomously are considered problematic in terms of entertainment. Therefore, the present inventors have decided to further incorporate “idle behavior” having no homeostasis purpose as a component of the situation-dependent behavior that expresses spontaneous behavior of the robot apparatus. In this section, the mechanism of the situation-dependent behavior hierarchy composed of homeostasis behavior and idle behavior as elemental behavior, the behavior control system of the robot device in this case, and the arbitration method between the self-issued motion and the reflex behavior will be explained in detail.

H-1. Behavior Control System FIG. 54 shows another configuration example of the behavior control system of the robot apparatus 1. However, the same components as those in FIG. 3 are denoted by the same reference numerals.

  The system 100 ′ acquires external information, that is, external stimulus through the visual recognition function unit 101, the auditory recognition function unit 102, and the contact recognition function unit 103. The recognition results by these recognition function units are transmitted to the situation-dependent behavior hierarchy 108, the reflex behavior unit 109, and the reflex event management unit 110 through the short-term storage unit 105 and the long-term storage unit 106.

  Each of the situation-dependent action hierarchy 108 and the reflex action part 109 includes a plurality of action modules, that is, element actions described as objects called “schema”. Each of the situation-dependent action hierarchy 108 and the reflex action unit 109 determines an action based on a recognition result, that is, an external stimulus, and outputs a motion command to which the strength of the motion command is added to the resource management unit 120. The motion command corresponds to the action value, that is, the AL value described above, and is expressed as a scalar value.

  Further, the reflection event management unit 110 calculates the reflection event density from the event input to the reflection action unit 109. The reflection event density is obtained by quantifying the degree of input of an event that causes reflection behavior, that is, an external stimulus. When the robot apparatus is not homeostasis, the occurrence probability of idle behavior is controlled based on the reflection event density (described later).

  The resource management unit 120 compares the strengths of the motion commands of the situation-dependent action and the reflex action transmitted from the situation-dependent action hierarchy 108 and the reflex action part 109, respectively, and outputs the motion with the larger value to the external environment. Then, the robot device 1 is caused to perform as an actual operation.

  Note that the reflex behavior unit 109 executes the control cycle at a sufficient speed compared to the situation-dependent behavior hierarchy 108.

H-2. Situation-dependent action hierarchy As already described, the situation-dependent action hierarchy 108 is composed of a plurality of element actions. For each elemental action, an action value is periodically calculated from an internal state, a recognition result stored in the short-term storage unit 105 or the long-term storage unit 106, and the action is output when the action is selected. In the configuration example described in detail in this section, the situation-dependent action hierarchy 108 uses, as an elemental action, an “idle action” having no homeostasis purpose in addition to an action for keeping the internal state within a certain range, that is, “homeostasis action”. I have.

  FIG. 55 illustrates an internal configuration example of the situation-dependent action hierarchy 108. In the situation-dependent action hierarchy 108 shown in the figure, each element action action is based on the emotion data input from the internal state management unit 104 and the external stimulus recognition result input from the short-term storage unit 105 or the long-term storage unit 106. The behavior value calculation unit 220 calculates the AL value as the behavior value. When the behavior value calculation unit 220 calculates the behavior value using the behavior value calculation database, the learning unit 240 further learns based on the result after the behavior is expressed and updates the behavior value calculation database in the elemental behavior. You may have. The learning method of the behavior value calculation database is described in, for example, Japanese Patent Application No. 2004-68133 already assigned to the present applicant.

  Then, the action selection unit 230 selects an element action that should be expressed based on the AL value of each element action, adds the strength of the command to the motion command, and outputs the motion command to the resource management unit 120. Then, the resource management unit 120 outputs a motion command of either a spontaneous behavior or a reflex behavior according to the strength of the motion command. The selected elemental action outputs its own action.

  In the example shown in FIG. 55, the situation-dependent action hierarchy 108 includes homeostasis action A, homeostasis action B,. Each element behavior is described as an object called a schema. In the illustrated example, the element selection is performed by the action selection unit 230 in a unified manner. However, as shown in FIG. 19, the element action may have a hierarchical structure.

  The elemental behavior of the homeostasis behavior is voluntarily output as an action for satisfying the internal state (for example, taking a charging action because the battery is low, resting because it is tired). For these elemental actions, the desire value is calculated only from the internal state. At the same time, an expected satisfaction value is calculated from the internal state and the recognition result, and an action value of each elemental action is calculated from the desire value and the expected satisfaction value. The behavior value is calculated for each elemental behavior. When the desire value increases, the behavioral value of the homeostasis behavior increases, so the behavior selection unit 230 selects the homeostasis behavior and outputs the motion command to the resource management unit 120. At that time, the action value calculated as the strength of the motion command is added.

  Further, when all the internal states are sufficiently satisfied, the desire value of each elemental action is small, so the action value is also small. In such a case, an idle action having no homeostasis purpose outputs a motion command (for example, nothing, tilting the head, practicing a golf putt, relaxing lying down, etc.). The action value of the element action of the idol action is a constant value. Therefore, when all the internal states are sufficiently satisfied, when the behavioral value is not homeostasis, that is, the behavioral value of the homeostasis behavior decreases and falls below the behavioral value of the idle behavior, the behavior selection unit 230 performs idle behavior. Will come to choose.

  In the present embodiment, in order to avoid or alleviate the competition between the idle action and the reflex action, the idle action is expressed with a probability of occurrence according to the reflex event density. The reflex event density is a numerical value representing the degree of input of an event that causes reflex behavior, that is, an external stimulus. The motion command selection for idle action will be described in detail later.

H-3. Behavioral value of homeostasis behavior For the elemental behavior of homeostasis behavior, the desire value is calculated only from the internal state. At the same time, an expected satisfaction value is calculated from the internal state and the recognition result, and an action value of each elemental action is calculated from the desire value and the expected satisfaction value. The resource management unit 120 selects a motion using the strength of the motion command as an index, and the strength of the motion command of the homeostasis behavior uses the behavior value of the elemental behavior. In this section, the method of calculating the behavioral value of homeostasis behavior is explained in detail.

  As shown in FIG. 55, for each elemental behavior of homeostasis behavior, a predetermined internal state and external stimulus are defined according to the behavior described in itself. External stimuli are treated as properties of the corresponding object. For example, the elemental action A whose action output is “eat” handles the type of the object (OBJECT_ID), the size of the object (OBJECT_SIZE), the distance of the object (OBJECT_DISTANCE), and the like as the internal state. “NOURISHMENT” (“nutrition”), “FATIGUE” (“fatigue”), etc. are handled. In this way, the types of external stimuli and internal states to be handled are defined for each elemental action, and the action value for the action (elemental action) corresponding to the corresponding external stimulus and internal state is calculated. Of course, one internal state or external stimulus may be associated with not only one elemental action but also a plurality of elemental actions.

  Further, the internal state management unit 104 receives external stimuli and information such as the remaining battery level of the own battery and the rotation angle of the motor, and receives internal state values (internal state vectors) corresponding to a plurality of internal states as described above. IntV) is calculated and managed. Specifically, for example, the internal state “nutrient state” can be determined based on the remaining amount of the battery, and the internal state “fatigue” can be determined based on the power consumption.

  The action value AL of homeostasis action indicates how much the robot apparatus wants to perform the element action (execution priority). The behavior value calculation unit 220 refers to the behavior value calculation database 221 in which the input external stimulus is associated with the expected internal state change expected to change after the behavior is expressed, and the external stimulus and the internal state at a certain time The action value AL for each element action A to D at that time is calculated. In the example shown in FIG. 55, the behavior value calculation unit 220 is provided for each elemental action. However, a single action value calculation unit may be configured to calculate the action values for all elemental actions. Good.

  The behavioral value AL for the elemental behavior of each homeostasis behavior is the desire value for each behavior corresponding to each current internal state, the satisfaction based on each current internal state, and the change in the internal state that is expected to change due to an external stimulus. The amount is calculated based on the expected satisfaction change based on the expected internal state change indicating the amount of change in the internal state that is expected to change as a result of the external stimulus being input and the behavior being expressed.

  FIG. 56 shows a flow of processing in which the behavior value calculation unit 220 calculates the behavior value AL from the internal state and the external stimulus. In the present embodiment, an internal state vector IntV (Internal Variable) having a value of one or more internal states as a component is defined for each element action, and the internal state vector corresponding to each element action from the internal state management unit 104 IntV is obtained. That is, each component of the internal state vector IntV indicates, for example, one internal state value (internal state parameter) indicating the emotion described above, and the action value of the elemental action corresponding to each component of the internal state vector IntV Used for calculation. Specifically, for the elemental action A having the action output “eat”, for example, an internal state vector IntV {IntV_NOURISHEMENT “nutrition state”, IntV_FATIGUE “fatigue”} is defined.

  Further, for each internal state, an external stimulus vector ExStml (External Stimulus) having one or more external stimulus values as components is defined, and the external stimulus vector ExStml corresponding to each elemental action is stored from each storage unit 105 to 106. obtain. Each component of the external stimulus vector ExStml indicates recognition information such as the size of the object, the type of the object, and the distance to the object, and is used to calculate the internal state value corresponding to each component of the external stimulus vector ExStml. used. Specifically, the internal state IntV_NOURISHMENT “nutrition state” defines, for example, an external stimulus vector ExStml {OBJECT_ID “object type”, OBJECT_SIZE “object size”}, and the internal state IntV_FATIGUE “fatigue” For example, an external stimulus vector ExStml {OBJECT_DISTANCE “distance to an object”} is defined.

  The behavior value calculation unit 220 receives the internal state vector IntV and the external stimulus vector ExStml as input, and calculates the behavior value AL. Specifically, the behavior value calculation unit 220 includes, from the internal state vector IntV, a first calculation unit MV that obtains a motivation vector (Motivation Vector) indicating how much the corresponding element behavior is desired, and the internal state vector IntV. And a second calculating unit RV that obtains a releasing vector indicating whether or not the corresponding element behavior can be performed from the external stimulus vector ExStml, and calculates an action value AL from these two vectors.

  For details of the method for calculating the behavioral value AL related to homeostasis behavior, see, for example, Japanese Patent Application No. 2004-68133 already assigned to the present applicant.

H-4. Motion command selection and strength of idol action Since idol action has no homeostasis purpose, that is, it has no relation to the internal state, the action value of the element action is given a constant value. In order to avoid or alleviate competition with reflex behavior, the strength of the motion command is determined based on the reflex event density so that the idle behavior is selected according to the probability of occurrence according to the reflex event density. Like to do. In this section, the calculation method of the behavior value of idle behavior will be explained in detail.

  The reflection event management unit 110 calculates the reflection event density from the event input to the reflection action unit 109. The element motion of the idle action determines the type and strength of the motion command to be output depending on the value.

In the present embodiment, as shown in the following two formulas, every time a reflection event is input, the reflection event density RED (Reflexive Event Density) is increased and the time is attenuated. However, IR (Increase Ratio) is an increase rate of the reflection event density RED per reflection event (where 0 <IR <1), and RED _max = 1.0. DT is the elapsed time, and τ is the half-life of time decay.

  The increase rate IR may be set for each type of reflection event. For example, it is possible to set a large value for an explicit reflection event (such as pressing a shoulder touch sensor) by the user, and a small value for a reflection event (generation of a crap sound) that is easily recognized by noise or the like. FIG. 57 illustrates a state in which the reflection event density RED changes according to the occurrence of the reflection event.

  Idle behavior preferably does not interfere with reflex behavior that needs to react directly to external stimuli. For this reason, in this embodiment, the motion command for idle action is determined according to the reflection event density.

  For example, when there is a large number of reflection event inputs from the user and the reflection event density is high, a high probability of a motion with a smaller amount of motion such as “do nothing” or “tilt” so that it can react more reactively To output. On the other hand, when the reflection event density is low because the user is left unattended and the reflection event density is low, the amount of movement such as “practice golf putting” or “lie down and relax” is avoided as much as possible. A large motion is output with high probability.

  Which motion (element behavior) is selected from the reflection event density is selected probabilistically by the following mechanism.

Reflection event density in motion m _i is best selected to REDm _i. The distance between the current reflection event density _{RED D mi = | REDm i -RED} | when the probability Pm _i the i-th motion m _i is selected is calculated as follows.

T is the Boltzmann temperature, and the action selection becomes more random when a large value is set. On the other hand, when the value is set to a small value, there is no stochastic element in the selection, and the motion is uniquely selected by the reflection event density RED (the motion of REDm _i having the closest RED). However, this selection is not performed during motion execution.

Figure 58 shows an example of probability Pm _i where m _i is selected using the reflection event density RED. In the example shown in the figure, four types of motion are defined as idle actions: “do nothing”, “tilt”, “practice a golf pad”, and “lie down and relax”. Frequently selected reflection event densities RED _{m0 to} RED _m3 are set. As mentioned above, a low reflection event density is set for a motion with a large amount of motion, and a higher reflection event is set for a motion with a small amount of motion in order to suppress the interference of the idle behavior with the reflection behavior while enhancing entertainment properties. Density is set. For example, a motion with a small amount of motion such as “do nothing” or “tilt neck” is large, and a motion with a large amount of motion such as “practice a golf pad” or “sit down and relax” is set small.

  Then, the occurrence probability of each motion obtained from the above equation is used as the strength of the motion command for each motion. Thereby, in the self-issued arbitration in the resource management unit 120, when the reflection event density is high, it becomes easy to select an idle action or a reflection action with a small operation amount, and when the reflection event density is low, an idle action with a large operation amount is selected. It is easy to be selected, or it becomes difficult to select reflex behavior. As a result, it is possible to arbitrate the timing for spontaneously outputting idle behavior and the type of motion based on the type and frequency of events that cause reflex behavior.

  In the situation-dependent behavior hierarchy 108 as described above, the behavior value for each elemental motion of homeostasis behavior is calculated, and the elemental motion of idle behavior is selected according to the type and frequency of the event that causes reflex behavior Is done. And the action selection part 230 compares the action value of each element exercise | movement of homeostasis action and idle action, and finally determines which should be selected as self-issued movement.

  While the behavioral value of homeostasis behavior is calculated from the desire value and the expected satisfaction value, a certain behavioral value is given to idle behavior. Accordingly, since the behavioral value of homeostasis behavior increases as the desire value increases, the behavior selection unit 230 selects the homeostasis behavior as the situation-dependent behavior. On the other hand, when all the internal states are sufficiently satisfied, when the behavioral value is not homeostasis, that is, when the behavioral value of the homeostasis behavior decreases and falls below the behavioral value of the idle behavior, the behavior selection unit 230 becomes idle. The action is selected as the situation dependent action.

H-5. Self-issued movement and reflex behavior mediation resource management unit 120 mediates self-issued movement and reflex behavior. Specifically, the strengths of the motion commands of the situation-dependent action and the reflex action transmitted from the situation-dependent action hierarchy 108 and the reflex action unit 109 are compared, and the motion with the larger value is output to the external environment, It is expressed as an action that the robot apparatus 1 actually performs.

  The reflex action is basically a reflex action that is triggered by the sensor input itself, but is composed of a plurality of element actions as in the situation-dependent action hierarchy. The elemental behavior of the reflex behavior unit 109 is directly input every time an event occurs from the visual recognition function unit 101, the auditory recognition function unit 102, and the contact recognition function unit 103, so that an immediate reaction (reflection) is possible. Each elemental action is a specific motion command (see direction of sound, behind) for the input of interest (clapping occurred, something suddenly appeared in front of you, shoulder touch sensor pressed, etc.) (See the direction of the touch sensor pressed) is output to the resource management unit. At that time, a constant value is added to the strength of the motion command.

  Below, the arbitration method of the self-issued movement by the situation-dependent action hierarchy 108 and the reflection action by the reflection action unit 109 will be described in detail.

  As a specific example of homeostasis behavior, consider battery charge behavior. The battery charging action focuses on the remaining battery level as an internal state, and the action value increases as the remaining battery level decreases. The strength of the command is a value of action value, and takes a value from 0 to 100.

  In addition, four types of idle actions are considered: “do nothing”, “tilt”, “practice a golf pad”, and “lie down and relax”. A constant value 20 is uniformly given as the action value of these element actions. As for {command strength, reflection event density at which the motion is best selected}, "do nothing" is {70, 80}, "tilt" is {75, 60}, "golf putt" "Practice of" is {85, 40}, and "Lie down and relax" is {90, 20}.

  In addition, as an example of elemental behavior related to reflex behavior, a clapp reflex behavior, a moving object reflex behavior, and a touch reflex behavior are considered. The clap reflex action receives a sound when a hand is slammed and outputs a motion command that points in the direction of the sound source. The moving object reflex action inputs a sudden approach of an object and outputs a motion command that sways back. The touch reflection action receives a touch sensor and outputs a command for viewing the direction of the pressed touch sensor. The action value of these reflection actions, that is, the strength of the motion command is set to a constant value 80.

  Examples of mediation between spontaneous behavior (homeostasis behavior, idle behavior) and reflex behavior will be described below.

Mediation of homeostasis and reflex behavior:
FIG. 59 shows an example of behavior mediation when the remaining battery level is medium and a crap is recognized from the outside world.

  At this time, in the situation-dependent action hierarchy 108, the action selection unit 230 compares the action value 70 of the battery charge action as the homeostasis action with the action value 20 of the idle action, and the battery charge action having a large action value is selected. The In such a case, the motion command of the battery charge action given the action value 70 as the strength of the command is output to the resource management unit 110.

  At this time, it is assumed that a crap is detected as an external stimulus. When the reflex action unit 109 selects the clap reflex action that responds immediately to the external stimulus, the reflex action unit 109 outputs a motion command to the resource management unit 110 with the command strength 80 given uniformly to all the reflex actions. To do.

  Then, the resource management unit 110 compares the strengths of the motion commands transmitted from the situation-dependent behavior layer 108 and the reflex behavior unit 109, and outputs the motion having the larger value. Therefore, in this case, a reflex action with a high command strength is selected.

  FIG. 60 shows an example of behavior arbitration when the remaining battery level is low and a crap is recognized.

  At this time, in the situation-dependent action hierarchy 108, the action selection unit 230 compares the action value 90 of the battery charge action as the homeostasis action with the action value 20 of the idle action, and the battery charge action having a large action value is selected. The In such a case, the motion command of the battery charge action given the action value 90 as the strength of the command is output to the resource management unit 110.

  At this time, it is assumed that a crap is detected as an external stimulus. When the reflex action unit 109 selects the clap reflex action that responds immediately to the external stimulus, the reflex action unit 109 outputs a motion command to the resource management unit 110 with a command strength 80 given to all reflex actions uniformly To do.

  Then, the resource management unit 110 compares the strengths of the motion commands transmitted from the situation-dependent behavior layer 108 and the reflex behavior unit 109, and as a result, a battery charge behavior having a high command strength is selected.

  That is, when the action value (desire) of the homeostasis action is low, the robot apparatus easily reacts to the trigger of the reflex action, but when the action value of the homeostasis action is high, the robot apparatus triggers the reflex action. Will not respond to it (it appears to be focused on that action).

Example of mediation between idol behavior and reflex behavior:
61 and 62 show examples of behavior mediation when the remaining battery level is sufficient and the crap is recognized from the outside world.

  In this case, since the internal state is sufficiently satisfied, the desire value of the battery charging action as the homeostasis action becomes small, that is, the action value decreases. Output as a command. The type of idle action to be output is determined by the reflection event density RED at that time (described above).

  FIG. 61 shows an example of behavior arbitration when the user calls the robot apparatus with a clapping hand and the reflection event density RED increases from a certain value to about 60.

  At this time, if the elemental action of the idle action output from the situation dependent action hierarchy 108 is “tilt”, the strength of the motion command is 75. On the other hand, since the strength of the command of the reflex action is as large as the constant value 80, the resource management unit 120 selects the reflex action having a large strength of the command.

  FIG. 62 shows an example of behavior mediation when the user calls the robot device with a clapping hand when the reflection event density decreases to about 40 as a result of the robot device being notified for a while. ing.

  At this time, if the elemental behavior of the idle behavior output from the situation dependent behavior hierarchy 108 is “golf put practice”, the motion command strength is 85. On the other hand, since the strength of the reflex action command is as small as 80, the resource management unit 120 selects an idle action with a high command strength.

  Thereafter, when the user continues to call the robot apparatus with a clapping hand, the reflection event density RED increases, and as a result, the resource management unit 120 outputs the reflection action as shown in FIG. Become.

  That is, in the idle behavior, when there are many reflection events from the user and the reflection event density is high, the robot apparatus responds to the trigger of the reflection behavior. On the other hand, when the reflection event density is low due to being left by the user, the robot apparatus does not respond to the trigger of the reflection action (it seems to concentrate on the action).

  The present invention has been described in detail above with reference to specific embodiments. However, it is obvious that those skilled in the art can make modifications and substitutions of the embodiment without departing from the gist of the present invention.

  The gist of the present invention is not necessarily limited to a product called a “robot”. That is, as long as it is a mechanical device that performs an exercise resembling human movement using an electrical or magnetic action, the present invention can be applied to products belonging to other industrial fields such as toys. Can be applied.

  Further, the behavior control mechanism according to the present invention can be suitably applied not only to the robot apparatus but also to other autonomous agents that perform self-issued movement and reflex behavior.

  In short, the present invention has been disclosed in the form of exemplification, and the description of the present specification should not be interpreted in a limited manner. In order to determine the gist of the present invention, the claims should be taken into consideration.

It is the figure which showed typically the function structure of the robot apparatus 1 provided to implementation by this invention. FIG. 3 is a diagram showing the configuration of the control unit 20 in more detail. It is the figure which showed typically the function structure of the action control system 100 of the robot apparatus 1 which concerns on embodiment of this invention. It is the figure which showed the flow of operation | movement by each object which comprises the action control system 100 shown in FIG. It is the figure which showed the flow of the information which enters into the target memory in the short-term memory | storage part 105 based on the recognition result in each recognition function part 101-103. It is the figure which showed the flow of the information which enters into the event memory in the short-term memory | storage part 105 based on the recognition result in each recognition function part 101-103. It is a figure for demonstrating the dialogue process with the user A and B by the robot. It is a figure for demonstrating the dialogue process with the user A and B by the robot. It is a figure for demonstrating the dialogue process with the user A and B by the robot. It is the figure which showed notionally the memory | storage process of the associative memory which concerns on one Embodiment of this invention. It is the figure which showed notionally the recall process of the associative memory based on one Embodiment of this invention. It is the figure which showed typically the example of a structure of the associative memory system to which a competitive neural network was applied. It is the figure which showed typically the object structure of the action control system 100 which concerns on embodiment of this invention. It is the figure which showed typically the form of the situation dependence action control by the situation dependence action hierarchy. FIG. 15 is a diagram illustrating a basic operation example of behavior control by the situation-dependent behavior hierarchy illustrated in FIG. 14. It is the figure which showed the operation example in the case of performing reflex action by the situation dependence action hierarchy 108 shown in FIG. It is the figure which showed the operation example in the case of expressing an emotion by the situation dependence action hierarchy shown in FIG. It is the figure which showed typically a mode that the situation dependence action hierarchy 108 was comprised by the some schema. It is the figure which showed typically the tree structure of the schema in the situation dependence action hierarchy. The internal structure of a schema is schematically shown. It is the figure which showed typically the internal structure of the Monitor function. It is the figure which showed the structural example of the action state control part typically. It is the figure which showed typically the example of another structure of the action state control part. It is the figure which showed typically the mechanism for controlling the normal situation dependence action in the situation dependence action hierarchy 108. FIG. It is the figure which showed the structure of the schema in the reflective action part 109 typically. It is the figure which showed typically the mechanism for controlling a reflective action by the reflective action part 109. FIG. It is the figure which showed typically the class definition of the schema used in the situation dependence action hierarchy 108. It is the figure which showed the state machine of the action function of schema. It is the figure which showed the state machine of the schema. It is the figure which showed typically the functional structure of the class in the situation dependence action hierarchy. It is the flowchart which showed the process sequence which performs the MakePronome function. It is the flowchart which showed the process sequence which performs a Monitor function. It is the flowchart which showed the process sequence which performs Actions function. It is the flowchart which showed the process sequence which performs Actions function. It is a figure for demonstrating Reentrant property of a schema. It is the figure which showed typically the hierarchical structure of the internal state management part 104 which concerns on this embodiment. It is the figure which showed typically the communication path | route between the internal state management part 104 and another functional module. It is the figure which showed the mechanism for the internal state management part 104 to change an internal state with a time change. It is the figure which showed the mechanism for the internal state management part 104 to change an internal state with operation | movement execution of a robot. It is the figure which showed the mechanism for the internal state management part 104 to change an internal state by the recognition result of an external environment. It is the figure which showed the structure for the internal state management part 104 to change an internal state by associative memory. It is the figure which showed the mechanism for the internal state management part 104 to change an internal state by innate reflection behavior. It is the figure which showed typically the relationship between a schema and an internal state management part. It is the figure which showed typically the Motivation calculation path | route by a Motivation calculation class object. It is the figure which showed typically the mechanism of the Motivation calculation process when a target object exists. It is the figure which showed typically the mechanism of the Motivation calculation process when a target object does not exist. It is the figure which showed the change method of Pleasantness. It is the figure which showed the change method of Activity. It is the figure which showed the change method of Certificate. It is the figure which showed the mechanism for calculating | requiring Certificate. It is the figure which showed the structure of ResourceManager. It is the figure which showed typically the control from SituatedBehaviorLayer to ReflexiveBehaviorlayer. It is the figure which showed typically the control from DeliverativeLayer to SituatedBehaviorLayer. FIG. 54 is a diagram illustrating another configuration example of the behavior control system of the robot apparatus 1. FIG. 55 is a diagram showing an example of the internal configuration of the situation-dependent action hierarchy 108. FIG. 56 is a diagram showing a flow of processing in which the behavior value calculation unit 220 calculates the behavior value AL from the internal state and the external stimulus. FIG. 57 is a diagram illustrating a state in which the reflection event density RED changes according to the occurrence of the reflection event. Figure 58 is a diagram m _i is an example of a probability Pm _i is selected by using the reflection event density RED. FIG. 59 is a diagram illustrating an example of behavior arbitration when the remaining battery level is medium and a crap is recognized from the outside world. FIG. 60 is a diagram illustrating an example of behavior arbitration when the remaining battery level is low and a crap is recognized. FIG. 61 is a diagram illustrating an example of behavior arbitration when the remaining battery capacity is sufficient and a crap is recognized from the outside world. FIG. 62 is a diagram illustrating an example of behavior arbitration when the remaining battery level is sufficient and a crap is recognized from the outside world.

Explanation of symbols

DESCRIPTION OF SYMBOLS 1 ... Robot apparatus 15 ... CCD camera 16 ... Microphone 17 ... Speaker 18 ... Touch sensor 19 ... LED indicator 20 ... Control part 21 ... CPU
22 ... RAM
23 ... ROM
24 ... Non-volatile memory 25 ... Interface 26 ... Wireless communication interface 27 ... Network interface card 28 ... Bus 29 ... Keyboard 40 ... Input / output unit 50 ... Drive unit 51 ... Motor 52 ... Encoder 53 ... Driver 100 ... Action control system 101 ... Visual recognition function unit 102 ... Hearing recognition function unit 103 ... Contact recognition function unit 105 ... Short-term memory unit 106 ... Long-term memory unit 107 ... Contemplation action layer 108 ... Situation-dependent action layer 109 ... Reflection action unit 220 ... Action value calculation unit 230 ... Action selection unit 240 ... learning unit

Claims (21)

In a robot device that generates an action based on an internal state or an external input,
A plurality of action control layers for controlling actions based on internal states or external inputs;
A resource management unit that manages resources of the robot device and resolves conflicts of operation commands related to driving of the robot device from each behavior control layer;
A robot apparatus comprising: Each of the behavior control layers is composed of one or more behavior modules that determine the behavior of the robot.
Each behavior module includes a behavior evaluation unit that outputs a behavior evaluation of the robot device according to an internal state or an external input, and a behavior command output unit that outputs a behavior command of the robot device, respectively.
In each of the action control layers, an action command for controlling the action of the robot apparatus is generated based on the action evaluation in the action evaluation unit of each action module and the resource used by the robot apparatus.
The robot apparatus according to claim 1. In each of the behavior control layers, the plurality of behavior modules have a tree-like hierarchical structure, and are used for behavior evaluation and output resources of the robot apparatus that are output from the behavior module in the lower layer to the behavior module in the upper layer. Selecting a behavior module based on and controlling the behavior of the robotic device;
The robot apparatus according to claim 2. The action command output means of each of the action modules outputs resources of the robot apparatus used when executing the action instruction in the robot apparatus,
In each of the behavior control layers, the behavior of the robot device is controlled based on the behavior evaluation and the resource used by the robot device output from the behavior module of the lower layer of the hierarchical structure to the behavior module of the upper layer.
The robot apparatus according to claim 2. The behavior control layer includes a reflection behavior control layer that directly receives an external input recognition result and directly determines an output behavior, and a situation in which the robot apparatus is placed based on an external input and an internal state of the robot apparatus. A situation-dependent action hierarchy that controls immediate action and a contemplation action control hierarchy that infers future situations and plans the behavior of the robotic device over a relatively long period of time,
The robot apparatus according to claim 2. The resource management unit mediates a conflict between operation commands of the reflex behavior layer and the situation-dependent behavior layer;
The robot apparatus according to claim 5. The resource management unit includes command conflict solution means for solving a conflict of commands from the reflection action hierarchy and the contemplation action hierarchy, and content management means for managing a command for each hardware resource of the robot apparatus.
The robot apparatus according to claim 5. The command conflict solution means
Holding the command, the hardware resource of the robot device used by the command, and the activity level of the command;
When a new command is transmitted from any of the behavior control layers, whether or not the hardware resource of the robot device used by the currently executing command and the hardware resource of the robot device used by the new command are in conflict Resolving command conflicts by comparing the activity levels associated with each other's commands, if any
The robot apparatus according to claim 7. The behavior evaluation unit in the behavior module includes a behavior induction evaluation value calculation unit that calculates an evaluation value that induces an output of a behavior command by the behavior command unit as an activity level, and a bias calculation that calculates a bias with respect to the activity level as an intention level And a use resource calculation means for specifying the resource of the robot device to be used when the action command by the action command means is executed.
The robot apparatus according to claim 5. The situation-dependent action layer outputs an action intention signal that suppresses or excites a reflection action according to whether or not the reflection action matches the intention of the situation-dependent action to the reflection action layer,
The reflex behavior layer includes behavior intention management means for managing a behavior intention signal, and an action contrary to the intention of the situation-dependent behavior by manipulating the intention level in each behavior module based on the behavior intention in the situation-dependent behavior layer. Suppress the module or excite a behavior module that fits the intention of context-dependent behavior,
The robot apparatus according to claim 9. The contemplation action layer outputs an action intention signal that suppresses or excites a situation-dependent action depending on whether or not the situation-dependent action matches the intention of the contemplation action, to the reflex action layer,
The situation-dependent action hierarchy includes action intention management means for managing an action intention signal, and an action module contrary to the intention of the contemplation action by manipulating the intention level in each action module based on the action intention in the contemplation action hierarchy To suppress behavior or excite behavior modules that fit the intention of contemplation behavior,
The robot apparatus according to claim 9. The behavior control layer directly receives a homeostasis behavior for keeping the internal state within a certain range, a situation-dependent behavior layer that spontaneously generates an idle behavior without a homeostasis purpose, and an external input recognition result. Includes a reflex behavior control hierarchy that directly determines output behavior,
A resource management unit that arbitrates between the self-issued movement output from the situation-dependent action hierarchy and the reflex action output from the reflex action control hierarchy;
The robot apparatus according to claim 1. The situation-dependent behavior hierarchy is composed of a plurality of elemental behaviors that output the respective homeostasis behavior and idle behavior,
The reflex behavior hierarchy is composed of a plurality of elemental behaviors that behaviorally output each reflex behavior.
The robot apparatus according to claim 12, wherein: Each has an action value calculation means for calculating an action value indicating the execution priority of each element action,
The situation-dependent action hierarchy further includes an action selection means for selecting an element action to be output as a self-issued action based on an action value.
The robot apparatus according to claim 13. The behavior value calculating means includes
While calculating the desire value from the internal state of the robotic device, calculating the expected satisfaction value from the internal state and the recognition result, calculating the action value of the elementary action of homeostasis action from the desire value and the predicted satisfaction value,
Give a certain value as the action value of elemental action of idle action,
The robot apparatus according to claim 14. A resource management unit that mediates between the element movement of the self-issued movement output from the situation-dependent action hierarchy and the element movement of the reflection action output from the reflection action hierarchy;
The robot apparatus according to claim 15. The situation-dependent action hierarchy and the reflex action hierarchy each give command strength to the element action to be output,
The resource management unit mediates between homeostasis behavior and reflex behavior, or idle behavior and reflex behavior based on the strength of the command,
The robot apparatus according to claim 16. The context-dependent behavior hierarchy gives command strength based on behavior value to elemental behavior of homeostasis behavior,
The robot apparatus according to claim 17. The situation-dependent action hierarchy gives a unique command strength to each element action of the idle action,
The robot apparatus according to claim 17. The reflex behavior hierarchy gives a constant value as command strength to all element behaviors,
The robot apparatus according to claim 17. A reflection event management unit that manages the reflection event density at which an external input event that causes reflection behavior occurs is provided.
The situation-dependent action hierarchy obtains a probability that each element action of the idle action is selected based on the reflection event density, and selects the element action according to the probability.
The robot apparatus according to claim 13.

Images