JP2007220076A - Interaction device - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To provide a robot apparatus for determining whether the human being is present in the outside world or not by using only a simple input/output sensor. <P>SOLUTION: A social robot 10 is configured by an optimal engine 11, real time controller 12 and an interaction manager 13 and own controller is set in order to maximize expectation of information defined between a hypothesis about an interaction object and own input/output. <P>COPYRIGHT: (C)2007,JPO&INPIT

Description

  The present invention relates to an interaction device that allows closer interaction between a robot and a user.

  The idea of infomax in perception goes back to research by Linsker et al. (See, for example, Non-Patent Documents 2 and 12).

  This approach was generalized to non-Gaussian networks by Bell and Seznovsky and became Infomax ICA. Infomax has become an important theoretical tool in computational neuroscience. However, most of this work looks from the Infomax passive perspective. The goal of the infomax processor is to easily send as much information about the input as possible to the next processing stage. On the other hand, infomax control works with active processors that can schedule actions to discover hypotheses of interest as quickly as possible. For example, nerve cells can be proliferated to discover the relevant state of the world as soon as possible through feedback connections, not just to convey information. A similar speculative view of the active role of neurons has been formulated by Kuprof in the form of the pleasurable nerve cell hypothesis.

Learning rules in Terry The problem of scheduling actions to find contingencies is formally associated with the two-group bandit problem. In the classic two-group problem, one of the two levers must be pulled to determine which of the two levers maximizes the return speed. We have corrected the problem to include an additional implicit variable H that determines whether the two conditions are equal or not equal. Therefore, in the two-group problem, the goal is to determine which of the two groups is better, whereas in the incident detection problem, the goal is to determine whether there is a difference between the two groups. There is. This subtle difference has important consequences. For example, in the standard 2 group bandit problem, it is possible to make a judgment by pulling only 1 group several times. That's because if one of the two groups yields an unusually large return, it is already evidence that the group is better. However, in the incidental event determination problem, information is not available until the two groups are pooled at least once.

  Conventionally, various proposals have been made on infomax approaches to perceptual processes and neural processes (for example, see Non-Patent Documents 2, 11, 12, and 20).

  However, these models are passive in that they are designed to convey information to the next processing stage. Instead, it emphasizes a model that selects behaviors over time to maximize long-term collection of information.

  Infomax control can also be found in the tradition of human motion control models that explain the behavior of optimization problems (see, for example, Non-Patent Documents 5, 7, 13, 27).

  However, the approach proposed here has unique features in that it applies to social behavior where the time scale and level of uncertainty are much larger than in the case of traditional motor control problems.

  In addition, the concept of information maximization was used to explain how people choose questions in a conceptual learning task (see, for example, Non-Patent Documents 16, 17, and 18).

  Infomax control has already been advocated to understand how people move their eyes or how they can move their active cameras to maximize data collection on events of interest. (For example, see Non-Patent Documents 4, 8, 9, 10, 21).

However, these models do not address some important issues.
(1) Previous models that focused on explaining behavior ordering did not address the issue of action timing. For example, in [Non-Patent Documents 16, 17, 18], conceptual learning is considered to be a strict shift activity in which the subject asks questions without time constraints and is given answers.
(2) The previous model does not solve the information maximization problem that has been raised. The model is at best “greedy” and in the worst case non-causal. For example, in [Non-Patent Document 17], a question is made to maximize the immediate information return rather than the information returned in the long term.

  In [Non-Patent Document 10], the observer is allowed to perform all possible eye movements first and then select the eye movement that happens to provide the most information. This approach is useful for modeling objectives, but is non-causal, that is, the future needs to be visible in order to trigger the current saccade.

  In the field of robot engineering, it is common to distinguish between behavioral robot architecture and cognitive robot architecture (see Non-Patent Document 1, for example).

  Behavioral architecture is based on direct mapping between sensors and actuators. This architecture emphasizes timing and quick response to changes in the environment. Cognitive architecture typically relies on planning and contemplation processes and the construction of world representation. Without an appropriate mathematical foundation, concepts like expression, contemplation, and knowledge are almost meaningless. For example, the infomax control framework proposed here is reactive in that the controller is simply a causal mapping between sensor information and behavior.

  The idea of infomax control is directly related to the Bayesian approach to the sequential decision process, especially the Bayesian solution to the n-group bandit problem. The author's contribution in this paper is how this well-known group of problems can be adapted for understanding real-time social interactions, and mutual information as an effective reinforcement signal. It is to show that it can be used.

  Game theory can be seen as a special case of control theory, but has a long history of application to human social behavior, especially in economics and conflict research. However, it is only recently that the importance of control to understand the importance of real-time social behavior has appeared in the literature. In order to understand the optimality of real-time social interaction, the inventors pointed out the potential value of probabilistic optimal control of the n-bandit problem in [Non-Patent Document 15]. Walpert, Doya and Kawato proposed an integrated framework for motor control and social interaction. Miyashita and Ishiguro pointed out that a simple PID controller can be used to create transmission behavior.

  Next, a simple social interaction will be described.

1 Contingency event detection and social development John Watson states that contingent event detection plays a critical role in the social and emotional development of infants. Accidental events are perceived directly by the human brain, and similarly the brain perceives other elements such as color and movement. In particular, in the early stages of infants, incidental events are a basic source of information about the definition and recognition of the caregivers (see, for example, non-patent documents 24 and 25).

  This view is derived from an experiment in which a 2-month-old child learned to move his head to activate a mobile above the baby head (see, for example, Non-Patent Document 24). Infants in the experimental group were given a mobile that responded to the infant's head movement. For the control group of infants, the mobile operated at the same rate as the experimental group, but in a random, non-incident manner. After 10 minutes each of this mobile experience and four times a day, the experimental group infants showed a much higher response rate than the control group infants. More importantly, at about the same time, the infants in the experimental group began to show a number of social responses that are typically directed at the caregiver. These social responses included vigorous social smiles, thirsts, and positive emotions towards mobile. Watson says that contingencies are used by young children as clues to define and identify the same, and that these cues are more important than other perceptual prejudices, such as the visible facial expressions of human faces Says.

  Watson formulated a Poisson model for judging social contingencies. In this model, background agents and social agents are modeled as Poisson processes. Watson's initial formulation did not address the question of how to schedule actions optimally or how to make an estimate under this model. Instead, he proposed a heuristic approach based on comparing probabilities that did not increase rapidly within a certain length interval.

  In 1986, the inventors conducted experiments to test how a 10-month-old child uses incidental event information to detect new social agents (see, for example, Non-Patent Documents 15 and 19). .

  An infant sits in front of a robot that does not resemble a human being. The “head” was a rectangular prism whose side included a geometric pattern (see FIG. 1A). The robot's head flashed light on its surface, made a sound, and was able to rotate left and right. Infants were arbitrarily assigned to experimental or control groups. In the experimental group, robots were programmed to react to the environment in a simulated form of human contingency characteristics. Each infant in the control group was associated with one infant in the experimental group and experienced the light, voice, and rotation of the central robot with the same time distribution as the corresponding test subject. However, in the control group, the robot did not respond to infant behavior or any other events in the room.

  Here, FIG. 1A is a schematic diagram of the head 50 of the robot used in Non-Patent Document 19. FIG. 1 (B) is a photograph of Infant-9. You can see the robot image in the mirror behind the baby.

1.1 Infants 43 seconds a day In the study, evidence was found that experimental infants treat robots as if they were social agents. For example, this group of infants uttered five times more often than the control group. Furthermore, when the robot rotated, it followed the robot's “gaze line” and showed some evidence of sharing attention (see, for example, Non-Patent Document 15). However, we were particularly surprised by the strength of the interactions that occurred between some infants and robots, the clear intentions of the infant's behavior, and the speed at which these interactions were deployed.

  The fact that some toddlers seemed to actively “judge” and act accordingly, in just a few seconds in several tests, whether or not the robot responded. It was. Of particular interest was the first 43 seconds of the experiment on one infant in the experimental group. The infant will be referred to as infant-9 (see FIG. 1B). As of July 14, 1986, when the study was conducted at the Human Development Institute at UC Berkeley, the age was 10 months. This 43-second video is available at http: /mplab.ucsd.edu. During this 43 seconds, Infant-9 performed seven voices, each time followed by a sound and light from the robot. Most people who saw the video of the experiment clearly detected the fact that the infant-9 was reacting to the robot by the third or fourth vocalization (25 seconds after the start of the experiment). I agree with you. Very importantly, when you watch the video, it is very clear that the child is actively asking the robot and testing whether the robot is reacting to him or not. This raises some interesting questions that are central to this paper.

1) What does "questioning" mean for an organism that does not have a language?
2) Why did Infant-9 schedule its vocalization in the way he did? For example, why did you not speak at a much faster or slower speed?
3) Was it reasonable for an infant-9 to make a decision that the robot is reacting within 3 to 4 reactions and within 20 to 30 seconds after the start of the experiment? Why wasn't the time and the number of reactions more or less than this?
The infomax control problem has recently become more common in perceptual and categorized literature, but these documents usually use a greedy one-step forward infomax rather than an optimal strategy.

  A major practical limitation of current systems is due to the simplicity of the current social agent model. In particular, the current system describes an agent as a passive responder, but not as an autonomous advocate of action with a communication intention. Extending the model to handle this problem is not complicated. However, it is better to spend time learning such models from the data than to hand-craft improved models for social agents.

  In addition to scheduling their responses in an optimal manner, Infant-9 progressively improved the tone and emotional quality of the response over the 43-second simulated period. It is possible to model this representation by linking its tone with changes in beliefs about the existence of social agents. This modification is effective in improving the interaction between the robot model and humans, but is not born from the current model in a principle-based manner, such as, for example, an alternation method.

  Infants-9 acted in an optimal way when it came to learning whether new social agents would respond, but most of the infants who participated in the experiment did not. The subjective feelings gained from watching these infants are that they are initially afraid of the situation.

  Based on the idea that an organism is driven by a goal and schedules its behavior in a way that optimizes the collection of information related to that goal, a general approach to organizing behavior is presented. Whereas traditional and instrumental learning models emphasize the role of external stimuli as behavioral enhancers (food, water, discomfort, breathing, and light electric shock are most typical), Infomax In control, stimuli and responses have no intrinsic value. Instead, given the current state of knowledge of the organism, its value relates to expected information returns. Infomax can be thought of as a self-managing form of control in which the organism itself assigns enhanced values to stimuli and responses in a dynamic manner. External enhancement factors are not required. Instead, the infomax controller modifies its internal state in order to better describe the available data and bring about actions that are expected to provide highly informational data.

  In detecting social contingencies in 10-month-old children, we illustrated how the Infomax controller concept can be used to understand simple social interactions. Interestingly, in this situation, the optimal infomax controller exhibits shift behavior similar to that found in infants of that age. That is, the controller responded, followed by a period of silence, as if waiting for questions. This “replacement” behavior was not built into the system. Given the general level of time delay and uncertainty in social interactions, rather, the behavior stems from the demand to maximize the information available. The results suggest that infants of that age are already asking questions despite the lack of language. That is, given the time delay and uncertainty levels typical of social interactions, infants schedule their actions in a way that maximizes expected information returns. This is important for parents to know at an intuitive level, but difficult to formally prove.

  The approach presented here actually works well even when applied to robots that need to operate in real time in everyday life situations. This not only provides confidence in the idea that contingencies are useful and computationally inexpensive sources, but also allows the infant's brain to use contingencies to define and detect the same. It also gives us confidence in the idea that

  Since infomax control has a mathematical basis for probability and control theory, it can be extended to other areas in a principle-based manner. For example, current analysis can be extended to rats, neurons, and even molecules. Current infomax models of neural activity provide neurons with a role as passive information relays, i.e. the role of neural responses is to convey as much information as possible about the information they receive. Infomax control may be designed to collect information about other neurons, not just to convey information to other neurons, in which neurons may “question” Provide a framework for examining interesting possibilities that might have been Of course, the feedback combination can be viewed as a channel for obtaining answers to questions.

  The inventors have illustrated a general approach to behavioral research inspired by David Ma, a pioneer in computational neuroscience (see, for example, Non-Patent Documents 6 and 14).

R. C. Arkin. Behavior-based Robotics. MIT Press, Cambridge, MA, 1998. T. Bell and T. Sejnowski. An information-maximization approach to blindseparation and blind deconvolution. Neural Computation, 7: 1129-1159, 1995. C. Breazeal. Designing Sociable Robots. MIT Press, Cambridge, MA, 2002.28 Reichle E. D., Rayner K., and A. Pollatsek. The E-Z reader model of eyemovement control in reading: comparisons to other models.Behavioral and Brain Sciences, 26: 445-526, 2003. Todorov E. and Jordan J.I.Optimal feedback control as a theory of motor coordination.Nature Neuroscience, 5: 1226-1235, 2002. S. Edleman and L. M. Vaina. David marr. International Encyclopedia of the Social and Behavioral Sciences, 2001. Tanaka H., Krakauer W., and Qian N. An optimization principle for determining movement duration.Under Review, 5, 2005. Denzler J. and Brown C. M. Information theoretic sensor data selection for active object recognition and state estimation.Transactions on Pattern Analysis and Machine Intelligence, 24: 145-157, 2002. Najemnik J. and Geisler W. S. Optimal eye movement strategies in visual search.Nature, 434, 2005. Renninger L. and Coughlan J., P. Verghese, and J. Malik.An information maximization model of eye movements.In SA Solla, TK Leen, and KR Miller, editors, Advances in Neural Information Processing Systems, volume 17, pages 1121 -1128. MIT Press, 2005. M. S. Lewicki. E_cient coding of natural sounds.Nature Neurosci, 5 (4): 356-363, 2002. R. Linsker. Self-organization in a perceptual network.Computer, 21: 105-117, 1988. Harris C. M. and Wolpert D. M. Signal dependent noise determines motor planning.Nature, 394: 780-784, 1998. David Marr. Vision.Freeman, New York, 1982. J. R. Movellan and J. S. Watson.The development of gaze following as a Bayesian systems identification problem.In Proceedings of the International Conference on Development and Learning (ICDL02) .IEEE, 2002. JD Nelson and JR Movellan. Active inference in concept induction.In T. Leen, TG Dietterich, and V. Tresp, editors, Advances in Neural Information Processing Systems, number 13, pages 45-51.MIT Press, Cambridge, Massachusetts, 2001 . J. D. Nelson, J. B. Tenenbaum, and J. R. Movellan. Active inference in concept learning.In Proceedings of the 23rd Annual Conference of the Cognitive Science Society, pages 692-697. LEA, Edinburgh, Scotland, 2001. Jonathan Nelson, Gary Cottrell, and Javier R. Movellan. Explaining eye movements during learning as an active sampling process.In Proceedings of the second international conference on development and learning (ICDL04), The Salk Institute, San Diego, October 20, 2004. Movellan J. R. and J. S. Watson. Perception of directional attention. In Infant Behavior and Development: Abstracts of the 6th International Conference on Infant Studies, NJ, 1987. Ablex. R. P. N. Rao, B. A. Olshausen, and M. S. Lewicki. Probabilistic Models

  The present inventors have explored the theory of probabilistic optimal control as an expression format for understanding active real-time learning in order to link and systematically explain the problems of real-time learning and real-time social interaction. The term “real time” is intended to emphasize the time pressure on the behavior that appears during learning. The term “active” refers to the fact that the behavior of the learning agent has a purpose, ie the fact that its legitimacy becomes apparent when considered as a solution to the optimization problem.

  Control theory includes reinforcement learning as a sub-field, but has traditionally been applied to optimize physical objectives (eg tracking moving objects, maintaining pole equilibrium, maintaining speed under variable loads). It was. From this point of view, learning is seen as a process of developing a controller by trial and error. In this paper, the author takes a different perspective and explores the learning process itself as a control problem. In order to emphasize its direct relationship with the perceptual infomax model, this idea is called infomax control (see, for example, Non-Patent Documents 2, 11, 12, and 20).

  This is a natural application of control theory (ie, “learning” is the goal of the controller) and at the same time brings a perspective to learning science that has not been fully pointed out in the literature. is there.

  The use of the word “reinforcement learning” is for some people that the reinforcement factor should be the target event (eg victory in backgammon games, helicopter crash prevention, food acquisition, avoidance of electric shock) It has had the unfortunate result of believing. Instead, the “enhanced” signal used in Infomax is associated with subjective beliefs. There is no need to explicitly tell the learning agent whether it is right or wrong; instead, “strengthening” is the agent's own ability to build strong beliefs. This is a general property of the fact-Bayesian approach, which can be modified to a subjective interpretation. This approach avoids justification by teaching the basis for updating beliefs about normative constraints in probability theory. At present, it is a general trend to distinguish between control theory and reinforcement learning due to the fact that control theory requires a world model and reinforcement learning does not require a world model. However, this distinction is not clear in the Bayesian approach. Of course, the Bayesian tradition simply means that the lack of models has vague prior beliefs about which models are persuasive and which models are not persuasive. It was established on the basis of In short, Bayesian approach has a model for “lack of model”. By this. The Bayesian approach controls a unique self-management character.

  The object of the present invention is to use infomax control to understand the development of simple real-time social interactions. “Real-time social interaction” means the rapid exchange of action signals under time pressure in a face-to-face social environment. The communication channel in this case has a feedback delay of a fraction of a second to a few seconds. Due to the fact that social agents are autonomous and difficult to predict, the level of uncertainty regarding the outcome of behavior and the uncertainty regarding the timing of such outcome are important.

  From this, the domain of the present invention has a traditional motor control domain where the delay is measured in units of 1/10 seconds and the uncertainty is negligible, and a long feedback delay and negligible time constraint Other forms of interaction (eg, communication through physical characters or email) will be different. However, the specifics of the area of social interaction are different from those of traditional motor control, but the underlying mathematical form may be the same.

  The idea of the present invention was born from a study for understanding the shocking behavior of a 10-month-old child in an experiment conducted by the present inventors in 1985 at UC Berkeley.

The purpose of the experiment was to understand how children learn the causal structure of social agents. To this effect, children interact with the robot and may or may not react to the robot. The children I focused on in this paper embody some characteristics that I think are indispensable for understanding human behavior, but these characteristics were missed by the learning model at the time. there were.
(1) The children learned to interact properly with the robot much earlier than we expected.
(2) The children were clearly active as if they were asking questions in a non-verbal way, if the robot did not respond. At that time, the author did not have an expression format to understand this learning behavior. Combined theoretic approaches such as back-propagation are too slow and passive, and traditional AI approaches have not dealt with problem uncertainty and real-time constraints.

  The inventors' interest in control theory was born during research on the development of robots designed to interact with people. Such robots faced the inherent problems associated with timing and the dynamic processing of uncertainties typical of interactions with biological systems. The inventors have become convinced that the theory of stochastic optimal control is the ideal form of expression to handle these problems, and in the process John Watson and the author have observed what they once observed in 1985. I realized it would be an excellent explanation.

  The inventors have proposed the idea of infomax control in relation to the development of social interaction, but this approach is universal and can potentially be applied to a very wide variety of problems. It is. Of particular interest is the fact that this idea gives a formal definition of what a “question” is, which can be universalized into non-linguistic organisms.

  Other objects of the present invention and specific advantages obtained by the present invention will become more apparent from the description of embodiments described below.

  The interaction device according to the present invention is characterized in that the self-controller is set in order to maximize the expectation of information defined between the hypothesis about the interaction target and the self-input / output.

  In addition, the interaction device according to the present invention includes a control unit that outputs an action at a timing when the expected acquired information amount for the presence of the interaction target is maximized for the interaction target based on the input / output information.

  In the present invention, the extraction of the concept of contingency in developmental psychology is implemented in the framework of Bayesian estimation, so that the certainty of the hypothesis whether there is a person to interact with or not is obtained from time to time in the form of probability values. Therefore, it is possible to realize a robot apparatus that can determine whether or not there is a person in the outside world using only a simple input / output sensor.

  Further, in the present invention, the response characteristic changes from moment to moment through interaction, and the dynamics thereof are similar to those of humans. Therefore, it shows more natural response characteristics and is particularly effective in the application scene of long-term interaction.

  Hereinafter, embodiments of the present invention will be described in detail with reference to the drawings. Needless to say, the present invention is not limited to the following examples, and can be arbitrarily changed without departing from the gist of the present invention.

2 Aiming at an architecture for social robots In this specification, unless otherwise specified, uppercase letters are used for random variables, lowercase letters are used for specific values taken by random variables, and Greek letters are used for fixed parameters. It is assumed that the characteristic of the probability space (Ω, F, P) in which the random variable is defined is inherent. If it becomes clear from the situation, the probability function is identified by its argument. For example, p (x, y) is an abbreviation for the random probability X taking the unique value x and the random variable Y taking the unique value y for the joint probability mass or joint probability density. A subscript colon is used to denote a sequence of numbers, eg, X _{1: t} = ^def {X ₁ ... X _t }. The work is done by a discrete time stochastic process. The parameter ΔtεR represents a sampling period, that is, a time (unit: second) between time stages. Choosing a unique value of Δt is equivalent to indicating that the relevant information about the basic continuous time process is in a frequency band lower than 0.5 / Δt hertz. The symbol ~ indicates the distribution of random variables. For example, X to Poisson (λ) indicates that X has a Poisson distribution with the parameter λ. The notation Yεσ {X} means that the random variable Y can be measured by the sigma algebra induced by the random variable X. Intuitively, this means that X contains all the values necessary to determine the value of Y. E is used for the expected value and Var is used for the covariance matrix. δ (·, ·) is used in the Kronecker delta function, where the two arguments are equal, the value is 1; otherwise, the value is 0. N = {0, 1, 2,...} Represents a natural number, and R represents a real number.
[25] Robot architecture is mapped by mapping between two stochastic processes, namely perceptual process Y = {Y ₁ , Y ₂ ,...} And movement process U = {U ₁ , U ₂ ,. Suppose there is. At time t, the robot can obtain information on Y ₁ : t and U _{1, t−1} , such information, namely U ₁ ∈σ {Y _{1: t} , U _{1, t−1.} }, An exercise command must be created. The fact that such mapping, functional statistics S _t to maintain relevant information about past _history, i.e.,

Depends on. In pondering architectural, S _t will be referred to as the world of representation, a large amount of the available resources are allocated to the maintenance of such a world of expression. Reactive architecture emphasizes the idea that the world is constantly changing and therefore there is little to draw attention to past history. In its purest form, the reactive architecture is:

  Here, this problem was investigated from the viewpoint of the minimum necessary robot equipped with a single binary sensor (for example, a sound detector) and a single binary actuator. There are two players: (1) a social agent that plays the role of the caretaker, and (2) a robot that plays the role of the infant. Agents and robots are in an environment that may have random background activity. The role of the robot is to find the social agent that reacts as quickly and accurately as possible.

Here, as shown in FIG. 2, for example, the social robot 10 having only a minimum necessary function for generating the binary actuator output U _t in response to the binary sensor input Y _t includes the optimization engine 11, the real-time controller, and the like. 12 and the intention manager 13.
[27] Robot actuator activity is represented by a binary random process {U _t }. The value of the variable U _t is 1 when the actuator of the robot is operating, and 0 otherwise. The presence or absence of a reacting social agent is represented by a random variable H. {H = 0}, that is, the absence of a reacting agent is called a “null hypothesis”, and {H = 1}, that is, the presence of a reacting agent is called an “alternative hypothesis”. The parameter π represents the prior probability of the alternative hypothesis, that is, the robot's initial belief regarding the presence of social agents before the collection of perceptual information.

2.1 Modeling social agents Although extremely simplified, the following model has the advantage of being mathematically easy to handle and the two essential characteristics of interest: (1) The level of response is also different, and (2) social agents have the advantage of maintaining considerable delays and reacting with a significant level of uncertainty in these delays. This social agent model is rich enough to explain how the idea of stochastic optimal control can be used to construct real-time social interaction problems.

It is assumed that the behavior of the social agent depends on two auxiliary processes: a timer {Z _t } and an indicator {I _t }. Timers allow modeling of time delays and temporal uncertainties typical of social interactions. The timer takes a value at {0,..., Τ ₂ ^a }, where τ ₂ ^a εN is a model parameter, the meaning of which will be described below. The timer keeps track of the number of time steps since the last robot action until τ ₂ ^a (see FIG. 3), ie:

The indicator vector I _t = (I _{1, t} , I _{2, t} , I _{3, t} ) ^T is the category in which time t is the following: (1) the “self-period” indicated by I _{1, t} ( 2) It consists of three binary variables indicating whether it belongs to the “agent self period” indicated by I _{2, t} or (3) the “background period” indicated by I _{3, t} . The meaning of these three periods will be described below.

The reaction time of the social agent is in the range of parameters 0 ≦ τ ₁ ^a ≦ τ ₂ ^a . That is, in order to react to the action from the robot, the agent needs every time step in the range of τ ₁ ^{a to} τ ₂ ^a . The “agent period” specified by the indicator process {I _{2, t} } is a period in which the agent can react to the previous robot action if the agent can exist. Therefore, it becomes as follows.

During the agent period, the robot sensor is driven by a Poisson process {D _{2, t} } having a velocity R ₂ . The distribution of R ₂ depends on whether the reacting agent is present in the form defined below.

Here, FIG. 3 is a graphical representation of the dynamics of the timer and the indicator variable, and the delay parameters were τ ₁ ^S = 1, τ ₂ ^S = 2, τ ₁ ^a = 4, τ ₂ ^a = 5.

2.2 Modeling the self-feedback process and background process Allows the robot sensor to react to the robot actuator, for example, the robot can hear its own utterance, and further delays in this self-feedback loop And taking into account uncertainty. In particular, the distribution of the self-feedback reaction time is assumed to be uniform with respect to the parameter τ ₁ ^S ≦ τ ₂ ^S , and τ ₂ ^S <τ ₁ ^a . Thus, the indicator variable for the self-feedback period is defined as follows:

During the self period, the operation of the sensor is facilitated by a Poisson process {D _{1, t} } with a velocity R ₁ .

The background process is modeled as a Poisson process {D _{3, t} } with velocity R ₃ . The background process promotes sensor activity that is neither due to self-feedback nor to the response of social agents to robot behavior. Note that background activities can include actions from external social agents that are not particularly responsive to the robot (for example, two social agents talk to each other, thereby May be activated). The background velocity R3 is given a prior beta distribution having parameters β _3,1 , β _3,2 reflecting the variability of background activity according to the situation. If β _3,1 = β _3,2 = 1, all reaction rates are a priori equally possible, ie

Reflecting, the distribution has no information value.

  The background indicator tracks the period during which self-feedback or reaction behavior from social agents may not occur, i.e .:

2.3 Robot Sensor Modeling Sensor activity is an exchange Poisson process. Promoted by Poisson process {D _{1, t} } during self-feedback period, promoted by {D _{2, t} } during agent period, and promoted by {D _{3, t} } during background period. That is, it is as follows.

Furthermore, it is necessary to specify the distribution of the reaction rate R2 during the agent period. If an agent is present, that is, if H = 1, then R ₂ is made independent of R ₁ and R ₃ , and R ₂ is a parameter β _2, which reflects the variability of background activity according to the situation _. Give a prior beta distribution with ₁ , β _2,2 . If β ₂ , 1 = β ₂ , 2 = 1, the prior distribution has no information value (blank state approach). If there is no agent, that is, if H = 0, the reaction rate during the agent period and the reaction rate during the background period are the same, ie, R ₂ = R ₃ . In my view, it is a beta with beta 11 and beta 12 as parameters, and that these betas have no effect on the behavior of the model, and therefore no parameters are specified. I need to say.

2.4 Auxiliary process The process {O _t , Q _t } will be used to record the sensor activity and its absence up to time t during the self period, the agent period and the background period. In particular, when t = 1, 2,...

2.5 Stochastic constraints Appendix I contains a summary of the parameters, random variables, and stochastic processes that specify the model.

  FIG. 4 shows a Markov constraint in the joint distribution of different variables included in the model. The arrow from variable X to variable Y indicates that X is the “parent” of Y. The probability of a random variable is conditionally independent from those variables, assuming all other variables are parent variables. A dotted line portion indicates an unobservable variable, and a solid line portion indicates an observable variable.

Here, in the generation model shown in FIG. 4, the controller C _{t + 1} maps all observation information to the action U _{t + 1} by time t. The effect of action depends on the timing which is determined by the presence or absence and Z _t agents H to react. The infomax controller maximizes the information return for the value of the implicit variable of interest, eg, H.

3 Development and learning. Inference and Control Here, “development” refers to the problem of finding the causal structure underlying social interaction, that is, the problem of finding the type of model shown in FIG. This is a difficult problem that may require collecting large amounts of data over months or years. Here, “learning” refers to a problem of finding an incidental event, that is, a problem of making an inference regarding an unobservable variable of a certain model. This is generally a process that requires less data than model development and may occur within seconds, minutes or even hours.

  Development and learning depend on two basic processes: inference and control. Inference refers to the problem of associating prior data with sensor data in a principled manner. Control refers to the problem of scheduling actions in real time to achieve an organism's goals.

3.1 Development In fact, the models we have developed so far simply state that robots may encounter two “causal clusters” when interacting with the world. (See FIGS. 5A and 5B).

  Here, (A) and (B) in FIG. 5 are diagrams showing two contingent event clusters created by the model. Variable H indicates which of the two variables is active in the current situation. FIG. 5A shows the contingency event cluster 1 “reactive agent absence”, and FIG. 5B shows the contingency event cluster 2 “reactive agent presence”.

In [Cluster 1], sensor activity tends to change with respect to background activity during the period [τ ₁ ^s , τ _s ² ] following a certain action. This is due to the effect of self-feedback.

In [Cluster 2], sensor activity tends to change during [τ ₁ ^s , τ ₂ ^s ], but also changes during the period [τ ₁ ^a , τ ₂ ^a ] following a certain action The second change in activity is due to the presence of responding social agents.

  A very pioneering architecture in social robotics conveniently relies on mind theory approaches from developmental psychology (see, eg, Non-Patent Documents 3 and 22). The emphasis of these approaches is the idea that young children are born with a high-level knowledge module for humans and significant agents. On the other hand, the proposed robot architecture does not use a clear label or conceptual theory.

  In the above description, causal models have been developed by hand, but current machine learning methods can also be used to find these causal clusters. As the theory approach of mind argues, during development, these clusters may not correspond to any concept that can be easily described in words. For us, it is enough to discover the existence of the above types of causal clusters and realize that such clusters are useful when operating in the world. This is well possible within the current machine learning technology.

3.2 Learning: Reasoning Here, it is assumed that the robot has already developed a causal model and how social agents are present or absent based on a series of sensor activities y1: t and actions u1: t Focus on whether to judge. _{(Y 1: t, u 1} : t, o t, q t, z t) is assumed to be any sample from _{(Y 1: t, O t} , Q t, Z t: t, U 1). Then

It becomes.

Note that the velocity variables R ₁ , R ₂ , R ₃ are independent under the prior distribution. Furthermore, when H = 1, these variables affect the sensor in a non-crossing set of time. Therefore, the speed variable is independent even under the posterior distribution. In particular,

It becomes.

Under the null hypothesis, R ₂ = R ₃ , ie sensor activity does not change during the “agent” period. Furthermore, the set of times when the sensor activity depends on R ₂ , R ₃ does not intersect the set of times when it depends on R ₁ . Therefore, R ₁ is independent of R ₂ and R ₃ under the posterior distribution. That is,

It becomes.

For any r such that p (r | y _{1: t} , u _{1: t} , h)> 0,

Is obtained.

  Therefore, there is something wrong with the transition from (17) to (18). In (17), if u () is ignored, the ratio q is obtained even if o + q = 0. In (18), it cannot be obtained.

as well as

Here, the fact that R ₂ = R ₃ with probability 1 under H = 0 was used. Therefore, the log likelihood ratio between the two hypotheses is:

  Furthermore, the posterior distribution for the hypothesis of interest is as follows:

3.3 Cases of interest More specifically, it is as follows.

This posterior distribution contains all the information available to the robot about the reacting agent. The two important properties are (1) not dependent on o _{1, t} , q _{1, t} , ie, self-feedback has no information value about the hypothesis, and (2) o _{1, t , If t} + q _{1, t} = 0 or o _{2, t} + q _{2, t} = 0, the log likelihood ratio is zero. In short, if the data is not collected in the agent or the background state, information on H is not obtained. Therefore, in order to obtain information about H, the robot must use its actuator at least once and not at least once.

  If the data has not yet been collected, the likelihood is 1, that is, the posterior likelihood is equal to the prior likelihood.

  If the agent data has not yet been collected, the likelihood is 1 regardless of how much background data has been collected.

If background data has not yet been collected and β ₂ = β ₃ , the likelihood is 1 regardless of how much agent data has been collected.

If background data has not yet been collected, but β ₂ ≠ β ₃ , the collection of agent data has no information value, and is particularly as follows.

  Collect background bits and suppose it is +1. This value is difficult to explain either by the null hypothesis or the alternative hypothesis. This is because no information is available. Let's say that we don't have background data, but we collect 1 bit from the agent time and it is +1. The null hypothesis has a simple result explanation time, but no alternative hypothesis. Therefore, no information is obtained. If the bit is 0, the null hypothesis will have a simpler result explanation time than the alternative hypothesis. Information has been obtained.

3.4 Learning: Infomax Control In this section, we focus on how to schedule robot actuator behavior to maximize the expected information return on the presence or absence of social agents. It is assumed that t represents the current time and T> t represents some future time. Let C = {C _r : r = t + 1,..., T} denote a closed-loop controller, that is, a set of functions that map a series of observations to actions, ie:

  Controller C treats it as a random object consistent with the Bayesian approach. The goal is to find the value that C takes, which is a necessary condition to minimize the uncertainty about H.

  The present invention provides a controller c that maximizes future information returns for H when C = c is a necessary condition.

  The expected information return when using controller c is the mutual information between H and the observable variable that will be obtained at that time, ie

Where T represents mutual information, H represents entropy (see Appendix III), and is assumed to be (Y _{t + 1: r} , C), where H is conditional from U _{t + 1: r} Used the fact that they are independent. The equation teaches that the information about H provided by the observable processes Y _{t + 1: s} , U _{t + 1: s} is equal to the reduction in uncertainty provided by those observable processes. Maximizing the information gain is equivalent to minimizing the future entropy of H, since the term H (H | y _{1: t} , u _{1: t} ) does not depend on the controller. We will use this fact to develop an information-based utility function. Assuming that an observable variable at time r is given, _let W _{r be} an uncertainty with respect to H (see Appendix III for definition of conditional expectation).

  Therefore, the following equation is obtained.

Given the controller c, the expected return for the observed sequence y _{1: t} , u _{1: t} is as follows _:

Where α _r ≧ 0 is a fixed number indicating the relative value of the information at different times in the future. Our goal is to find a controller c that maximizes the expected return.

Assuming that an optimal controller is given, the optimal expected return for the sequence (y _{1: t} , u _{1: t} ) is defined as that expected return.

  Optimal controller and optimal expected return is Bellman's optimality formula, ie

  here,

It is easy to show that

For partially observable Markov processes, it is generally difficult to solve Bellman's formula exactly. The reason is that the number of possible sequences increases too quickly as a function of time. Fortunately, in our case there is a recursive statistic S _t = ^def (O _t , Q _t , Z _t ) that summarizes the observable sequence without any loss of information about H, The problem becomes simple. This allows the optimal controller to be used using standard dynamic programming recursive algorithms (see Appendix II).

4 Analysis of optimal controller
The dynamic programming problem was solved using a cluster of 24 2.5 GHz Power PC G5 CPUs. The calculation time was approximately 12 hours. The model parameters were set as follows. T = 40; τ ₁ ^s = 0; τ ₂ ^s = 0; τ ₁ ^a = 1; τ ₁ ^a = 3; π = 0.5. Next, logistic regression was used to model the controller's behavior for the time 15 <t <25 because the time is not too close to the beginning and end of the window of interest of the controller, That is, because tε (see, for example, Non-Patent Documents 1 and 40). Logistic regression predicted optimal controller behavior with 96.46% accuracy for all possible conditions. The final model was as follows:

  Interpretation: Optimal controller derivation was somewhat difficult, but the final product is a simple reaction system that can be easily operated in real time. What was provided by guidance was a guarantee that this simple controller was optimal for the task at hand. There is no better control under this model. Note that a greedy one-stage controller (eg, see Non-Patent Documents 16 and 17) that ignores future expected returns will fail this task. The reason is that when reacting, the next time step is occupied by self-feedback, and it happens to have no information value, so that eventually a greedy controller will make a decision that it will never act. By including expected future returns, it is possible to understand that the controller can automatically proactively and in the long run, taking action is better than not taking action. become.

  In order to determine when to act, the controller

Is used. This statistic is provided by the new observations from the period during which R _i actively drives the sensor, ie the self-feedback period for R ₁ , the agent period for R _{2 and} the background period for R _3. Is the expected reduction in variance with respect to R _i . Thus, the optimal controller appears to want to keep the uncertainty regarding R ₃ and R ₂ within a certain ratio. If R ₂ is too uncertain an agent rate, the controller selects to act. If R ₃ is a background rate too uncertain, the controller chooses to keep silent, thereby acquiring information about the background activity rate. When variance for the background rate R ₃ is at least nine times the size of the variance for the background rate R _2, it is interesting to note that the action is awakened. The reason for this magnification may be the fact that behavior is more costly than lack of behavior in terms of information return. If the robot behaves at time t, the self-feedback observation results have no information value about H, so the robot does not acquire information during time [t + τ ₁ ^s , t + τ ₂ ^s ]. Furthermore, during the time [t + τ ₁ ^a , t + τ ₂ ^a ], the controller commands the robot not to act, so during these periods the robot can only get information about R _2. impossible, it is impossible to obtain information about R _3. In contrast, if the robot does not act at time t, then no more time is wasted than self-feedback. This is before the action takes place, why might uncertainty about the agent activities speed R ₂ is illustrative of what is required greater than uncertainty about background activity rate R _3.

5 Infomax Control as a Form of Self-Managed Learning The important thing here is that the system is never explicitly told about its failure or success, ie the true value of H. In principle, the system can easily learn the optimal policy by interacting with the world, never being told if the agent was present or absent. This is important for learning models that may not be able to take advantage of external critics. Recently, some people have called this form of learning self-managed learning.

  Reinforcement learning can be viewed as a division of optimal control theory that relies on sampling methods to find optimal controllers. As such, a reinforcement learning approach could be used instead of dynamic programming to develop an optimal infomax controller. What dynamic programming has given is an official guarantee that the controller was optimal, that is, there was no better controller than this controller.

6 Infants' understanding of 43 seconds per day In this section, as described in section 1.1, to obtain a qualitative understanding of the first 43 seconds of the experimental session with Infant-9, Apply the controller. During this time, Infant-9 uttered seven times, and the time point of utterance was {5.58, 9.44, 20.12, 25.56, 32.1, 37.9, 41.7 from the beginning of the experiment. } Seconds later. After these utterances, the robot always uttered voice and light simultaneously. The time interval (unit: milliseconds) between two consecutive infant utterances is as follows: {4.22, 10.32, 5.32, 6.14, 5.44, 3.56 }Met. Most people admit that by the third or fourth utterance, infants are aware that there is an agent that reacts in the room.

For the infomax controller presented in section 3.4, it is necessary to set five parameters: a sampling period for time censoring, two self-delay parameters, and two agent delay parameters. A pilot study is performed to make an approximation for these parameters. With regard to the agent latent parameters τ ₁ ^a and τ ₂ ^a, we asked four people to talk to the computer's animated characters without telling the purpose of the study. Participants were 4, 6, 24, and 35 years old. To binarize voice sensor activity, an optimal encoder was used and the probability of this binary sensor activation was plotted as a function of time for the entire 150 tests. Each test started with the voice of the animated character and ended 4 seconds later. The result is shown in FIG. The graph in FIG. 6A shows voice sensor activity as a function of time from the start of the character's utterance over 150 tests. Each horizontal line is a different test. The first vertical bar is due to self-feedback from the character. From the end of the utterance of the animated character to about 1200-1400 msec, another peak of sensor activity occurs, which is due to the utterance of the human participant. The graph of FIG. 6B shows the probability of sensor activity as a function of time reduced throughout the test. Note the first peak of activity due to self-feedback and the gradual increase and decrease in sensor activity due to human reaction. Based on this graph, the optimal controller is simulated with the following parameters: Δt = 800 msec, τ ₁ ^s = τ ₂ ^s = 0, τ ₁ ^a = 1, τ ₂ ^a = 3. In short, the self-delay is negligible with respect to the expected delay of human reaction, and human activity occurs within 800-2400 seconds. In order to simulate the worst case scenario, more data is needed to set π = 0.01 and thus to determine that a reacting system exists.

  (A), (B), (C), and (D) of FIG. 7 show the results of the simulation. The horizontal axis in all graphs is time (unit: second). The graph in FIG. 7A shows the utterance of the optimal controller that plays the role of infant-9. The controller made 6 voices over a 43 second period. The average time interval between vocalizations was 5.92 seconds compared to 5.833 seconds for infant-9. This difference is not significant when using a standard T test (T (9) = 0.08, p = 0.94).

  The graph in FIG. 7B shows the system's beliefs regarding the presence of reacting agents. By the fourth reaction 30 seconds after the start of the experiment, this probability exceeds 0.5 level. The graph of FIG. 7C shows the posterior probability distribution regarding the agent reaction rate and the background reaction rate until the end of the period of 43 seconds. The graph of FIG. 7D shows the ratio between the uncertainty related to the sensor speed during the agent period and the uncertainty related to the speed during the background period. Note that when this ratio reaches 9, the simulated infant reacts.

  Therefore, given the level of time delay and uncertainty commonly found in social interactions, this model is designed to allow infants to schedule their reactions and optimally respond to social agents. This indicates that a judgment has been made. This model was “inquired” to the robot by the infant-9 in the sense that the infant-9 ’s utterance was scheduled to maximize the information returned about the robot's responsiveness. There is also consistency with this idea. Another point of interest is that the optimal controller takes turns, i.e., after an action has taken place, the controller waits an average of 5.92 seconds before the next utterance. The time interval between utterances is not fixed and will depend on the relative uncertainty regarding the level of agent-background reactivity. For example, if unexpected background activity occurs, the controller automatically extends the time interval between utterances to better “understand” changes in background activity. In the event of unexpected agent activity, the controller increases the reaction rate and accelerates the collection of information about the agent period.

7 Real-Time Robot Implementation In order to study this problem, the optimal infomax controller described above was implemented in the humanoid robot RobovieM developed at the Intelligent Robotics Laboratory of ATR. Robots are not necessarily required to test real-time controllers, but they greatly help to improve the quality of interactions developed between humans and machines, and thus make more realistic controller test methods possible. Provided. RobovieM has 22 degrees of freedom (shoulder: 1 degree of freedom, waist: 1 degree of freedom, arm: 2 degrees of freedom, leg: 2 degrees of freedom). The height is 29 cm and the weight is about 1.9 kg. The corresponding 22 servos are controlled by an H8 16 MHz microcontroller. The real-time infomax controller was implemented in Java and implemented on a host computer, such as the MacPowerBook G4, which graphically displays different states of the controller in real time, such as the posterior distribution of different shadow variables. Communication between the host computer and the controller was performed wirelessly using Bluetooth to a serial adapter from Wireless Cables. Current versions of the Infomax controller require a 1-bit sensor and a 1-bit actuator. For the sensor, the average voice energy over a 500 msec window was chosen and truncated using a 1-bit optimal coder. The actuator was a small loudspeaker that emits a 200 msec robot sound. The controller's self-time delay parameter was selected by measuring the time delay between issuing the sound producing command and receiving feedback from the voice sensor. Agent delay parameters were the same as in the infant-9 simulation (see Section 6).

  Based on the controller's belief about the presence / absence of the reacting agent in addition to the robot's utterance, the posture indicates a high level of attention when the controller believes that the agent is present, and the agent does not exist It changed to a posture that showed boredom when the controller believed.

7.1 Unsteady environment In the model proposed here, the agent and background states represented by the variables R and H are random, but stationary. For practical implementation, it is necessary that R and H can change with time. Unfortunately, in such cases, it can be shown that the computation of the optimal controller is cumbersome. We approximate the situation by assuming that past observations become exponentially irrelevant as a function of time. Under this approximation, we simply collect the exponentially smoothed moving averages of O _t , Q _t and apply a standard controller to these moving averages. Reflecting the idea that the situation should not be expected to be steady beyond 30 seconds, the exponential smoothing time constant was 30 seconds.

7.2 Qualitative assessment The purpose of the present invention was to make it possible to use contingencies as a reliable source of information. The contingency is reliable and the requirements for computation and bandwidth are very low. Because of the lack of quantitative evaluation, we will present a qualitative evaluation based on our experience of demonstrating the system at a public assembly. In a standard office environment where the level of noise is relatively high, the controller makes a decision after several tests to determine whether there is a reacting agent. Particularly effective is the transition point where an agent moves from talking to a robot to talking to someone else. The system was demonstrated at 4 scientific lectures and 2 conferences, namely ICDL04 and NIPS04. Demonstrations at lectures with relatively low noise levels generally work well. In ICDL04, it was relatively noisy like a poster room, and it took a little extra time for the controller to make a reliable decision. Overall, the level of performance was impressive, given the difficulty of the situation. In NIPS04, the conditions were extremely noisy. Loud conversations in many cases were not sufficient for mutual understanding. In order for the controller to perform reliably under these conditions, humans had to speak loudly and be near the robot.

  If this method is applied to humanoid robot and voice mho Dal, it can be used with camera input mho Dal.

  Combining this with the various expressive capabilities of the robot provides a device that allows a closer interaction between the robot and the user.

  In the basic invention part, the voice handled the Moo Dal as the input / output of the robot.

  As a result, the actuator output by the sensor input by the image input or the gesture output can be processed by the indirect corner control.

  Although it is only an example, if the movement for having the said fixed quantity is detected by the camera image input and optical defect calculation technique in the outside world, the sensor input 1 will input.

  In addition, a constant gesture output command that brought peace in advance is executed for output 1.

  As a result, the contingency event search by the image input mho Dal becomes effective. In entertainment robots, the elemental ability of expressions that does not tire the user is important.

8 Application Examples of the Invention As examples of application of the basic invention, two examples are shown below.

Application example 1
Gestures are observed and, further, the expression considers output, so that the gestures are imitated based on indirect corner control.

  There are possible methods for using the probability after the gesture is observed and calculated by the underlying invention in the degree of imitation.

  Robot indirect actuators are used, which can be observed, imitated, and output the human motion that is the subject of interaction so that much knowledge is gained from the camera image input.

  If it is observed and the degree of imitation can be numerically controlled, it can be considered to reflect this numerical value in the posterior probability calculated by the basic invention.

  For illustrative purposes, it is observed and human indirect corners are estimated from the image input as a method of imitation, and a method for controlling the robot indirect corners relative to the target angle is conceivable.

[Joint corner control value] = k1 × [posterior probability value] × [indirect corner value]
If the actuator is moved according to the above, the actuator can show a similar response in the robot after work, so that the probability value is such that the interaction between the user and the robot is accidental. A high value.

  The process of normal progress is observed, it is observed more than imitation, and imitation is dynamic and can be understood, it depends and can attract the user's interest .

  Here, k1 (also k2) is a parameter.

Application example 2
Changes in the facial expression of the robot can be considered as facial expressions.

  The facial expression mechanisms vary, but the simplest implementation is a method for changing the brightness of the eye LED in terms of parameters.

  The brightness of the LED is set as follows.

[LED brightness] = k2 × [probability value after work]
As a result, the user is at the forefront and the brightness of the eyes increases with interactions that are accidental.

  For the user, the dynamic change process of “this robot recognizes my existence slowly” can give a better impression.

  Of course, expressions other than LEDs are also possible.

  For example, there is a robot mechanism that changes the shape of eyebrows and lips by simple motion control.

  The control for showing a strong smile can be applied in a simple manner by a similar principle so that the established value is reflected in the motor output.

  For example, the application effect of the invention can bring about the characteristics of the change process used by using confidence and dynamic continuous quantities, for example, “output of human gestures about the pleasure of discovery” Compared to rule-based technology, people can be closer to action. In addition, humans can further interact with users.

9 Robot Device Mounted with the Present Invention The present invention can be mounted on, for example, a robot device as shown in FIG. The biped robot device 30 in FIG. 8 is a practical robot that assists human behavior in daily life and other situations. The robot device 30 is also an entertainment robot that can act according to the inner state (anger, sadness, joy, fun, etc.).

  As shown in FIG. 8, the robot apparatus 30 includes a head unit 32, a right / left arm unit 33 </ b> R / L, and a right / left leg unit 34 </ b> R / L coupled to a designated position of the torso unit 31. In these reference signs, the letters R and L are suffixes indicating right and left, respectively. The same applies to the following description.

  FIG. 9 schematically shows the structure of the degree of freedom of joint provided to the robot apparatus 30. The neck joint supporting the head unit 102 has three degrees of freedom, that is, a neck joint roll axis 101, a neck joint longitudinal axis 102, and a neck joint roll axis 103.

  Each of the arm units 33R / L constituting the upper limb includes a shoulder joint pitch axis 107, a shoulder joint roll axis 108, an upper arm roll axis 109, an elbow joint pitch axis 110, a forearm roll axis 111, A wrist joint pitch axis 112, a wrist joint roll axis 113, and a hand portion 114 are included. The hand portion 114 is actually an articulated multi-degree-of-freedom structure including a plurality of fingers. However, the movement of the hand portion 114 hardly affects the posture / walking control of the robot apparatus 1. For simplicity, this specification assumes that the degree of freedom of the hand portion 114 is zero. Therefore, the degree of freedom of each arm unit is 7.

  The trunk portion 2 has three degrees of freedom, that is, a trunk pitch axis 104, a trunk roll axis 105, and a trunk roll axis 106.

  Each of the leg units 34R / L constituting the lower limb includes a hip joint roll shaft 115, a hip joint pitch shaft 116, a hip joint roll shaft 117, a knee joint pitch shaft 118, an ankle joint pitch shaft 119, and a foot. A joint roll shaft 120 and a foot portion 121 are included. In the present specification, the intersection of the hip joint pitch axis 116 and the hip joint roll axis 117 is defined as the hip joint position of the robot apparatus 30. The human foot portion corresponding to the foot portion 121 has a structure including a multi-joint multi-degree-of-freedom sole. For the sake of simplicity, this specification assumes that the degree of freedom of the sole of the robot apparatus 30 is zero. Therefore, each leg unit has 6 degrees of freedom.

  In total, the degree of freedom of the entire robot apparatus 30 is 32 (3 + 7 × 2 + 3 + 6 × 2). However, the degree of freedom of the entertainment-oriented robot apparatus 30 is not limited to 32. It is obvious that the degree of freedom, ie the number of joints, can be increased or decreased according to design or production conditions, required specifications, etc.

  In practice, an actuator is used to realize each of the above degrees of freedom provided to the robotic device 30. In consideration of removing bulges that appear unnecessary to resemble a natural human body and providing posture control for an unstable bipedal structure, preferably a small and lightweight actuator is mainly used. . More preferably, a small AC servo actuator connected directly to a gear with a single chip servo control system mounted in the motor unit is used.

  FIG. 10 shows an outline of a control system configuration of the robot apparatus 30. As shown in FIG. 14, the control system includes an inference control module 200 and a motion control module 300. The inference control module 200 controls emotional identification and emotional expression in the form of dynamic responses to user input and the like. The motion control module 300 controls the harmonic motion of the whole body of the robot apparatus 1 such as driving of the actuator 350.

  The inference control module 220 includes a CPU (Central Processing Unit) 211, a RAM (Random Access Memory) 212, a ROM (Read Only Memory) 213 for executing calculation processes related to emotional identification and emotional expression, An external storage device (such as a hard disk drive) 214 is included. The inference control module 220 is an independent driven information processing unit capable of performing a process including all necessary elements inside the module.

  The inference control module 220 is supplied with image data from the image input device 251, audio data from the audio input device 252, and others. In response to these external stimuli, the inference control module 220 determines the current emotion or intention of the robot apparatus 30. The image input device 251 includes, for example, a plurality of CCD (charge coupled device) cameras. The voice input device 252 has, for example, a plurality of microphones.

  The inference control module 220 issues a command to the motion control module 300 to make a series of motions or actions based on decision making, that is, movement of the limbs.

  The motion control module 300 includes a CPU 311, a RAM 312, a ROM 313, and an external storage device (such as a hard disk drive) 314 for controlling the harmonic motion of the whole body of the robot device 30. The motion control module 300 is an independent driven information processing unit capable of performing a process including all necessary elements inside the module. The external storage device 314 can store, for example, an off-line calculated walking pattern, a target ZMP trajectory, and other action plans. ZMP is a floor surface point that generates a zero moment caused by the repulsive force of the floor during walking. The ZMP trajectory means a trajectory when the ZPM moves during the walking motion of the robot apparatus 30. For the concept of ZMP and the application of ZMP to the stability criteria for legged robots, see Miiomir Bukobratovic “LEGGED LOOMOTION ROBOT” (Japanese translation by Nikkan Kogyo Shimbun, Kato Ichiro et al. Leg ").

  The motion control module 300 includes an actuator for realizing each of the degrees of freedom distributed over the whole body of the robot apparatus 30 shown in FIG. 9, an attitude sensor 51 for measuring the attitude or inclination of the torso unit 2, Connected to landing confirmation sensors 352 and 353 for detecting whether the right and left soles are away from the floor or in contact with the floor, and a power controller 354 for managing a power source such as a battery. Yes. These devices are connected to the motion control module 300 through a bus interface (I / F) 301. The posture sensor 351 is, for example, a combination of an acceleration sensor and a gyro sensor. Landing sensors 352 and 353 include proximity sensors, micro switches, and the like.

  The inference control module 200 and the motion control module 300 are built on a common platform. The two modules are connected to each other by bus interfaces 201 and 301.

  The motion control module 300 controls the harmonic motion of the whole body by each of the actuators 350 in order to realize the behavior commanded by the inference control module 200. In response to the action commanded by the inference control module 200, the CPU 311 reads the corresponding operation pattern from the external storage device 314. Alternatively, the CPU 311 generates an operation pattern inside.

  In accordance with the determined operation pattern, the CPU 311 sets the motion of the foot portion, the ZMP trajectory, the motion of the trunk, the motion of the upper limb, the horizontal position and height of the waist, and the like. Next, the CPU 311 transfers the command value to the actuator 350. The command value determines the operation according to the setting contents.

  In order to detect the posture or inclination of the trunk unit 31 of the robot apparatus 30, the CPU 311 uses an output signal from the posture sensor 351. In addition, the CPU 311 uses the output signals from the landing confirmation sensors 352 and 353 to detect whether each of the leg units 5R / L is not used or standing. With this method, the CPU 311 can adaptively control the harmonic motion of the whole body of the robot apparatus 30.

  Further, the CPU 311 controls the posture and operation of the robot apparatus 30 so that the ZMP position always faces the center of the ZMP stabilization region.

  The motion control module 300 informs the inference control module 200 of the processing state, that is, to what extent the motion control module 300 has executed an action according to the judgment made by the inference control module 200.

  With this method, the robot apparatus 30 can determine the environment and the surrounding environment based on the control program, and can act autonomously.

  In the robot apparatus 30, for example, the ROM 213 of the inference control module 200 stores a program (including data) in order to implement the above-described image recognition function. In this case, the CPU 211 executes an image recognition program.

  Since the image recognition function is installed, the robot apparatus 30 can accurately extract the pre-stored model from the image data supplied through the image input apparatus 251. For example, when the robot apparatus 30 autonomously walks, it may be necessary to detect the intended model from the surrounding image recorded by the CCD camera of the image input apparatus 251. In this case, the model is often partially hidden by other obstacles. The viewpoint and brightness can be changed. Even in such a case, the image recognition technique described above can accurately extract the model.

  In the above embodiment, considering the certainty of the robot apparatus itself, the action output of the robot apparatus is determined so as to maximize the mutual information amount defined between the outside world and the own output.

10 Other Embodiments of the Present Invention In contrast, in the embodiment described below, the mutual information amount defined from the certainty that the interaction target is considered from the viewpoint of the robot apparatus is determined. The action output of the robot apparatus is determined so as to maximize it.

  This is based on the feelings of the interaction partner (equivalent to the concept called “Theory of Mind” in psychology). As described above, this is equivalent to performing an action stance opposite to the previous embodiment, in which action is acted on from this side (robot apparatus side). Hereinafter, as a specific example, a robot apparatus having a simple voice input / output apparatus will be described as an example.

  Here, consider a scene in which a robot apparatus interacts with an interaction target (such as a human being or another robot) by simple voice exchange.

  A robot apparatus 40 shown in FIG. 11 includes an audio input device 41, which is a microphone apparatus. Further, the sound input device 41 inputs a discrete value 1 when the volume of the outside world becomes a certain level or more. Otherwise, 0 is input every unit time. The audio output device 42 is a speaker device, and outputs an arbitrary sound (for example, a calling sound such as “Pirolo”) predetermined per unit time based on a command from the controller 43 when the control output value is 1, When the control output value is 0, silent output is performed.

  The controller 43 determines the control output at the current time step based on the input / output history up to the previous time step and sends it to the audio output device 42. In the following, the theoretical methodology of the mind as described at the beginning, that is, the setting method of the controller 43 that acts in a manner that is more favorable to the partner (see this scale later) by looking at the behavior of the other party. State.

  As shown in FIG. 12, if the robot apparatus 40 performs voice output “u1 = 1” at a certain time, the sensor input (estimated partner input) of the other party becomes “y2 = 1” regardless of the external noise. Think. Furthermore, if the other party performs the audio output “u2 = 1”, it becomes “y1 = 1” whatever the external noise.

  The robot apparatus 40 updates and holds the following five variables relating to the opponent's state every hour based on its own sensor input / output.

  Record how many steps have passed since the interaction target action “u2 = 1”. This value is specified as a parameter with the maximum value kappa and will not exceed this value. By using the time counter z (actually the value z (t−1) before the temporary point), it is determined based on the following rule whether the interaction target currently belongs to the Agent period or the Background period.

if z (t-1) <(kappa-1) then currently in Agent period
if z (t-1)> = (kappa-1) then currently in Background period
The time counter z itself is updated according to the following rules.

if u2 (t) = 1 then z (t) = 1
if u2 (t) = 0 then z (t) = z (t-1) +1, although z (t) = kappa if z (t-1) = kappa
Agent period means the timing at which a contingent interaction can occur, that is, if there is a response within this interval, the interaction at that time is contingent. The background period means the other period, that is, the period that misses the timing and is regarded as a reaction that is not contingent.

  In the following, the basic idea of Bayesian estimation based on the four defined variables is to compare the response histories in each of these two intervals, and as the response in the former (Agent period) is statistically larger, (In this case, the robot) is formulated in the framework of Bayesian estimation in the binomial distribution so as to increase the certainty of believing.

  The four variables are defined as follows.

tBG: Total number of background periods experienced so far
tAG: Total number of Agent periods experienced so far
sBG: Total number of tBG records of sensor 1 (y2 = 1 or noise N = 1)
sAG: Total number of tAG sensors 1 (y2 = 1 or noise N = 1). Actually, consider y2 = u1 u2 = u1. (Theoretical approach of mind)
By assuming the beta distribution for the above history variable group and noise distribution, interaction is obtained from the data (the above five variables) obtained from time to time based on an arbitrary prior distribution (prior distribution related to the parameters of the beta function). The certainty of the object can be calculated in the form of a probability value p2 (t). As in the previous embodiment, the technique used in this calculation process is Bayesian estimation, in particular, the use of the nature of natural conjugate distribution.

  Here, in this embodiment, in setting the controller 43 to behave so as to be in a favorable situation for the interaction target, the following two methods can be considered as measures of preference.

  Scale 1: A method using the probability value p2 as it is.

  In this case, the higher the probability value is, the more preferable it is. That is, it is a measure that the interaction object is more preferable to him as the certainty of the interaction object (human or robot) in front of him (as seen from him) is certain.

  Scale 2: A method using entropy p2 * log (p2) + (1-p2) * log (1-p2) defined from the probability value p2.

  In this case, as described in the following reference, it is equivalent to the information amount maximization norm, but this meaning in the present invention is a measure that is preferable as the acquired information amount of the interaction target is maximized. In the following reference, although it is another problem setting, when the controller setting based on the information amount maximization norm is performed, it is reported that the behavior of the controller becomes close to human.

Reference: “An Infomax Controller for Real-Time Detection of Social Contingency”, Javier R. Movellan, Proc. Of the 4th IEEE International Conference on Development and Learning 2005
When any one of the scales listed here is regarded as a reward value, a controller that maximizes the expected earned reward value in the future can be obtained by an existing reinforcement learning method (for example, Q-learning). This method can be performed by computer simulation, and if the optimum controller finally obtained is copied to the controller 43 of the robot apparatus 40, the intended one can be realized.

  In order to perform reinforcement learning, it is necessary to define the state in addition to the reward value. In this case, the state is nothing but the set of five variables defined above. If the robot behavior value u1 is determined by random selection and a number of computer simulations are performed by Q-learning, an optimal controller based on the acquired Q value can be obtained.

  As described above, the controller 43 of the robot apparatus 40 that behaves favorably for the interaction target can be realized.

  The invention is not limited to the embodiments described above with reference to the accompanying drawings. Further, those skilled in the art will recognize that various changes and substitutions and equivalents can be made without departing from the spirit or scope of the appended claims.

11 Appendix I: Model Summary Parameters:

  Static random variables:

Stochastic process:
t = 1, 2,. . . The following process is defined for

Stochastic constraints:
FIG. 4 shows Markov constraints in the joint distribution of different variables included in the model.

12 Appendix II: Solution of Bellman's Optimality Formula This appendix assumes the notation and components presented in Section 3.4. For every t′ε [t + 1, T], let S _{t ′} = ^def (O _{t ′} , Q _{t ′} , Z _{t ′} ), and (y _{1: t ′} , u _{1: t} , o _{t ′} , q _{Let t '} , zt _' ) be a fixed arbitrary sample from (Y1 _{: t '} , U1 _{: t} , _Ot' , _{Qt '} , _Zt' ). First, we show that the following characteristics are satisfied for t ′> t.

(1) There exists a function g _{t ′} such that S _{t ′} = g _{t ′} (S _t′−1 , Y _{t ′} , U _{t ′} ).

(2) p (y _{t ′ + 1} , | y _{1: t ′} , u _{1: t ′ + 1} ) = p (y _{t ′ + 1} , | s _{t ′} , u _{1: t ′ + 1} )
(3) W _{t ′} ∈σ {S _{t ′} }
Characteristics (1) is _{S t Y} t and _{'S t'-1} by regression _method' ', _{U t'} merely describes a simply can be calculated from, Assuming the definition of _{S t ',} This is clearly true. Regarding characteristic (2),

Please pay attention to. Here, I _{t ′} is a three-dimensional binary vector having a single 1 whose position is determined by Z _{t ′} . Accordingly, E [R · I _{t ′ + 1} | o _{t ′} , q _{t ′} , z _{t ′} , u _{t ′ + 1} , h] is an expected value of the speed of the Poisson process selected by z _{t ′} . From (15), for i = 1, 2, 3, the following equation is obtained:

  Thus, using (86), the following equation is obtained:

Therefore, the expected distribution at time t ′ + 1 is a function of S _{t ′} and U _{t ′ + 1} , and (2) is obtained from this. Characteristic (3) is obtained directly from (24).

  These characteristics are then used to derive an algorithm for finding the optimal infomax controller. First, the characteristics (1) and (3) are combined to obtain the following equation.

  By using this fact in combination with (1), the following equation is obtained.

  Next, it is assumed that t ′ = T. Since there is no return after T, the following equation is obtained.

Thus, for the case where t ′ = T, it is clear that the value of the sequence (y _{1: t ′} , u _{1: t ′} ) can be calculated as a function of its associated statistical value st ′.

For the case where t ′ = T−1, the functions F ′, N ′, V ′, C ′ for calculating F, N, V, C of the sequence (y _{1: t ′} , u _{1: t ′} ) are It is defined on the basis of the statistical value s _{t ′} of the numerical sequence.

  The same logic can be used for the case where t '= T-2.

Here, s _{t ′ + 1} = ^def g _{t ′} (s _{t ′} , y _{t ′ + 1} , u _{t ′ + 1} ). Steps (78)-(83) can be optimally applied for the case where t '= T-2, T-3,... T, thus restoring the optimal controller. Note that for each time t′ε [t, T], the controller maps the value of the statistic s _{t ′} to the action U _{t ′} .

13 Appendix III: Definitions Beta variables:

  The following formula provides a parameter for the beta distribution that matches the desired mean m with the variance s2.

  Gamma function

  The gamma function has the following characteristics.

  Logistic function

  Expected value when a sigma aggregate is given: Y is an integer random variable on the probability space (Ω, F, P), ie, E (| Y |) ∈R, and σ∈F is sigma Assume it is an algebra. The conditional expectation for a given Y is a P semi-certain eigenrandom variable with the following characteristics:

  E (Y | σ) is σ measurable.

For any _{A∈σ, ∫ A E (Y |} σ) dP = ∫ A YdP.

For E (Y ² ) εR, E (Y | σ) is a P quasi-certain eigen-sigma measurable random variable closest to Y in the least-squares sense, ie:

  Let X be a random variable in the same probability space as Y, and σ (X) be a sigma algebra induced by X. The expected value of Y when X is given is the expected value of Y when σ (X) is given, that is, as follows.

  entropy:

  Conditional entropy:

  Mutual information:

(A) is the schematic of the head of the robot used in the nonpatent literature 19, (B) is a photograph of the infant-9. You can see the robot image in the mirror behind the baby. It is a figure which shows the structure of the social robot provided only with the required minimum function. It is a graphical representation of the dynamics of timers and indicator variables. The generation model is a graph display. (A) and (B) are diagrams of two contingent event clusters created by the model. (A) is a raster diagram of 150 tests, and (B) is a diagram showing the probability that the voice sensor is operating as a function of time. (A) is a diagram showing the response of an infomax controller that simulates an infant, (B) is a diagram showing the posterior probability when there is a reacting agent as a function of time, and (C) is 43 seconds. The figure which shows the posterior distribution of a subsequent agent speed and background speed, (D) is a figure which shows the ratio of the uncertainty regarding the reaction speed of an agent, and the uncertainty regarding the background reaction speed. It is the perspective view which showed the external shape of the robot apparatus by this embodiment. It is the schematic of the freedom degree structure about a robot apparatus. It is the figure which showed the system configuration | structure of the robot apparatus. It is the figure which showed the other system structure of the robot apparatus which concerns on this invention. It is a figure with which it uses for description of robot apparatus field operation | movement shown in FIG.

Explanation of symbols

  30, 40 Robot device, 41 Audio input device, 42 Audio output device, 43 Controller, 251 Image input device, 252 Audio input device, 253 Audio output device, 254 Communication interface, 211 CPU, 212 RAM, 213 ROM, 214 External Storage device, 201 bus interface, 311 CPU, 312 RAM, 313 ROM, 314 External storage device, 301 bus interface

Claims (8)

  An interaction device that sets a self-controller to maximize the expectation of information defined between hypotheses about the interaction object and self-input / output.   The interaction device according to claim 1, wherein the hypothesis is whether or not an interaction target exists.   The interaction device according to claim 1, wherein the interaction target is a user.   2. The interaction device according to claim 1, wherein the input / output is an audio microphone input / loud speaker output.   2. The interaction device according to claim 1, wherein the interaction device includes an expression mechanism and outputs an expression together with an a posteriori probability of whether an interaction target exists.   The interaction device according to claim 1, wherein the expression mechanism is a mimic motion output mechanism.   An interaction apparatus comprising: a control unit that outputs an action at a timing at which an expected acquisition information amount for the presence of an interaction object is maximized based on input / output information.   8. The interaction apparatus according to claim 7, further comprising an audio microphone and a loudspeaker, wherein the input / output information is audio information.

Images