JP2022154041A - Subject feeling estimation model, device and method, and behavior modification promotion model - Google Patents

Abstract

To provide a subject feeling estimation model capable of estimating a subject feeling of a user.SOLUTION: This model makes a computer function as: a belief model for generating or updating belief information including a probability of causing a new state as a result of a behavior under a certain subject feeling level of a user; a desire model for receiving a reward corresponding to value information of the user to generate a desirable state of the user and determining a policy being a set of behaviors that can bring about the desirable state; an intention model for generating causal relation information relating to causal relations among a state, a value, a reword and a behavior on the basis of the belief information, the value information and the reward; a behavior model for determining and outputting a behavior on the basis of the policy and the causal relation information; and a subject feeling model for determining or updating the subject feeling level of the user on the basis of a generated new state and a feature amount relating to a prescribed feature of the user, and updating the subject feeling level to be used in the belief model into a determined or updated subject feeling level.SELECTED DRAWING: Figure 1

Description

TECHNICAL FIELD The present invention relates to persuasive technology that promotes behavioral change of people through "persuasion".

Attention is focused on persuasive technology that encourages behavioral change of the user by means of "persuasion" in a broad sense that can change the user's beliefs, perceptions, and desires in a predetermined direction. For example, in the fields of health and welfare, education, and urban transportation, the application of this persuasive technique is vigorous, as disclosed, for example, in Patent Document 1, Patent Document 2, and Patent Document 3, respectively. is being advanced to.

In addition, persuasive technology will continue to develop in the future as an indispensable technology for realizing autonomous AI systems that provide various services to users through AI (Artificial Intelligence) communication with users. It is thought that there are many.

Rita Orji and Karyn Moffatt, "Persuasive technology for health and wellness: State-of-the-art and emerging trends", Health Informatics J. 24(1), pp.66-91. 2018, <https://doi .org/10.1177/1460458216650979> Yohana Dewi Lulu Widyasari et al., "Persuasive technology for enhanced learning behavior in higher education", International Journal of Educational Technology in Higher Education, 16:15, 2019, <https://doi.org/10.1186/s41239-019 -0142-5> Evangelia Anagnostopoulou et al., "Persuasive Technologies for Sustainable Mobility: State of the Art and Emerging Trends", Sustainability 2018, 10(7), pp.2128, 2018, <https://doi.org/10.3390/su10072128>

However, with conventional persuasive techniques such as those described above, even if AI “persuades” the user, it is difficult for the user to make more appropriate behavioral changes in the situation of a normal environmental world that is complicated or changes from moment to moment. The reality is that it is still difficult to encourage, more specifically, to encourage more appropriate decision-making and action-making.

The inventors of the present application believe that the reason for this difficulty lies in the fact that the user himself/herself is connected or interlocked between the actions he/she takes on his/her own will after being "persuaded" by AI and the state of the environmental world that appears as a result. I thought that there were many cases where I could not feel sexuality. In other words, conventional persuasive techniques do not take into account changes in the user's "sense of initiative" (the feeling that his or her own actions have an impact on the surroundings). We have found that it is extremely difficult to perform appropriate "persuasion" that is satisfactory to the user, without the

Accordingly, the present invention provides a sense of subjectivity estimation model, a sense of subjectivity estimation device, and a sense of subjectivity that can estimate a user's sense of agency and, in consideration of the sense of agency, encourage changes in the user's intentions or actions. The purpose is to provide an estimation method and a behavioral change promotion model.

According to the present invention, a sense of subjectivity estimation model that causes a computer to function to estimate a sense of subjectivity of a user while determining behavior with respect to the state of the environment world including the user using a reward, comprising:
a belief model for generating or updating belief information, which is information including the probability that a new state will occur as a result of performing a certain action on a certain state under a certain sense of subjectivity level of the user;
Receive value information related to value for the user and the corresponding reward, generate a desired state desired by the user, and determine a policy that is a set of actions that can bring about the desired state. Desire model and
an intention model that generates causal relationship information relating to causal relationships between states, values, rewards, and actions based on the belief information, the value information, and the reward;
an action model that determines and outputs an action to be taken for an observed state based on the policy and the causal relationship information;
determining or updating and outputting the sense of subjectivity level of the user based on the output new state caused by the action and the feature amount related to the predetermined feature of the user under the new state; A sense of subjectivity estimation model is provided that causes a computer to function as a sense of subjectivity model that updates the sense of subjectivity level used in the belief model to the determined or updated sense of subjectivity level.

As an embodiment of the sense of subjectivity estimation model according to the present invention, the behavior model is:
an action planning unit that generates an optimal policy based on the policy, the causal relationship information, and the contents of communication including a predetermined question made with the user;
It is also preferable to have an action determination unit that determines and outputs an action to be taken for the observed state using the generated optimal policy.

Furthermore, as another embodiment of the sense of subjectivity estimation model according to the present invention, the sense of subjectivity estimation model receives information related to the value information and information related to reward corresponding to the information related to the value information from the user. Then, based on the information on the value information and the information on the remuneration, generate or update the value information and the remuneration corresponding thereto, and cause the computer to further function as a value matching model that outputs to the above desire model. is also preferred. Here, it is also preferable that this value matching model is constructed using an algorithm related to cooperative inverse reinforcement learning (CIRL).

Furthermore, as yet another embodiment of the sense of subjectivity estimation model according to the present invention, the sense of subjectivity estimation model is:
Receiving an observed state and a corresponding output action, and selecting an alternative state as a possible state candidate based on integrated causal relationship information that integrates the causal relationship information for each of at least a plurality of users. a state generator that generates and outputs;
Generate a loss representing the difference from the desired state from the new state generated by the output action and the alternative state, train the state generator with the loss, and use the loss a classifier that trains itself;
an evaluator that generates a predicted reward that is a reward corresponding to the alternative state generated by the trained state generator, and that trains a behavior model to determine the behavior with the predicted reward. It is also preferable to have a computer function additionally as a generating and evaluating model. It is also preferable that this alternative state generation/evaluation model is constructed using an algorithm related to generative adversarial networks (GAN).

Furthermore, in the sense of subjectivity estimation model according to the present invention, the belief model is preferably constructed using an algorithm related to the Partially Observable Markov Decision Process (POMDP). Also, the causal relationship information in the intention model is preferably information related to a Bayesian network algorithm.

According to the present invention, there is also provided a sense of subjectivity estimation device that estimates the sense of subjectivity of the user from the state observed in the environment world using the sense of subjectivity estimation model described above.

According to the present invention, there is further provided a sense of subjectivity estimation method in a computer for estimating a sense of subjectivity of a user while determining behavior with respect to the state of the environment world including the user using a reward, comprising:
a step of generating or updating belief information, which is information including the probability that a new state will occur as a result of performing a certain action on a certain state under a certain sense of subjectivity level of the user;
Receive value information related to value for the user and the corresponding reward, generate a desired state desired by the user, and determine a policy that is a set of actions that can bring about the desired state. a step;
generating causal relationship information relating to causal relationships between states, values, rewards and actions based on the belief information, the value information and the reward;
determining and outputting an action to be taken for the observed state based on the policy and the causality information;
determining or updating and outputting the sense of subjectivity level of the user based on the output new state caused by the action and the feature amount related to the predetermined feature of the user under the new state; and a sense of subjectivity model for updating the sense of subjectivity level used in the step of generating or updating the belief information to the determined or updated sense of subjectivity level.

According to the present invention, there is furthermore a behavior modification promotion model that functions a computer that encourages behavior modification of a user while determining behavior with respect to the state of the environment world including the user using a reward,
a belief model for generating or updating belief information, which is information including the probability that a new state will occur as a result of performing a certain action on a certain state under a certain sense of subjectivity level of the user;
Receive value information related to value for the user and the corresponding reward, generate a desired state desired by the user, and determine a policy that is a set of actions that can bring about the desired state. Desire model and
an intention model that generates causal relationship information relating to causal relationships between states, values, rewards, and actions based on the belief information, the value information, and the reward;
Based on the policy, the causal relationship information, and the contents of communication including the predetermined question with the user, an optimal policy is generated, and using the optimal policy, an action model that determines and outputs an action to be taken for an observed state;
determining or updating the sense of subjectivity level of the user based on the output new state caused by the action and the feature amount related to the predetermined feature of the user under the new state; A behavioral change promotion model is provided that causes a computer to function as a sense of subjectivity model that updates the sense of subjectivity level used in the model to the determined or updated sense of subjectivity level.

According to the subjective feeling estimation model, the subjective feeling estimating device, the subjective feeling estimating method, and the behavior change promotion model of the present invention, the user's sense of agency is estimated, and the user's intention or It can encourage change in behavior.

1 is a schematic diagram showing an embodiment of a sense of subjectivity estimation model according to the present invention, and a functional block diagram showing a functional configuration in an embodiment of a sense of subjectivity estimation device according to the present invention; FIG. 1 is a schematic diagram for explaining a theoretical foundation system in one embodiment of the present invention; FIG.

BEST MODE FOR CARRYING OUT THE INVENTION Hereinafter, embodiments of the present invention will be described in detail with reference to the drawings.

[Sense of agency estimation model, behavioral change promotion model]
FIG. 1 is a schematic diagram showing an embodiment of a subjective feeling estimation model according to the present invention, and a functional block diagram showing a functional configuration in an embodiment of a subjective feeling estimation device according to the present invention.

The sense of subjectivity estimation model 1 of this embodiment shown in FIG. , and the user H can act on the environmental world to change the state of the environmental world toward a desired state.

Here, the sense of agency estimation model 1 acquires information (for example, physiological indicators such as heart rate and breathing rate) related to the user's predetermined characteristics through the communication described above, and calculates the "sense of agency" of the user H. (the sense that one's actions have an impact on the surroundings; hereinafter abbreviated as sense of agency) is quantified. Furthermore, while communicating with the user H, this "sense of subjectivity" is appropriately updated in the cycle of formulating policies and taking actions in response to the state of the environmental world. Further, for example, in this embodiment, it is also possible to determine and output the user H's real-time "sense of subjectivity" and the dynamic variation of the user H's "sense of subjectivity".

Furthermore, in this embodiment, the sense of subjectivity estimation model 1 communicates appropriately with the user H and maintains and improves the "sense of subjectivity" of the user H even in a complex and ever-changing environmental world. It is also a behavior modification promotion model capable of promoting behavior modification in user H, specifically, more appropriate decision-making and action determination, while (without diminishing).

Specifically, the sense of subjectivity estimation model (behavior change promotion model) 1 determines the "behavior" for the state of the environment world including the user H using the "reward" calculated based on the state and the behavior. is a model for estimating user H's sense of agency, and as shown in FIG. 1, at least
(A) A belief that generates or updates "belief information", which is information including the probability that a new state will occur as a result of performing a certain action in a certain state under a certain "subjectivity level" of the user H. model 11;
(B) Receive "value information" related to value for user H and corresponding "reward", and generate a desired state desired by user H based on the "value information" and "reward" and a desire model 12 that determines a "policy" that is a set of actions that can bring about the desired state;
(C) an intention model 14 that receives "belief information,""valueinformation," and "rewards," and generates "causal relationship information" relating to causal relationships between states, value information, rewards, and behavior;
(D) an action model 15 that determines and outputs an "action" to be performed for an observed state based on the "policy" and "causal relationship information";
(E) A "new state" caused by the output "behavior" and a "feature amount" (for example, heart rate, breathing rate, etc.) related to a predetermined feature of the user H under the "new state" ), determines or updates the user H's "subjectivity level" based on the "new state" and the "feature amount", and outputs the above (A) This model causes the computer to function as a subjective feeling model 17 that updates the "subjective feeling level" used in the belief model 11 to the determined or updated "subjective feeling level".

In this way, the sense of subjectivity estimation model 1 can estimate the "sense of subjectivity," which has been difficult to estimate in the past (in particular, it was impossible to estimate in a complex and ever-changing environmental world). Furthermore, by taking into consideration the user H's "sense of subjectivity (level)" that has been appropriately updated and determining his or her behavior in relation to the environment, it is possible to encourage the user H to change his/her behavior, that is, to change his/her intention or behavior. . For example, according to the sense of subjectivity estimation model 1, user H's voluntary behavior (in response to a proposal or persuasion from the sense of subjectivity estimation model 1) and the state of the environmental world that has appeared as a result. In the meantime, it is possible to make appropriate proposals and persuasion, etc. so that the user H himself can feel the connection and interlocking, that is, the "sense of independence" of the user H can be improved and not diminished. .

Next, with reference to FIG. 2, the theoretical basis of the functional configuration in the sense of subjectivity estimation model (behavioral change promotion model) 1 of this embodiment will be described.

FIG. 2 is a schematic diagram for explaining the theoretical foundation system in one embodiment of the present invention.

In general, the user's sense of agency (SoA) is determined from the occurrence of disturbing events that interrupt the good flow of the user's intentional actions (towards a predicted state in the environmental world). change under strong influence. How the sense of subjectivity level changes in the chain of state-will-behavior-new state will be described below with reference to FIG.

First, the belief-desire-intention model (Georgeff et al., 5th International Workshop, ATAL'98 Proceedings, pp.1-10, 1998) is known as a model for deriving future intentions. In this model, 'belief' (Fig. 2) and 'desire' (Fig. 2) are the perceived state structure of the environmental world and the desired/desired state of the environmental world, respectively. It is the stored information about the structure.

In addition, "will" (Fig. 2) is determined from "belief" and "desire", and specifically, it is information that includes actions assumed to bring about the desired state structure of the environmental world. . In this way, "intention" specifies and controls behavior, but the specification and control of such behavior are updated under the influence of changed "belief" and "desire".

Here, the sense of subjectivity is a feeling that is determined by whether or not the "intention" to achieve the desired state structure is embodied in the "action" (Fig. 2). will be given by the "action plan/choice" (Fig. 2) that received ".

The inventors of the present application have integrated the above-mentioned knowledge of "intention" with conventional theories of cognitive science and neuroscience concerning the emergence and cessation of a sense of agency, and devised a basic system as shown in Figure 2. be.

Here, in cognitive science and neuroscience, the validity of a model using a "comparator" (Fig. 2) for behavioral cognition has been discussed, while the theory of motor learning control exists. According to this "comparator" model, motor drive (action) is performed with a prediction of the motor output produced based on the efferent copy of the command to the motor. Next, in the "comparator", the predicted result and the actually measured motor output are compared, and if they match, the motor output is recorded as being caused by the driving (behavior) of the motor itself. be. On the other hand, if they do not match, it is assumed that the "sense of subjectivity" in the sense of controlling the motor drive has been interrupted or disrupted.

In contrast, the theory of retrospective inference (RI) in cognitive science and neuroscience states that if the intended or predicted state and the observed actual state match, a sense of subjectivity arises. estimation” (Fig. 2). In addition to "intention", other higher-order (relevant to cognition) factors, such as clues related to the context of the external world and social situations, are also considered in the "estimation" of the sense of subjectivity. Concretely, in this theory, if the observed state occurs as expected, the action will be carried out smoothly, and the idea of action and attitude will be put in the corner of consciousness. On the other hand, if the observed state differs from the prediction, the brain performs retrospective inference (RI), asking for an answer as to whether the behavior caused the observed state.

A specific configuration of the subjective feeling estimation model (behavioral change promotion model) 1 as an embodiment of the present invention, which is a computer implementation of the theoretical basic system described above, will now be described in detail.

By the way, normally, the human brain holds a mental model that expresses the mind of another person, and uses it to process the mental state of the other person. In the so-called Theory of Mind (ToM), which is a theory of cognitive science about this processing ability, people retain such processing ability, and therefore how others behave in various contexts and You can intuitively get a sense of why it behaves the way it does.

Here, the sense of subjectivity estimation model (behavioral change promotion model) 1 described below emulates this theory of mind (ToM), so to speak, and includes "belief", "desire", "intention", and "action plan/choice". and "estimation" is attributed to another person (user H in this embodiment), why the other person (user H) behaves like that, and how to perceive the state of the environment world as a result of the action In other words, it is necessary to understand how the other person (user H) has a sense of subjectivity and how it works.

[Model configuration, subjective feeling estimation method]
Hereinafter, the functional configuration in one embodiment of the sense of subjectivity estimation model (behavior change promotion model) 1 according to the present invention will be described in more detail. Similarly, according to FIG. 1, the sense of subjectivity estimation model (behavior change promotion model) 1 is
(A) Belief model 11, Desire model 12, Value matching model 13, Intention model 14,
(b) an action model 15 including an action planning unit 151 and an action determining unit 152;
(c) an alternative state generation/evaluation model 16 including a CBN (Causal Bayesian Network) aggregate 161, a state generator 162, a classifier 163, and an evaluator 164;
(d) The subjective feeling model 17 is provided as a functional component embodied by (a program installed in) a computer. Hereinafter, each functional configuration unit described above will be specifically described.

<Belief model>
Also in FIG. 1, the belief model 11 assumes that a new state s' is created as a result of performing a certain action a in a certain state s under a certain "subjectivity level" soa of the user H. It is a model that generates or updates "belief information", which is information containing the probability of occurrence. In this embodiment, this belief model 11 is constructed using an algorithm related to the Partially Observable Markov Decision Process (POMDP) (MONAHAN GE Management Science 28(1), 1-16, 1982). .

Specifically, Belief Model 11 is
(a) Let state space S be a set of possible states s of the environment world including user H and sense of subjectivity estimation model 1,
(b) Assume that a set of actions a that can be performed (output) by the user H and the sense of subjectivity estimation model 1 is an action space A;
(c) Let transition probability T(s'|s, a) be the conditional probability that a transition to state s' occurs when action a is performed in state s,
(d) Let the cost of performing action a in state s be c = c(s, a),
(e) Observation probability O(o|s', a) is the probability that subjective feeling estimation model 1 obtains observation o from the environment world when a transition to state s' occurs when action a is performed in state s. As
The sense of subjectivity estimation model 1, which performed action a in state s, derives belief B(s'), which is the probability that the environmental world takes state s', as "belief information" when observation o is obtained.

More specifically, if the belief at the previous point in time is B(s) and β = 1/Prob(o|b, a) is the normalization constant, then the current update result of B(s) is ) Belief B(s') is expressed by the following formula (1) B(s')=β・O(o|s', a)・_Σ
can be calculated by This is information that can be taken as the degree of "belief/confidence" about how likely the states of the environment are to occur.

Here, the "subjectivity level" soa is an element of the state space S as a state that the user H can take, and the belief B(s') is derived from the subjectivity model 17 described later. The updated value is obtained by taking the sum (Σ _s∈S in the above equation (1)) for {s} including the received updated “subjectivity level” soa′.

By the way, assuming that the user H is the driver of a car traveling on the road X, and the environment world is the road traffic condition including the user H's car and the environment around the road network, the state is, for example, "towards the destination W "The traffic condition at the position of the user H on the road X is 'normal'", "The traffic condition ahead of the road X is 'congested'", "The weather is 'clear'", . can be done. Also, the "subjectivity level" may be set to five levels, for example, "high", "a little high", "neutral", "a little low", and "low".

Here, in the entire sense of subjectivity estimation model 1 including this belief model 11, unlike usual, the sense of subjectivity estimation model 1 and the user H are not completely separate entities. Processing is performed with the expectation that user H will cooperate. In addition, the subjective feeling estimation model 1 does not adopt the reward (function) used in ordinary POMDP in this belief model 11, but instead uses the reward r is adopted.

<Desire model>
Also in FIG. 1, the desire (desire, desire) model 12 receives "value information" related to the value v for the user and the corresponding reward r, and based on the "value information" and the reward r, the user's It is a model that generates a desired state sd (εS), which is a desired state, and determines a policy π that is a set of actions a that can bring about the desired state sd.

In this embodiment, the desire model 12 is specifically configured with a deep neural network (DNN, Deep Neural Networks) algorithm,
(a) A set of values v (for example, social values) for user H, τ=<v1, v2, . degree of affection, degree of sense of subjectivity of others> and
(b) A desired state sd (εS ), the policy π(o)=<a1, a2, . to determine an>.

Here, both the set τ of values v in (a) above and the reward r in (b) above are given or indicated by the user H in interaction with the subjective feeling estimation model 1. ing. Incidentally, such a set τ of values v is generated by the value matching model 13 described later.

<Value integration model>
Generally speaking, people often misrepresent or misrepresent to others what they really want in a given situation. This poses a big problem when trying to convey one's wishes to AI and have it act as expected.

For example, people expect AI robots to brew coffee for themselves (people) rather than enjoying coffee themselves (the value of the correct answer). People will also want self-driving cars to recognize what values are important to them while driving. For example, we expect them to recognize the essential values of complying with traffic rules, staying away from pedestrians, and not getting stuck in traffic jams when children are driving. However, it has been extremely difficult for AI to accurately recognize such human desires, in other words, to match (align) the value for humans and the value that AI incorporates.

For example, AI that performs general reinforcement learning (RL) learns the optimal policy by maximizing the cumulative reward that is added by discounting the farther future reward. The rewards given are merely mathematical abstract quantities, not intrinsic to the real environment. In addition, we form a model that answers questions such as what values should be considered and why certain values are important, not as abstract quantities in mathematics, but as actually obtained quantities. is very difficult to do.

Conventionally, attempts have also been made to solve this value matching problem using Inverse Reinforcement Learning (IRL) (Andrew Ng and Stuart J Russell, ICML 2000: Proceedings of the Seventeenth International Conference). However, humans do not want AI to understand all of their wishes (for example, personal wishes such as enjoying coffee time do not need to be understood), and AI needs to distinguish and process them. It was very difficult and a problem had arisen. Furthermore, IRL assumes that the observed human behavior and attitudes are optimized, and makes adjustments using various useful information contained in the observed human behavior and attitudes. I couldn't.

Therefore, the value matching model 13 according to the present invention uses Cooperative Inverse Reinforcement Learning (CIRL) (Dylan Hadfield-Menell et al., 30th Conference on Neural It is constructed using algorithms related to Information Processing Systems (NIPS) 2016).

Specifically, the value matching model 13, as part of the CIRL processing flow, from the user,
(a) Information related to "value information", in this embodiment, a set τ=<v1, v2 of value v for user H (value v recognized by user H as being important (for example, in social life)) , ..., vn>, that is, information about each value v1, v2, ..., vn;
(b) receive information related to the reward r corresponding to the information related to the "value information", in this embodiment information about the reward r of each value v1, v2, ..., vn; Based on the information of (b), the model generates or updates the “value information” and the reward r corresponding thereto, and outputs them to the desire model 12 .

Here, the value matching model 13 (as the sense of subjectivity estimation model 1) is
(a) Initially, the user H personally recognizes the value, in this embodiment, does not clearly recognize the value set τ = <v1, v2, ..., vn>,
(b) Observation related to value set τ for user H via an interface (IF) capable of inputting measurement results output from measuring means such as cameras, or capable of text input/output, voice input/output, etc. or make an inquiry,
(c) Based on the premise that "users usually act based on value set τ", information on value set τ (for example, user's actions, attitudes, answers, and rewards and information related to the results),
(d) Based on the collected information, value set/reward generation processing is performed with the aim of maximizing the estimated value set τ.

Incidentally, in order to determine a more consistent value set τ, it is also preferable that the above-described interaction between the sense of subjectivity estimation model 1 (value matching model 13) and the user H is continuously repeated.

Also, in this embodiment, the value matching model 13 makes an inquiry/request to, for example, the user H via the above interface (IF), and based on the content of the response, a self-report (self-report) related to the subjectivity of the user H. ) soa_r and output to the subjective feeling model 17, which will be described later in detail.

<Intention model>

The belief-desire-intention model (Georgeff et al., 5th International Workshop, ATAL'98 Proceedings, pp.1-10, 1998) is known as a model for deriving future intentions. Here, intention is expressed as an action that would cause a desired state (result) in the environmental world. However, in deriving the sense of subjectivity level soa of the user H, there is the problem of how the sense of subjectivity estimation model 1 (or the user H) grasps the relationship between the cause (behavior) and the result (state), that is, the causal relationship. must be resolved.

In order to solve this problem, the intention model 14 in FIG.
(a) belief information received from the belief model 11;
(b) based on the value set τ (value information) and corresponding rewards received from the desire model 12, between states, values, rewards, and actions (in this embodiment, the costs also included in the belief information); It is a model that generates "causal relationship information" related to causal relationships.

In this embodiment, the intention model 14 is constructed using a Causal Bayesian Network (CBN) algorithm. In this embodiment, the output "causal relationship information" is information related to the CBN algorithm, specifically the configured CBN configuration information itself (see the lower left side of FIG. 1). The information includes the conditional probability of the resulting state s' given the state s and the action a.

where CBN is a directed acyclic graph model, a set of nodes {Y _i } with directed edges E identified by the parent function set {pa(Y _i )}, and a set of conditional probabilities {Prob (Y _i |pa(Y _i )}, where each node Y _i corresponds to one of state, action, cost, value, and reward. indicates that there is a possibility of a causal transition between Y _i and Y _j connected by it, specifically Y _i causes Y _j with some probability, in other words and Y _j are represented by a conditional probability distribution with Y _i as a condition.

Furthermore, in this embodiment, the CBN defines a do operator: do(Y _j =y _j ). When this do operator is applied to the CBN, pa(Y _i )=Φ and Prob(Y _i )=δ(Y _i ;y _i ). In other words, the do operator is an operator corresponding to the "intervention (in behavior change theory)" implemented by the sense of subjectivity estimation model 1. FIG.

In addition, the intention model 14 initially estimates the causal relationship of the environmental world through "intervention", but in the end, "what would have happened if a different "intervention" had been implemented in the past stage?" It becomes necessary to answer counterfactual and hypothetical questions such as Therefore, in this embodiment, the intention model 14 can also take a "counterfactual virtual mode."

As explained above, the intention model 14 is
・It is said that human intentions are formed based on the idea of ``how are the events in the environment connected by causal relationships?'' and expectations of ``what will be brought about as a result of the intended action.'' , is constructed based on a new hypothesis set by the inventors of the present application.

That is, the intention model 14 is the function of the "comparator" and "estimation" in the theoretical basis system of the present invention shown in FIG. It incorporates the function of judging and estimating whether it is in a state of being Incidentally, the conditional probability of the resulting state s' when the state s and the action a are given, that is, how high the probability of the state s' occurring is information included in the output of the intention model 14. is a reflection of the correspondence with the intended state.

Furthermore, the intention model 14 stores information regarding whether or not the state assumed by the user H matches the actual state via the belief information received from the belief model 11, that is, information regarding the sense of subjectivity level soa. is reflected.

<Action model>
Also in FIG. 1, the behavioral model 15 is:
(a) Policy π = <a1, a2, ..., an> received from Desire Model 12, and
(b) causal relationship information (CBN configuration information) received from the intention model 14
Based on the above, the action a to be performed for the observed state s is determined and output.

Of these, the action planning section 151
The policy π of the above (a), the causal relationship information (CBN configuration information) of the above (b), and further,
(c) Through an interface (IF) capable of inputting measurement results output from measuring means such as a camera, or capable of text input/output, voice input/output, etc., a predetermined question made between User H Optimal policy π*, which is the policy considered optimal, is generated based on the content of the communication including.

In this embodiment, the action planning unit 151 uses a known XAIP (eXplainable AI Planning agent) (Chakraborti et al., Proceedings of the Twenty-Eighth International Joint Conference on Artificial Intelligence IJCAI-19, pp.1335-1343, 2019) It is constructed using the action decision processing in Here, in XAIP, an action plan problem P, a transition function ζ _P : S×A→S×C, and an action plan algorithm A: P×t→π are usually defined. Here, t is a quantity representing properties such as optimality and soundness (different from τ in the present invention). On the other hand, the inventors of the present application employ the mental model Π of the user H instead of the action planning problem P to construct the action planning section 151 .

Here, the mental model Π is causal relationship information (CBN configuration information) received from the intention model 14 in this embodiment. , carefully consider the time required to reach destination W, avoid traffic jams, drive as fast as possible, and avoid getting off the train as much as possible."

Specifically, the action planning unit 151 defines this mental model Π of the user H, the transition function ζ _Π : S×A→S×C, and the action planning algorithm PA: π×DM→π*. Here, DM is the value/reward model that constitutes the desire model 12, where the policy π=<a1 , a2,..., an>. Also, π* is the optimal policy as described above.

This optimal policy π* is a set of actions that transition the actual state s to the desired state sd, that is, the transition function is given by the following equation (2) ζ _Π (π, s)=<sd, Σ _ai∈π ( ci＋ri)>
It becomes like this. where ci and ri are the cost and reward to be optimized when executing policy π, respectively. Note that one of the features of the transition function _ζΠ of the above equation (2) is that it includes a reward ri that is not used _in known XAIP.

In order to determine the optimum policy π*, the action planning unit 151 follows the action planning algorithm PA to determine the state s(i+1) (=ζ _Π (π , si)) performs the policy optimization process so that it completely or roughly matches the desired state sd. A specific embodiment of this policy optimization process will be described below.

In this embodiment, basically, the action planning unit 151 needs to maintain a model similar to the user H's mental model. If they do not match, an "explanation" is given to the user H, and the matching of the models is advanced.

For example, when the user H (driver) "continues to drive on the road X", the policy is to "take a detour to the road Y". Road Y is not congested" may be adopted.

As one approach to "clarification" here, inference reconciliation is implemented. Specifically, the action plan algorithm PA ^χ of the action plan unit 151 (hereinafter χ represents the model 1 as AI) generates the policy π, whereas the action plan algorithm PA ^H of the user H is If it does not generate the same policy π, the action planner 151 performs the explanation ε so that PA ^H generates the same policy π, ie PA ^H : Π×D→ ^ε π.

This explanation ε can be, for example, a communication content including a specific question/question about the policy π generated from PA ^χ to be given to the user H (the action plan algorithm PA ^H ). Here, this question/question is ``why is a certain action a in policy π?'', ``why policy π and not another policy π''', or ``why policy π is optimal ( ie why π(s) is a and not a'). By the way, such a dialogue can be implemented via an interface (IF) capable of inputting the measurement results output from the above-mentioned (c) measuring means such as a camera, or capable of text input/output, voice input/output, etc. -ing

Also, apart from the predictive arbitration described above, it may be considered to change the mental model ΠH of the user ^H by this explanation ε. Specifically, the user H may evaluate the policy π that the action plan algorithm PA ^χ is optimal, with a completely different "spirituality". Therefore, the explanation ε is used so that the user H also agrees with the determined policy π. For example, under PA ^χ : Π×D→π, let H ^H be the mental model of user ^H that satisfies PA ^H :H ^H ×D→π. , H ^H (that is, Π ^H + ε→H ^H ).

As yet another approach, the action planner 151 may implement explanations ε that emphasize differences in (values in) value sets τ generated under different assumptions. Implementation of such an explanation ε can be expressed as, for example, ε←τΔτ ^H between the user H and the sense of subjectivity estimation model 1 (behavior planning unit 151), and between the user H1 and the user H2, ε ←τ ^H1 Δτ ^H2 can be expressed. Of these, the latter is compatible with the intention of the subjectivity estimation model 1, which considers the value of the user H1 to be higher than the value of the user H2. It is also possible to realize a well-balanced estimation of the target sense of subjectivity level.

Specifically, under PA ^χ : Π×D→π, τ ^H +ε→τ′ ^H and PA ^χ : π×R× An explanation ε such that τ′ ^H →π is generated and implemented. Alternatively, while the sense of subjectivity estimation model 1 considers the value of user H1 to be higher than the value of user H2 (that is, PA ^χ : Π×R×τ ^H2 →π), τ ^H1 +ε→τ′ ^H1 =τ An explanation ε that satisfies ' ^H2 may be generated and implemented.

Incidentally, as is clear from the above explanation, unlike the situation in the value matching model 13 described above, the action planning unit 151 has a higher problem-solving ability than the user H (action planning algorithm). It has a planning algorithm. That is, PA ^χ >PA ^H.

As described above, several methods for AI to "explain" to the user and achieve model matching have been described. ) are based on three basic principles. That is, as a first basis, the sense of subjectivity estimation model 1 has a high degree of transparency regarding the presentation of action plans and estimation results to the user H, and the user H can ask questions and request explanations. is now possible. It is expected that this will make the sense of subjectivity estimation model 1 trustworthy for the user H, and further improve the relationship between them.

As a second basis, the sense of subjectivity estimation model 1 can explain to the user H that it understands the behavior of the user H itself. Note that this makes it possible to increase the degree of sympathy for the user H's sense of subjectivity estimation model 1 . Furthermore, as a third basis, the sense of subjectivity estimation model 1 works together with the user H to plan and estimate the behavior of the user. It should be noted that this also makes it possible to improve the sense of subjectivity level itself of user H, which is the estimation target of the present invention.

Also in FIG. 1, the action determining unit 152 of the action model 15 uses the optimal policy π* generated by the action planning unit 151 to decide and output the action a to be performed for the observed state s. Specifically, π(s)=a is executed. Incidentally, the actions determined here are the subjective feeling estimation model 1 (when the subjective feeling estimation model 1 performs control as an autonomous AI), the user H , and both of the sense of subjectivity estimation model 1 and the user H (in the case where the sense of subjectivity estimation model 1 and the user H cooperate to perform control).

Here, when the action to be determined is the action of the sense of subjectivity estimation model 1, the action a output from the action determination unit 152 is transmitted via a predetermined interface (IF) (for example, driving an actuator, displaying on a display, or , and voice output from a speaker) to the environmental world including the user H, and the state s of the environmental world receiving this changes to the state s'.

In addition, the state s' observed in the world as a result of action a is defined as "the intended (predicted) state resulting from action a implemented in the action model (and resulting from the action of another agent). It is fed back to the intention model 14 to determine "whether or not".

By the way, assuming that the user H is the driver of a car traveling on the road X, and the environment world is the road traffic condition including the user H's car and the environment around the road network, the determined action: "detour to the road Y For example, a new state: "I am driving on road Y toward destination W" or "The traffic condition of road Y I am driving on is". Congestion "not" will occur. Also, as a result, for example, the "sense of subjectivity level" of user H, which was originally "low", changes to "high".

<Alternative state generation/evaluation model>
Also referring to FIG. 1, the Alternate State Generation and Evaluation Model 16 solves the problem of having to generate and evaluate new policies in previously unseen, unforeseen or infrequently occurring situations. It is a model for In other words, it can be regarded as a model that automatically generates intervention content for "intervention" to encourage behavioral change.

Specifically, the alternative state generation and evaluation model 16 includes a state 'generator' (162) that generates an alternative state s ^al as a possible new state, and an 'evaluator' (162) that evaluates the generated alternative state s ^al ” (164), and is a well-known “actor-critic framework” (Aras Dargazany, arXiv:2004.04574 Artificial Intelligence [cs.AI], 2020) (Zhewei Huang et al., arXiv:1903.04411 This model is based on Computer Vision and Pattern Recognition [cs.CV], 2019). Incidentally, this “actor-critic framework” is built using Generative Adversarial Networks (GAN) and Deep Reinforcement Learning (DRL) algorithms.

However, in this well-known actor-critic framework, processing proceeds under the assumption that AI can accurately model the current environment world, whereas alternative state generation・In the evaluation model 16, knowledge from various contexts learned in the past and knowledge (aggregate of them) from different mental models (CBN) of a plurality of users are adopted to predict (for user H) It learns from various unforeseeable but possible situations and solves problems.

Similarly, in FIG. 1, the CBN aggregation 161 of the alternative state generation/evaluation model 16 is a CBN1 that is an intention model (mental model, causal relationship information) of each of a plurality of different users H1, H2, . . . , Hp. , _CBN2 , _. At least some of CBN1, CBN2, .

Specifically, when given _{CBN H1} = (DAG<Y _H1 , E _H1 >, Prob _H1 ) and CBN _H2 = (DAG<Y _H2 , E _H2 >, Prob _H2 ), Formula (3) CBN _∪ = (DAG _∪ <YH1 _∪YH2 , _{EH1 ∪EH2} >, Prob _{Ｈ1 ∪Prob Ｈ2} )
can be expressed as Here, DAG<Y, E> indicates a directed acyclic graph composed of nodes (variables) Y and edges E, and Prob is the transition probability corresponding to each edge E.

Also in FIG. 1, the state generator 162 of the alternative state generation/evaluation model 16 is:
(a) receiving an observed state s and a corresponding output action a (from the action determination unit 152),
(b) based on the aggregate CBN _∪ (integrated causal relationship information) of intention models received from the CBN aggregate 161,
Generate and output alternative states s ^al as possible state candidates.

Here, the alternative state s ^al is a new state that the user H did not anticipate/anticipate or consider possible, for example, a more suitable alternative solution action for an unknown problem that has not been seen in the past. is used to determine

By the way, the state generator 162 corresponds to the generation part of the well-known generative adversarial network (GAN), but instead of generating, for example, fake data as in the past, it solves a new problem. It generates alternative states s ^al to create new strategies for

Similarly, in FIG. 1, the classifier 163 of the alternative state generation/evaluation model 16 prevents the alternative state s ^al generated by the state generator 162 from being unimaginably fictitious in the context in which the user H is present. determine whether or not That is, the discriminator 163 determines whether the generated new alternative state s ^al can be useful in solving a real problem, and determines whether the alternative state s ^al is such a state. It is intended to encourage training of the instrument 162 .

Specifically, in this embodiment, the classifier 163 is composed of a deep neural network (DNN, Deep Neural Networks) algorithm including a plurality of fully connected layers, and a new From the state s′ and the alternative state s ^al (generated by the state generator 162), a loss ALoss representing the difference from the desired state sd is generated. and (b) train itself (classifier 163).

Here, in a general adversarial generation network (GAN), the adversarial loss is the degree of difference between the current state s' and the generated state sg, that is, max _ψ (Ex _{s ~ μ} [ψ( s')]-Ex _sg˜ug [ψ(sg)]) is calculated. where Ex is the expected value, and μ and ug are the sample probability distributions of the current state and the generated state, respectively.

On the other hand, in this embodiment, the discriminator 163 outputs the hostile loss ALoss itself, which is a scalar.
As (a) ALoss(s'): loss between current state s' and desired state sd, and (b) AL(s ^al ): loss between alternative state s ^al and desired state sd , ALoss(s') and AL(s ^al ).

Also in FIG. 1, the evaluator 164 of the alternative state generation and evaluation model 16 generates a predicted reward, which is the reward corresponding to the alternative state s ^al generated in the state generator 162 (trained by the adversarial loss ALoss). do. Here, this predicted reward no longer includes the reward calculated by the action a output from (the action determination unit 152 of) the action model 15 .

Also, the evaluator 164 causes the action model 15 (the action decision section 152 thereof) to undergo action decision training with this predicted reward. As a result, it is possible to update the behavior determination process in (the behavior determination unit 152 of) the behavior model 15 so that it can be applied to situations that have not been seen in the past, cannot be predicted, or rarely occur. be.

Specifically, the predicted reward at time t can be the Q learning value function Q(s ^al _t ) of reinforcement learning. This Q learning value function Q(s ^al _t ) is expressed by the following equation (4) Q(s ^al _t )=r(s ^al _t , a _t )+γQ(s ^al _t+1 )
, is updated as a discounted reward. Here, r(s ^al _t , at) is the reward for performing action a _t under state s ^al _t .

Next, in this embodiment, the behavior determination unit 152 of the behavior model 15 uses this predicted reward (Q learning value function) Q(s ^al _t ) to calculate π*(s) for deriving the behavior a as r( We train to maximize s ^al _t , π*(s ^al _t )) + Q(ζ(s ^al _t , π*(s ^al _t ))). Here, the transition function ζ(s ^al _t , π*(s ^al _t )) is the alternate state s ^al _t+1 at time t+1. Such an alternative state s ^al _t+1 is generated by the sense of subjectivity estimation model 1 when the user H is in a predicament in which it is impossible to answer the presented question due to the limitation of his ability. It can also be regarded as a presented answer. Also, presenting such an answer to the user H contributes to increasing the reliability of the user H's sense of subjectivity estimation model (behavioral change promotion model) 1 .

<Sense of subjectivity model>
Also referring to FIG. 1, the sense of subjectivity model 17 estimates and outputs real-time variations in sense of subjectivity (SoA) levels dependent on dynamic context as behavioral phenotypes in this embodiment.

In a previous psychological experiment (Tapal, A. et al., Frontiers in Psychology, 8, Article 1552. 2017, <https://doi.org/10.3389/fpsyg.2017.01552>), a specific A direct measurement of subjectivity is being carried out through self-reports (self-reports), which are the results of a person's direct evaluation of his or her sense of agency (SoA, Sense of Agency) about an event.

In the past, there are also examples of direct measurement of subjectivity using the perceptual difference between the measured value of the subject's subjectivity and the measured value from the outside. However, in any method, it is essential that the person being measured responds directly to the sense of subjectivity, and as a result, the person being measured is forced to intermittently suspend their actions, so the sense of subjectivity level fluctuates greatly. It has become difficult to apply in situations where

On the other hand, the inventors of the present application have found that changes in the sense of subjectivity level soa are reflected in physiological indices (e.g., heart rate, breathing speed, etc.), posture, gestures, and speech prosodic indices ( (For example, tone, accent, intonation, speech rate, speech pitch, and speech volume, etc.) clearly appear in temporal changes (as a hypothesis, it works well). Here, these measurements have traditionally been effectively used to estimate emotional states (including emotions and moods).

In the field of so-called Affective Computing, AI is used to recognize the emotions of the target person and adapt to their emotions, based on information on behavior and attitude phenotypes obtained from wearable sensors and environmental sensors. Research to explore responses is being vigorously carried out. Furthermore, there are some studies that prove that human subjectivity and emotion are constantly interacting in daily life (e.g., Matthis Synofzik et al., Front. Psychol., 4( 127), 2013 <https://doi.org/10.3389/fpsyg.2013.00127>, Antje Gentsch1 and Matthis Synofzik, Front. Hum. Neurosci., 8:608, 2014 <https://doi.org/10.3389/ fnhum.2014.00608> etc.).

For example, the sense of agency is based on emotional factors, such as whether the expected outcome of the action is positive or negative, whether the motivation to perform the current action is high or low, and whether the behavior is in a friendly environment. (Julia F Christensen et al., Exp Brain Res. 237(5), 1205-1212, 2019 <https://doi.org/10.1007/ s00221-018-5461-6>) is also disclosed.

In order to formulate the findings and discoveries described above, the sense of subjectivity model 17 includes a sense of subjectivity recognition function Ω for outputting the current sense of subjectivity level soa: ρ1×ρ2×ρ3×ρ4× . stipulate. Here, ρ1, ρ2, . , speech rate, speech pitch, speech volume, etc.).

After defining such a sense of subjectivity recognition function Ω, the sense of subjectivity model 17 is specifically composed of a deep neural network (DNN, Deep Neural Networks) algorithm,
(a) a new state s' caused by the action a output from the action determination unit 152;
(b) receive a feature quantity related to a given feature ρ of the user H (affected by the new state s′) under this new state s′, and obtain these new state s′ and feature quantity Based on ρ, the sense of subjectivity level (soa) of the user H is determined or updated and output. Further, the sense of subjectivity level soa used in the belief model 11 is updated to the determined or updated sense of subjectivity level soa'. This makes it possible to estimate or anticipate the causes and characteristics of, for example, the interruption or discontinuity of the sense of subjectivity.

Incidentally, the subjective feeling level soa' output from the subjective feeling model 17 updates the "belief information" of the belief model 11, and furthermore, this updated belief information is the "causal relationship information" (CBN configuration information) of the intention model 14. ) embodies the conventional psychological theory that ``perceived control'', which is related to the sense of subjectivity, affects ``intention''. It is a thing.

Here, the sense of subjectivity model 17 is
(c) Self-report (self-report) soa_r on user H's sense of subjectivity received from value matching model 13
It is also preferable to determine or update and output the sense of subjectivity level (soa) of user H based on also. This is because the integrated value model 13, as part of the processing flow of CIRL (collaborative inverse reinforcement learning), asks the user H the subjective feeling level As a result of the inquiry, the processing is performed when a self-report soa_r related to user H's sense of agency is acquired.

The process of generating and updating the sense of subjectivity level soa in the sense of subjectivity model 17 has been described above. As a result, it is possible to further demonstrate to user H the high level of empathy and reliability. Furthermore, the sense of subjectivity estimation model 1 can also persuade user H that the presented policy will only improve user H's sense of subjectivity level and not reduce it. In this way, user H can also be provided with an intelligent and sensible response.

[Sense of subjectivity estimation device, sense of subjectivity estimation program]
Hereinafter, referring back to FIG. 1, the sense of subjectivity estimation device 9 that incorporates the sense of subjectivity estimation model 1 described above and estimates the sense of subjectivity of the user H who is the estimation target user will be described. By the way, with the same configuration, this device can also be used as a behavior change promotion device equipped with a behavior change promotion model (subjectivity estimation model) 1 .

A sense of subjectivity estimation device 9 of this embodiment shown in the lower left part of FIG. Specifically, it is a device that determines the sense of subjectivity level of the user H. FIG.

Specifically, in FIG. 1, the user interface (IF) 91 of the sense of subjectivity estimation device 9 includes the interface (IF) of the value matching model 13, the interface (IF) of the action planning unit 151, and the behavior model 15 (behavior determination unit 152), which takes in measurement results related to user H, inputs and outputs information related to various types of communication with user H, and acts on the environment world with determined actions. fulfill It also serves as an input unit for collecting information necessary for training the subjective feeling estimation model 1, such as the state of the environment world, and information serving as conditions for subjective feeling estimation.

The training unit 92 generates training data from the received information necessary for training the sense of subjectivity estimation model 1, and uses this to train the sense of subjectivity estimation model 1. FIG.

The sense of subjectivity estimation unit 93 determines the sense of subjectivity level of the user H using the trained sense of subjectivity estimation model 1 based on the received information serving as conditions for estimation of the sense of subjectivity. Here, in this embodiment, it is possible to determine and output the real-time sense of subjectivity level of user H and dynamic changes in the sense of subjectivity level of user H even in a complex or ever-changing environmental world. It has become.

The output unit 94 transmits information related to the determined sense of subjectivity level to an external information processing device (if it has a communication function) or displays it (if it has a display function). .

Here, the training unit 92 and the sense of subjectivity estimation unit 93 are main functional components that implement an embodiment of the sense of subjectivity estimation method according to the present invention, and store an embodiment of the sense of subjectivity estimation program according to the present invention. It can also be regarded as a function of a processor memory that has been processed. For this reason, the subjective feeling estimation device 9 may be a dedicated device for subjective feeling estimation, but it may be a cloud server, a non-cloud server device, a personal computer (for example, a cloud server, a non-cloud server device, a personal computer ( PC), notebook or tablet computer, smart phone, wearable computer, or the like.

As described in detail above, according to the present invention, it is possible to estimate a user's sense of subjectivity, which has been difficult to estimate in the past (especially, it was not possible to estimate under a complex and ever-changing environmental world). In addition, taking into account the user's sense of subjectivity, which has been appropriately updated, it is possible to determine behavior in relation to the environment, thereby encouraging behavioral change of the user, that is, change in intention or behavior.

For example, by adopting an appropriate embodiment, it is possible to distinguish between actions taken by the user of his/her own will (in response to suggestions and persuasion from the sense of subjectivity estimation model according to the present invention) and the state of the environmental world that appears as a result. It is also possible to make appropriate proposals and persuasion, etc., so that the user himself/herself can feel a sense of connection and interlocking, that is, without diminishing the user's sense of independence.

In addition, because the present invention produces the effects described above, it will be possible to improve and develop the contents of mutual understanding and collaborative activities between humans and AI that will be seen in various situations in the future. is also expected to make a significant contribution to

Furthermore, for example, in order to provide high-quality education to children, that is, education that respects children's independence and motivation to study, using the subjectivity estimation model and the behavior change promotion model according to the present invention, children It is also possible to carry out educational activities that include suggestions and guidance to maintain and improve the sense of independence of children. In other words, according to the present invention, it contributes to Goal 4 of the Sustainable Development Goals (SDGs) led by the United Nations, "Provide inclusive, equitable and quality education and promote lifelong learning opportunities for all." It is also possible to

In addition, for example, in order to provide adults with decent work (work that is worthwhile and humane), the sense of subjectivity estimation model and behavioral change promotion model according to the present invention are used to maintain and improve the sense of subjectivity of adults. It can also provide job-finding or job advice that encourages adults to make appropriate job behavior changes. In other words, according to the present invention, it is possible to contribute to Goal 8 of the SDGs led by the United Nations, "Promote inclusive and sustainable economic growth, employment and decent work for all". be.

Furthermore, for example, for drivers of automobiles traveling in urban areas, in order to achieve the objectives of these drivers reliably and smoothly, the sense of subjectivity estimation model and the behavioral change promotion model according to the present invention are used to determine the subjectivity of the drivers. It is also possible to implement navigation that does not impair the driver's feeling, that is, that is easy for the drivers to understand. In other words, according to the present invention, it is possible to contribute to Goal 11 of the SDGs led by the United Nations, "Make cities inclusive, safe, resilient and sustainable."

Furthermore, for example, in order to provide consumers with sustainable consumption and lifestyles, the sense of subjectivity estimation model and the behavioral change promotion model according to the present invention are used so that the sense of subjectivity of consumers is not diminished, that is, consumption It is also possible to implement advice and proposals on consumer behavior that are easy for people to understand. In other words, according to the present invention, it is possible to contribute to Goal 12 of the SDGs led by the United Nations, "ensure sustainable consumption and production patterns."

For the various embodiments of the present invention described above, various changes, modifications and omissions within the spirit and scope of the present invention can be easily made by those skilled in the art. The above description is exemplary only and is not intended to be limiting. The invention is to be limited only as limited by the claims and the equivalents thereof.

1 Sense of agency estimation model (behavior change promotion model)
11 belief model 12 desire model 13 value matching model 14 intention model 15 action model 151 action planning unit 152 action decision unit 16 alternative state generation/evaluation model 161 CBN (Causal Bayesian Network) aggregate 162 state generator 163 discriminator 164 evaluation Device 17 Subjectivity model 9 Subjectivity estimation device 91 User interface (user IF)
92 training unit 93 sense of subjectivity estimation unit 94 output unit

Claims (11)

A sense of subjectivity estimation model that causes a computer to function to estimate a sense of subjectivity of a user while determining behavior with respect to the state of the environment world including the user using a reward,
a belief model for generating or updating belief information, which is information including the probability that a new state will occur as a result of performing a certain action on a certain state under a certain sense of subjectivity level of the user;
Receive value information related to value for the user and the corresponding reward, generate a desired state desired by the user, and determine a policy that is a set of actions that can bring about the desired state. Desire model and
an intention model that generates causal relationship information relating to causal relationships between states, values, rewards, and actions based on the belief information, the value information, and the reward;
an action model that determines and outputs an action to be taken for an observed state based on the policy and the causal relationship information;
determining or updating and outputting the sense of subjectivity level of the user based on the output new state caused by the action and the feature amount related to the predetermined feature of the user under the new state; 2. A sense of subjectivity estimation model characterized by causing a computer to function as a sense of subjectivity model for updating the sense of subjectivity level used in the belief model to the determined or updated sense of subjectivity level. The behavioral model is
an action planning unit that generates an optimal policy based on the policy, the causal relationship information, and the contents of communication including a predetermined question made with the user;
2. The subjective feeling estimation model according to claim 1, further comprising an action determination unit that determines and outputs an action to be taken for an observed state using the generated optimal policy. receive information related to the value information and information related to the remuneration corresponding to the information related to the value information from the user, and based on the information related to the value information and the information related to the remuneration, the value information and the 3. A sense of subjectivity estimation model according to claim 1 or 2, further comprising a computer functioning as a value matching model for generating or updating the corresponding reward and outputting it to said desire model. 4. The sense of subjectivity estimation model according to claim 3, wherein the value matching model is constructed using an algorithm related to Cooperative Inverse Reinforcement Learning (CIRL). Receiving an observed state and a corresponding output action, and selecting an alternative state as a possible state candidate based on integrated causal relationship information that integrates the causal relationship information for each of at least a plurality of users. a state generator that generates and outputs;
Generate a loss representing the difference from the desired state from the new state caused by the action and the alternative state, train the state generator with the loss, and train itself with the loss. a discriminator that performs
an evaluator that generates a predicted reward that is a reward corresponding to the alternative state generated by the trained state generator, and that uses the predicted reward to train the behavior model to determine the behavior. 5. The sense of subjectivity estimation model according to any one of claims 1 to 4, further comprising a computer functioning as an alternative state generation/evaluation model. 6. The sense of subjectivity estimation model according to claim 5, wherein the alternative state generation/evaluation model is constructed using an algorithm related to Generative Adversarial Networks (GAN). 7. Sense of subjectivity estimation according to any one of claims 1 to 6, wherein said belief model is constructed using an algorithm related to Partially Observable Markov Decision Process (POMDP). model. 8. The sense of subjectivity estimation model according to any one of claims 1 to 7, wherein the causal relationship information is information related to a Bayesian network algorithm. 9. A sense of subjectivity estimating apparatus that uses the sense of subjectivity estimation model according to any one of claims 1 to 8 to estimate a sense of subjectivity of the user from an observed state in the environment. A method for estimating a sense of subjectivity in a computer for estimating a sense of subjectivity of a user while determining behavior with respect to the state of the environment world including the user using a reward, comprising:
a step of generating or updating belief information, which is information including the probability that a new state will occur as a result of performing a certain action on a certain state under a certain sense of subjectivity level of the user;
Receive value information related to value for the user and the corresponding reward, generate a desired state desired by the user, and determine a policy that is a set of actions that can bring about the desired state. a step;
generating causal relationship information relating to causal relationships between states, values, rewards and actions based on the belief information, the value information and the reward;
determining and outputting an action to be taken for the observed state based on the policy and the causality information;
determining or updating and outputting the sense of subjectivity level of the user based on the output new state caused by the action and the feature amount related to the predetermined feature of the user under the new state; and a sense of subjectivity model for updating a sense of subjectivity level used in the step of generating or updating the belief information to the determined or updated sense of subjectivity level. A behavior modification promotion model that functions a computer that encourages behavior modification of a user while determining behavior with respect to the state of the environment world including the user using a reward,
a belief model for generating or updating belief information, which is information including the probability that a new state will occur as a result of performing a certain action on a certain state under a certain sense of subjectivity level of the user;
Receive value information related to value for the user and the corresponding reward, generate a desired state desired by the user, and determine a policy that is a set of actions that can bring about the desired state. Desire model and
an intention model that generates causal relationship information relating to causal relationships between states, values, rewards, and actions based on the belief information, the value information, and the reward;
Based on the policy, the causal relationship information, and the contents of communication including the predetermined question with the user, an optimal policy is generated, and using the optimal policy, an action model that determines and outputs an action to be taken for an observed state;
determining or updating the sense of subjectivity level of the user based on the output new state caused by the action and the feature amount of the predetermined feature of the user under the new state; A behavior change promotion model characterized by causing a computer to function as a sense of subjectivity model for updating the sense of subjectivity level used in the above to the determined or updated sense of subjectivity level.

Images