JP7448502B2 - Sense of agency estimation model, device and method, and behavioral change promotion model - Google Patents

Description

The present invention relates to persuasive technology that promotes behavioral change in people through "persuasion."

Persuasive technology that promotes behavior change in users through "persuasion" in a broad sense that can change users' beliefs, perceptions, and desires in a predetermined direction is attracting attention. For example, in the field of health and welfare, the field of education, and the field of urban transportation, the application of this persuasive technology is being actively pursued, as disclosed in Patent Document 1, Patent Document 2, and Patent Document 3, respectively. is being advanced.

In addition, persuasive technology will continue to develop as an indispensable technology for realizing autonomous AI systems that provide a variety of services to users through communication with users. It is thought that it will happen.

Rita Orji and Karyn Moffatt, "Persuasive technology for health and wellness: State-of-the-art and emerging trends", Health Informatics J. 24(1), pp.66-91. 2018, <https://doi .org/10.1177/1460458216650979＞ Yohana Dewi Lulu Widyasari et al., "Persuasive technology for enhanced learning behavior in higher education", International Journal of Educational Technology in Higher Education, 16:15, 2019, <https://doi.org/10.1186/s41239-019 -0142-5＞ Evangelia Anagnostopoulou et al., "Persuasive Technologies for Sustainable Mobility: State of the Art and Emerging Trends", Sustainability 2018, 10(7), pp.2128, 2018, <https://doi.org/10.3390/su10072128>

However, with the conventional persuasive techniques described above, even if AI "persuades" the user, it is difficult to induce more appropriate behavior changes in the user's complex or ever-changing normal environmental situations. The reality is that it is still difficult to encourage, specifically to encourage more appropriate decision-making and behavioral decisions.

The inventors of the present application believe that the reason for this difficulty is that the user himself/herself has created a connection or interlock between the action taken of one's own will after being "persuaded" by AI and the state of the environmental world that appeared as a result. I believe that this is due to the fact that there are many cases where people are unable to feel their sexuality. In other words, conventional persuasive techniques do not take into account any changes in the user's ``sense of agency'' (the feeling that one's actions have an impact on the surroundings), so it is important to reduce this ``sense of agency''. They discovered that it has become extremely difficult to provide appropriate "persuasion" that satisfies users.

Therefore, the present invention provides a sense of agency estimation model, a sense of agency estimation device, and a sense of agency that can estimate a user's sense of agency and encourage changes in the user's intention or behavior by taking the sense of agency into consideration. The purpose is to provide an estimation method and a behavior change promotion model.

According to the present invention, there is provided a sense of agency estimation model that allows a computer to function to estimate a user's sense of agency while determining behavior regarding the state of the environmental world including the user using rewards,
a belief model that generates or updates belief information that is information that includes the probability that a new state will occur as a result of performing a certain action for a certain state under a certain sense of agency level of the user;
Receive value information related to value for the user and the corresponding reward, generate a desired state that is the state desired by the user, and determine a policy that is a set of actions that can bring about the desired state. Desire model and
an intention model that generates causal relationship information regarding a causal relationship between states, values, rewards, and actions based on the belief information, the value information, and the reward;
Based on the policy, the cause-and-effect relationship information , and the content of the communication including the predetermined questions conducted with the user , an optimal policy that is the optimal policy is generated, and using the optimal policy, a behavior model that determines and outputs actions to be taken in response to observed conditions;
Determining or updating the sense of agency level of the user based on the output new state caused by the action and the feature amount related to the predetermined characteristics of the user under the new state, and outputting the determined level. A sense of agency estimation model is provided that causes a computer to function as a sense of agency model that updates the sense of agency level used in the above belief model to the determined or updated sense of agency level.

Furthermore, as another embodiment of the sense of agency estimation model according to the present invention, the sense of agency estimation model receives from the user information related to the value information and information related to the reward corresponding to the information related to the value information. and further cause the computer to function as a value matching model that generates or updates the value information and the corresponding remuneration based on the information regarding the value information and the information regarding the remuneration, and outputs it to the above-mentioned desire model. is also preferable. Here, it is also preferable that this value matching model is constructed using an algorithm related to Cooperative Inverse Reinforcement Learning (CIRL).

Furthermore, as yet another embodiment of the sense of agency estimation model according to the present invention, the sense of agency estimation model is as follows:
Receive the observed state and the corresponding output action, and select an alternative state as a possible state candidate based on integrated causal relationship information that integrates the causal relationship information for each of at least a plurality of users. a state generator that generates and outputs;
Generate a loss representing the difference from the desired state from the new state caused by the output action and the alternative state, train the state generator using the loss, and use the loss to train the state generator. A discriminator that trains itself,
an evaluator that generates a predicted reward that is a reward corresponding to the alternative state generated by a trained state generator, and trains a behavioral model to determine the behavior using the predicted reward; It is also preferable to have the computer further function as a generation/evaluation model. It is also preferable that this alternative state generation/evaluation model is constructed using an algorithm related to generative adversarial networks (GAN).

Further, in the sense of agency estimation model according to the present invention, it is preferable that the belief model is constructed using an algorithm related to a Partially Observable Markov Decision Process (POMDP). Further, it is also preferable that the causal relationship information in the intention model is information related to a Bayesian network algorithm.

According to the present invention, there is also provided a sense of agency estimation device that estimates the user's sense of agency from the observed state in the environmental world using the sense of agency estimation model described above.

According to the present invention, there is further provided a sense of agency estimation method in which a computer estimates a user's sense of agency while determining an action regarding the state of the environmental world including the user using a reward,
generating or updating belief information, which is information that includes a probability that a new state will occur as a result of performing a certain action on a certain state under a certain sense of agency level of the user;
Receive value information related to value for the user and the corresponding reward, generate a desired state that is the state desired by the user, and determine a policy that is a set of actions that can bring about the desired state. step and
a step of generating causal relationship information regarding a causal relationship between a state, a value, a reward, and an action based on the belief information, the value information, and the reward;
Based on the policy, the cause-and-effect relationship information , and the content of the communication including the predetermined questions conducted with the user , an optimal policy that is considered to be the optimal policy is generated, and the generated optimal policy is a step of determining and outputting an action to be taken for the observed state using the method;
Determining or updating the sense of agency level of the user based on the output new state caused by the action and the feature amount related to the predetermined characteristics of the user under the new state, and outputting the determined level. A method for estimating a sense of agency is provided, comprising: updating the sense of agency level used in the step of generating or updating the belief information to the determined or updated sense of agency level.

According to the present invention, there is further provided a behavior change promotion model that operates a computer that promotes behavior change in a user while determining behavior regarding the state of the environmental world including the user using rewards,
a belief model that generates or updates belief information that is information that includes the probability that a new state will occur as a result of performing a certain action for a certain state under a certain sense of agency level of the user;
Receive value information related to value for the user and the corresponding reward, generate a desired state that is the state desired by the user, and determine a policy that is a set of actions that can bring about the desired state. Desire model and
an intention model that generates causal relationship information regarding a causal relationship between states, values, rewards, and actions based on the belief information, the value information, and the reward;
Based on the policy, the cause-and-effect relationship information, and the content of the communication including the predetermined questions conducted with the user, an optimal policy that is the optimal policy is generated, and using the optimal policy, a behavior model that determines and outputs actions to be taken in response to observed conditions;
Based on the output new state caused by the action and the feature amount related to the predetermined characteristics of the user under the new state, the user's sense of agency level is determined or updated, and the above beliefs are determined. A behavior change promoting model is provided that causes a computer to function as a sense of agency model that updates the sense of agency level used in the model to the determined or updated sense of agency level.

According to the sense of agency estimation model, sense of agency estimation device, sense of agency estimation method, and behavioral change promotion model of the present invention, a user's sense of agency is estimated, and the user's intention or intention is determined based on the sense of agency. It can encourage changes in behavior.

1 is a schematic diagram showing an embodiment of a sense of agency estimation model according to the present invention, and a functional block diagram showing a functional configuration of an embodiment of a sense of agency estimation device according to the present invention. FIG. 1 is a schematic diagram for explaining a theoretical basic system in an embodiment of the present invention.

Hereinafter, embodiments of the present invention will be described in detail using the drawings.

[Sense of agency estimation model, behavior change promotion model]
FIG. 1 is a schematic diagram showing an embodiment of a sense of agency estimation model according to the present invention, and a functional block diagram showing a functional configuration of an embodiment of a sense of agency estimation device according to the present invention.

The sense of agency estimation model 1 of this embodiment shown in FIG. , is a model in which it is possible to take actions on the environmental world, including the user H, and change the state of the environmental world toward a desired state.

Here, the sense of agency estimation model 1 acquires information related to predetermined characteristics of the user (for example, physiological indicators such as heart rate and breathing rate) through the above-mentioned communication, and estimates the user H's "sense of agency". (The feeling of having an impact on those around you through your actions; hereafter abbreviated as sense of agency) is quantified. Furthermore, while communicating with user H, this "sense of ownership" is updated as appropriate during the cycle of formulating policies and taking actions in response to the state of the environmental world. Furthermore, in this embodiment, for example, it is also possible to determine and output the user H's real-time "sense of subjectivity" and the dynamic fluctuations of the user H's "sense of subjectivity."

Furthermore, in this embodiment, the sense of agency estimation model 1 maintains and improves the "sense of agency" of the user H by appropriately communicating with the user H even in complex and ever-changing environmental world situations. At the same time, it also serves as a behavior change promotion model that can encourage user H to change his or her behavior, specifically, to encourage more appropriate decision-making and behavioral decisions.

Specifically, the sense of agency estimation model (behavior change promotion model) 1 determines "behavior" for the state of the environmental world, including the user H, using the "reward" calculated based on the state and the action. This is a model for estimating user H's "sense of agency," and as shown in Figure 1, at least
(A) A belief that generates or updates "belief information" that is information that includes the probability that a new state will occur as a result of performing a certain action in a certain state under a certain "level of agency" of the user H. Model 11 and
(B) Receive "value information" related to value for user H and the corresponding "reward" and generate a desired state that is the state desired by user H based on the "value information" and "reward" and a desire model 12 that determines a "policy" that is a set of actions that can bring about the desired state;
(C) an intention model 14 that receives "belief information", "value information" and "reward" and generates "causal relationship information" regarding the causal relationship between the state, value information, reward, and behavior;
(D) a behavior model 15 that determines and outputs the "action" to be taken in response to the observed state based on the "policy" and "causal relationship information";
(E) A “new state” caused by the output “behavior” and “feature amounts” related to predetermined characteristics of user H under the “new state” (e.g. heart rate, breathing rate, etc.) Based on the "new state" and "features", the user H's "sensitivity level" is determined or updated and output, and the above (A) This is a model in which a computer functions as a sense of agency model 17 that updates the "level of sense of agency" used in the belief model 11 of 2008 to the determined or updated "level of sense of agency."

In this way, sense of agency estimation model 1 can estimate the "sense of agency," which has traditionally been difficult to estimate (especially in a complex and ever-changing environment). Furthermore, by determining the behavior toward the environmental world by taking into consideration User H's appropriately updated "sense of agency (level)," it is also possible to encourage User H to change his or her behavior, that is, change in intention or behavior. . For example, according to the sense of agency estimation model 1, there is a difference between an action that the user H takes of his or her own will (in response to a proposal or persuasion from the sense of agency estimation model 1) and the state of the environmental world that appears as a result of the action. In the meantime, it is also possible to make appropriate proposals and persuasion by improving User H's sense of agency, so that User H feels a sense of connection and interactivity, without diminishing it. .

Next, the theoretical basis of the functional configuration of the sense of agency estimation model (behavior change promotion model) 1 of this embodiment will be explained using FIG. 2.

FIG. 2 is a schematic diagram for explaining the theoretical basic system in one embodiment of the present invention.

In general, a user's sense of agency (SoA) is based on the occurrence of a disturbing event that interrupts the good flow of the user's intentional actions (which are aimed at a predicted state in the environmental world). It fluctuates under strong influence. Below, using FIG. 2, we will explain how the sense of agency level changes in the chain of state-intention-action-new state.

First, the belief-desire-intention model (Georgeff et al., 5th International Workshop, ATAL'98 Proceedings, pp.1-10, 1998) is known as a model for deriving intentions that will occur in the future. In this model, “beliefs” (Figure 2) and “desires” (Figure 2) represent the perceived state structure of the environmental world and the desired/desired state of the environmental world, respectively. It is a collection of information about the structure.

In addition, "intention" (Figure 2) is determined from "belief" and "desire," and specifically, it is information that includes actions that are assumed to bring about the desired/desired state structure of the environmental world. . In this way, ``intention'' specifies and controls behavior, and this ``intention'' specifies and controls behavior, and is updated under the influence of changed ``beliefs'' and ``desires.''

In addition, the sense of agency here is a feeling determined by whether or not the "intention" that will achieve the desired/desired state structure is embodied in the "action" (Figure 2). ” (Figure 2).

The inventors of this application have integrated the above-mentioned knowledge about "intention" with conventional theories of cognitive science and neuroscience regarding the emergence and discontinuation of a sense of agency, and have devised a basic system as shown in Figure 2. be.

Conventionally, in cognitive science and neuroscience, there has been a theory of learning control of motors, and the validity of models using "comparators" (Figure 2) for behavioral cognition has been debated. According to this "comparator" model, the actuation of the motor is performed in conjunction with a predicted motor output that is generated based on an efferent copy of the command to the motor. Next, the "comparator" compares this predicted result with the actually measured motor output, and if the two match, this motor output is recorded as being caused by the drive (action) of the motor itself. Ru. On the other hand, if they do not match, it is assumed that an interruption or discontinuation of the "sense of ownership" in the sense of controlling the motor drive has occurred.

In contrast, the theory of retrospective inference (RI) in cognitive science and neuroscience holds that a sense of agency has occurred if the intended or predicted state matches the observed actual state. ``estimation'' (Figure 2). In addition to ``intention,'' other higher-order (cognition-related) factors, such as cues related to the context of the outside world and social situations, are also considered in ``estimating'' the sense of agency. Specifically, according to this theory, if the observed state occurs as predicted, the behavior will be carried out smoothly, and thoughts about the behavior and attitude will be left in the back of consciousness. On the other hand, if the observed state differs from the prediction, the brain performs retrospective inference (RI) to answer whether the action caused the observed state.

Hereinafter, a detailed explanation will be given of the specific configuration of the sense of agency estimation model (behavior change promotion model) 1 as an embodiment of the present invention, in which the theoretical basic system explained above is implemented in a computer.

By the way, the human brain normally maintains a mental model that represents the mind of another person, and uses this model to process the process of sensing the mental state of another person. The so-called Theory of Mind (ToM), which is a cognitive science theory about this processing ability, states that because people possess this processing ability, they are able to understand how others behave in various contexts. It is said that it is possible to intuitively gain an understanding of why a person behaves in a certain way.

The sense of agency estimation model (behavior change promotion model) 1 described below emulates this theory of mind (ToM), and includes "beliefs," "desires," "intentions," and "action plans and choices." and "estimation" are attributed to another person (user H in this embodiment), why does the other person (user H) act in this way, and how does he or she perceive the state of the environmental world as a result of the action? In other words, we understand what the other person's (user H's) sense of agency looks like and how it works.

[Model configuration, sense of agency estimation method]
Hereinafter, the functional configuration of an embodiment of the sense of agency estimation model (behavior change promotion model) 1 according to the present invention will be described in more detail. Similarly, according to Figure 1, the sense of agency estimation model (behavior change promotion model) 1 is as follows:
(A) Belief model 11, desire model 12, value matching model 13, intention model 14,
(a) a behavior model 15 including an action planning unit 151 and a behavior determining unit 152;
(c) an alternative state generation/evaluation model 16 including a CBN (Causal Bayesian Network) aggregate 161, a state generator 162, a discriminator 163, and an evaluator 164;
(d) A sense of agency model 17 is provided as a functional component implemented by (a program installed in) a computer. Each of the above functional components will be specifically explained below.

<Belief model>
Similarly, in FIG. 1, the belief model 11 assumes that as a result of a certain action a for a certain state s under a certain "level of agency" soa of the user H, a certain new state s' is created. This is a model that generates or updates "belief information," which is information that includes the probability of occurrence. In this embodiment, the belief model 11 is constructed using an algorithm related to Partially Observable Markov Decision Process (POMDP) (MONAHAN GE Management Science 28(1), 1-16, 1982). .

Specifically, Belief Model 11 is
(a) Let the state space S be the set of states s that the environmental world including the user H and the sense of agency estimation model 1 can take,
(b) Let the set of actions a that the user H and the sense of agency estimation model 1 can perform (that can be output) be an action space A,
(c) Let the conditional probability that a transition to state s' occurs when action a is performed in state s be the transition probability T(s'|s, a),
(d) Let the cost of performing action a in state s be c=c(s, a),
(e) If a transition to state s' occurs when action a is performed in state s, the probability that sense of agency estimation model 1 obtains observation o from the environmental world is the observation probability O(o|s', a) As,
Sense of agency estimation model 1, which performed action a in state s, derives belief B(s'), which is the probability that the environmental world takes state s', as "belief information" when observation o is obtained.

More specifically, if the belief at the previous point is B(s) and β=1/Prob(o|b, a) is the normalization constant, then the current update result of (B(s)) is ) Belief B(s') is expressed by the following formula (1) B(s')=β・O(o|s', a)・Σ _s∈S Τ(s'|s, a)B(s)
It can be calculated by This is information that can be interpreted as the degree of "belief/certainty" about the likelihood of possible states in the environmental world.

Here, the "sense of agency level" soa, which will be explained in detail later, is an element of the state space S as a state that the user H can take, and the belief B(s') is derived from the sense of agency model 17 described later. The updated value is obtained by taking the sum (Σ _s∈S in the above equation (1)) of {s} including the received updated "sense of agency level"soa'.

By the way, if user H is the driver of a car traveling on road "The traffic situation on road X at the location of user H is 'normal'", "The traffic situation ahead on road X is 'congestion'", "The weather is 'sunny'", ...and the actions may be, for example, ``continue driving on road Can be done. Further, the "level of sense of subjectivity" may be set to five levels, for example, "high", "a little high", "neutral", "a little low", and "low".

Here, in the entire sense of agency estimation model 1 including this belief model 11, unlike usual, the sense of agency estimation model 1 and the user H are not completely separate entities, and the sense of agency estimation model 1 is The process is performed in the hope that user H will cooperate. In addition, the main sense of agency estimation model 1 does not use the reward (function) used in normal POMDP in the belief model 11, but instead uses the reward r is adopted.

<Desire model>
Similarly, in FIG. 1, the desire model 12 receives "value information" related to the value v for the user and the corresponding reward r, and based on the "value information" and the reward r, the desire model 12 This is a model that generates a desired state sd (∈S), which is a desired state, and determines a policy π, which is a set of actions a that may bring about the desired state sd.

In this embodiment, the desire model 12 is specifically configured with a deep neural network (DNN) algorithm,
(a) A set of values v (e.g. social values) for user H τ=<v1, v2,..., vn>, for example τ=<efficiency, frugality, altruism, happiness, self degree of love, degree to which one senses a sense of agency in others> and
(b) The desired state sd (∈S ) is used as a value/reward model DM that outputs the policy π(o)=<a1, a2,..., an> is determined.

Here, both the set of values τ in (a) above and the reward r in (b) above are given or shown by the user H in the interaction with the sense of agency estimation model 1. ing. Incidentally, such a set τ of values v is generated by the value matching model 13, which will be explained later.

<Value integration model>
Generally speaking, people often mistakenly or misrepresent to others what they actually want in a given situation. This becomes a major problem when trying to convey one's wishes to an AI and have it act in accordance with one's expectations.

For example, people expect AI robots to make coffee for themselves (people) rather than for the AI robots to enjoy coffee themselves (this is considered the value of a correct answer). Additionally, people will want autonomous (self-driving) cars to recognize what values are important to them while driving. For example, we expect them to recognize as essential values such as following traffic rules, keeping a distance from pedestrians, and not getting stuck in traffic jams when a child is in the car. However, it has traditionally been extremely difficult for AI to accurately recognize the needs of such people, or in other words, to match the value for people with the value that AI incorporates.

For example, AI that performs general reinforcement learning (RL) learns the optimal policy by maximizing the cumulative reward that is added by discounting rewards that are far in the future. The reward that is given is an abstract mathematical quantity, and is not something inherent in the real environmental world. Furthermore, we have created a model that answers the questions of what values people should consider and why they consider certain values important, not as abstract mathematical quantities, but as quantities that can actually be obtained. It is very difficult to do so.

In the past, attempts have also been made to solve this value matching problem using inverse reinforcement learning (IRL) (Andrew Ng and Stuart J Russell, ICML 2000: Proceedings of the Seventeenth International Conference). However, people do not want AI to understand all of their wishes (for example, personal wishes such as enjoying coffee time do not need to be understood), and AI needs to distinguish between them and process them. This was a very difficult problem. Furthermore, IRL assumes that the observed person's behavior and attitude are optimized, and makes adjustments by utilizing various useful information contained in the observed person's behavior and attitude. I couldn't.

Therefore, the value matching model 13 according to the present invention uses Cooperative Inverse Reinforcement Learning (CIRL) (Dylan Hadfield-Menell et al., 30th Conference on Neural It is constructed using algorithms related to Information Processing Systems (NIPS) 2016).

Specifically, the value matching model 13 receives requests from the user as part of the CIRL processing flow.
(a) Information related to "value information", in this embodiment, a set of values v for user H (values v that user H recognizes as important (for example, in social life)) τ=<v1, v2 ,..., vn>, that is, information about each value v1, v2,..., vn,
(b) Information regarding the reward r corresponding to the information regarding "value information", in this embodiment, information regarding the reward r for each value v1, v2,..., vn is received, and these (a) and Based on the information in (b), this model generates or updates "value information" and the corresponding reward r, and outputs it to the desire model 12.

Here, the value matching model 13 (as the sense of agency estimation model 1) is
(A) Initially, the user H does not clearly recognize what he personally recognizes as the value, in this embodiment, the value set τ=<v1, v2,..., vn>;
(b) Observations related to the value set τ are provided to the user H via an interface (IF) that allows input of measurement results output from measurement means such as a camera, or allows for text input/output, voice input/output, etc. or make inquiries,
(C) Based on the basic premise that "users usually act based on the value set τ," information regarding the value set τ (e.g., the user's actions/attitudes and answers, as well as the rewards obtained in relation to them, etc.) information related to results),
(D) Based on the collected information, value set/reward generation processing is performed with the aim of maximizing the estimated value set τ.

Incidentally, in order to determine a more consistent value set τ, it is also preferable that the interaction between the sense of agency estimation model 1 (value matching model 13) and the user H as described above is continuously and repeatedly performed.

Further, in this embodiment, the value matching model 13 makes inquiries and requests to the user H via the above-mentioned interface (IF), and generates a self-report (self-report) regarding the user H's sense of agency based on the response contents. ) soa_r can also be generated and output to the sense of agency model 17, which will be explained in detail later.

<Intention model>

The previously explained belief-desire-intention model (Georgeff et al., 5th International Workshop, ATAL'98 Proceedings, pp.1-10, 1998) is known as a model for deriving intentions that will occur in the future. Here, intention is expressed as an action that will cause a desired state (result) in the environmental world. However, in deriving user H's sense of agency level soa, the problem is how sense of agency estimation model 1 (and user H) understands the relationship between causes (actions) and results (states), that is, the causal relationship. must be resolved.

In order to solve this problem, the intention model 14 in FIG.
(a) Belief information received from belief model 11,
(b) Based on the value set τ (value information) received from the desire model 12 and the corresponding reward, the relationship between the state, value, reward, and action (in this embodiment, the cost included in the belief information) This is a model that generates "causal relationship information" related to causal relationships.

In this embodiment, the intention model 14 is constructed using a Causal Bayesian Network (CBN) algorithm. In addition, in this embodiment, the output "causal relationship information" is information related to the CBN algorithm, specifically, the configuration information of the configured CBN itself (see the lower left side of FIG. 1), and specifically, This information includes the conditional probability of the resulting state s' given the state s and the action a.

Here, CBN is a directed acyclic graph model, which includes a group of nodes {Y _i } with a directed edge E specified by a group of parent functions {pa(Y _i )}, and a conditional probability group {Prob (Y _i |pa(Y _i )}, where each node Y _i corresponds to one of the state, action, cost, value, and reward. Also, the directed edge E indicates that there is a possibility of a causal transition between Y _i and Y _j connected by this, and specifically, Y _i causes Y _j with a certain probability, in other words. This shows that the probability of occurrence of Y _j is expressed by a conditional probability distribution with Y _i as a condition.

Furthermore, in this embodiment, a do operator: do(Y _j =y _j ) is defined in the CBN. When this do operator is applied to the CBN, pa(Y _i )=Φ and Prob(Y _i )=δ(Y _i ;y _i ). In other words, the do operator corresponds to the "intervention (in behavior change theory)" implemented by the sense of agency estimation model 1.

In addition, the intention model 14 initially estimates causal relationships in the environmental world through ``interventions,'' but ultimately, ``what would have happened if a different ``intervention'' had been implemented in the past?'' It becomes necessary to answer counterfactual and hypothetical questions such as "Is this true?" Therefore, in this embodiment, the intention model 14 can also take a "counterfactual virtual mode."

As explained above, the intention model 14 is
・Human intention is said to be formed based on thoughts about ``what kind of causal relationships exist between events in the environmental world'' and predictions about ``what will result from an intended action.'' , has been constructed based on a new hypothesis set by the inventors of the present application.

In other words, the intention model 14 functions as a "comparator" or "estimation" in the theoretical basic system of the present invention shown in FIG. It incorporates the function of determining and estimating whether the state is the same or not. By the way, the conditional probability of the resulting state s' when state s and action a are given, that is, how high the probability that state s' will occur is information included in the output of the intention model 14. However, it exactly reflects the correspondence relationship with the intended state.

Furthermore, the intention model 14 includes information regarding whether or not the state assumed by the user H and the actual state match, through the belief information received from the belief model 11, that is, information regarding the sense of agency level soa. is reflected.

<Behavioral model>
Also in FIG. 1, the behavioral model 15 is
(a) The policy π=<a1, a2,..., an> received from the desire model 12,
(b) Causal relationship information received from the intention model 14 (CBN configuration information)
Based on this, the model determines and outputs the action a to be taken for the observed state s, and in this embodiment includes an action planning section 151 and an action determining section 152.

Among these, the action planning department 151 is
The policy π of (a) above, the causal relationship information (CBN configuration information) of (b) above, and further,
(c) Through an interface (IF) that can input measurement results output from a measurement means such as a camera, or that can perform text input/output, voice input/output, etc., ask the user H a certain question. An optimal policy π*, which is the optimal policy, is generated based on the content of the communication included.

In this embodiment, the action planning unit 151 uses the well-known XAIP (eXplainable AI Planning agent) (Chakraborti et al., Proceedings of the Twenty-Eighth International Joint Conference on Artificial Intelligence IJCAI-19, pp.1335-1343, 2019). It is constructed using the action decision processing in . Here, in XAIP, an action planning problem P, a transition function ζ _P :S×A→S×C, and an action planning algorithm A: P×t→π are usually defined. Here, t is a quantity representing properties such as optimality and healthiness (different from τ of the present invention). In contrast, the inventors of the present application adopt the mental model Π of the user H instead of the action planning problem P to construct the action planning section 151.

Here, the mental model Π is the causal relationship information (CBN configuration information) received from the intention model 14 in this embodiment, and for example, in a situation where the user H is driving a car, the mental model Π is ``keep going on the current road X''. This model compiles the mental information of the user H, such as ``to carefully consider the time required to reach the destination W, avoid traffic jams, drive as fast as possible, and avoid getting off the train as much as possible.''

Specifically, the action planning unit 151 defines the mental model Π of the user H, the transition function ζ _Π : S×A→S×C, and the action planning algorithm PA: Π×DM→π*. Here, DM is a value/reward model that constitutes the desire model 12, and here, the policy π = < a1 which is a set of actions a (in state s) that can bring about the desired state sd (∈S) , a2,..., an>. Furthermore, π* is the optimal policy as described above.

This optimal policy π* is a set of actions that transition the actual state s to the desired state sd, that is, the transition function is the following equation (2) ζ _Π (π, s)=<sd, Σ _ai∈π ( ci＋ri)>
It becomes like this. Here, ci and ri are the cost and reward that should be optimized when executing the policy π, respectively. Note that one of the characteristics of the transition function ζ _Π in the above equation (2) _is that it includes a reward ri that is not adopted in the known XAIP.

In order to determine the optimal policy π* according to the action planning algorithm PA, the action planning unit 151 determines the state s(i+1) (=ζ _Π (π , si)), the policy optimization process is performed so that the state completely or approximately matches the desired state sd. A specific embodiment of this policy optimization process will be described below.

Basically, in this embodiment, the behavior planning unit 151 needs to maintain a model similar to the mental model of the user H, and therefore, the predicted behavior and mental model of the user H are consistent with the user's own behavior and model. If they do not match, "explanation" is provided to user H, and the matching of the models is proceeded.

For example, when user H (driver) "continues to drive on road "Road Y is not congested" may be adopted.

Here, as one method for "explanation", predictive reconciliation (inference reconciliation) is implemented. Specifically, the action planning algorithm PA ^χ (hereinafter, χ represents model 1 as AI) of the action planning unit 151 generates the policy π, whereas the action planning algorithm PA ^H of the user H generates the policy π. If the same policy π is not generated, the action planning unit 151 performs the explanation ε so that PA ^H generates the same policy π, that is, PA ^H : Π×D→ ^ε π.

This explanation ε can be, for example, communication content including specific questions and inquiries about the policy π generated from PA ^χ , which is made to the user H (his action plan algorithm PA ^H ). Here, this question/question is ``Why is a certain action a in policy π?'', ``Why is it in policy π and not another policy π'?'', or ``Why is policy π optimal? In other words, it is also preferable to take the form of an explanatory or persuasive dialogue about ``Why is π(s) a and not a'?'' Incidentally, such dialogue can be carried out via the above-mentioned (c) interface (IF) that allows input of measurement results output from measurement means such as a camera, or that allows text input/output, voice input/output, etc. -ing

Furthermore, apart from the above-described predictive arbitration, it may be considered to change the mental model Π ^H of the user H based on this explanation ε. Specifically, the user H may evaluate the policy π, which assumes that the action planning algorithm PA ^χ is optimal, with a completely different "spirituality." Therefore, the explanation ε is used so that the user H also agrees with the determined policy π. For example, if PA ^χ :Π×D→π, let H ^H be the mental model of user H that satisfies PA ^H :H ^H ×D→π, and then let the mental model Π ^H be , H ^H (that is, Π ^H +ε→H ^H ).

As yet another method, the action planning unit 151 can also implement explanation ε that emphasizes the difference between (the values in) the value sets τ generated under mutually different assumptions. Implementation of such explanation ε can be expressed as ε←τΔτ ^H between user H and sense of agency estimation model 1 (action planning unit 151), and between user H1 and user H2, ε It can be expressed as ←τ ^H1 Δτ ^H2 . Of these, the latter is compatible with the intention of sense of agency estimation model 1, which considers the value of user H1 to be higher than the value of user H2. It is also possible to achieve a balanced estimation of the target level of sense of agency.

Specifically, under the condition that PA ^χ : Π × D → π, in the interaction between user H and sense of agency estimation model 1, τ ^H +ε → τ' ^H and PA ^χ : Π × R × An explanation ε such that τ′ ^H →π is generated and implemented. Alternatively, while sense of agency estimation model 1 considers the value of user H1 to be higher than the value of user H2 (that is, PA ^χ :Π×R×τ ^H2 →π), τ ^H1 +ε→τ' ^H1 = τ ' An explanation ε that satisfies ^H2 may be generated and implemented.

Incidentally, as is clear from the above explanation, unlike the situation in the value matching model 13 described above, the action planning unit 151 is configured to perform actions that have higher problem-solving ability than (the action planning algorithm of) the user H. It has a planning algorithm. In other words, PA ^χ >PA ^H.

Above, we have explained several methods for AI to provide "explanation" to the user and achieve model matching, but in any case, the method in this embodiment is based on persuasive technology ) is based on the three basic principles. That is, as a first basis, the sense of agency estimation model 1 has high transparency in presenting action plans and estimation results to the user H, and the user H is not allowed to ask questions or request explanations. is now possible. Note that this makes the sense of agency estimation model 1 reliable for the user H, and it is expected that the relationship between the two will further improve.

As a second basis, the sense of agency estimation model 1 can explain to the user H that it understands the user H's behavior. Note that this makes it possible to increase the degree of empathy of the user H with the sense of agency estimation model 1. Furthermore, as a third basis, the sense of agency estimation model 1 collaborates with the user H to plan and estimate actions for the user H. Note that this also makes it possible to improve the sense of ownership level of the user H, which is the estimation target of the present invention.

Similarly, in FIG. 1, the behavior determining unit 152 of the behavior model 15 uses the optimal policy π* generated by the behavior planning unit 151 to determine and output the behavior a to be performed for the observed state s. Specifically, it executes π(s)=a. Incidentally, the actions determined here are those of the sense of agency estimation model 1 (when the sense of agency estimation model 1 is controlling as an autonomous AI) and the user H (when the user H is mainly controlling). , and both of the sense of agency estimation model 1 and the user H (in the case where the sense of agency estimation model 1 and the user H perform control cooperatively).

Here, if the action to be determined is the action of the sense of agency estimation model 1, the action a output from the action determination unit 152 is processed via a predetermined interface (IF) (for example, driving an actuator, displaying it on a display, etc.). , the state s of the world including the user H changes to the state s'.

In addition, the state s' observed in the world as a result of action a is defined as "the intended (predicted) state is due to action a carried out in the behavioral model (but not due to the actions of other agents)". It is fed back to the intention model 14 in order to determine whether or not.

By the way, if user H is the driver of a car traveling on road For example, new states such as "I am driving on road Y and heading towards destination W" or "What is the traffic situation on road Y that I am driving on?" or "Detour to road Y" Traffic jams will occur. Also, as a result, the "sensitivity level" of user H, which was initially "low", changes to "high", for example.

<Alternative state generation/evaluation model>
Also in FIG. 1, the alternative state generation/evaluation model 16 solves the problem of having to generate and evaluate new policies in situations that have not been seen in the past, are not predicted, or occur only rarely. This is a model for In other words, it can be viewed as a model that automatically generates intervention content for "intervention" to encourage behavior change.

Specifically, the alternative state generation/evaluation model 16 includes a state “generation” device (162) that generates an alternative state s ^a as a possible new state, and an “evaluator” that evaluates the generated alternative state s ^a ” (164) and the well-known “actor-critic framework” (Aras Dargazany, arXiv:2004.04574 Artificial Intelligence [cs.AI], 2020) (Zhewei Huang et al., arXiv:1903.04411 This is a model constructed based on Computer Vision and Pattern Recognition [cs.CV], 2019). Incidentally, this "actor-critic framework" is constructed using Generative Adversarial Networks (GAN) and Deep Reinforcement Learning (DRL) algorithms.

However, in this well-known "actor-critic framework," AI processes based on the assumption that the current environmental world can be accurately modeled, whereas・The evaluation model 16 employs knowledge from various contexts learned in the past and knowledge from multiple users' different mental models (CBNs) to make predictions (for user H). It is designed to learn about various situations that could occur but could not be foreseen, and to solve problems.

Similarly, in FIG. 1, the CBN aggregate 161 of the alternative state generation/evaluation model 16 includes CBN1, which is the intention model (mental model, causal relationship information) of each of a plurality of different users H1, H2, ..., Hp. , CBN2, ..., CBNp is an aggregate CBN _∪ (=CBN _{Hi} ), in other words, integrated causal relationship information. Note that at least some of CBN1, CBN2, .

Specifically, the set CBN _∪ is, for example, given CBN _H1 = (DAG<Y _H1 , E _H1 >, Prob _H1 ) and CBN _H2 = (DAG<Y _H2 , E _H2 >, Prob _H2 ) The following formula (3) CBN _∪ = (DAG _∪ <Y _H1 ∪Y _H2 , E _H1 ∪E _H2 >, Prob _H1 ∪Prob _H2 )
It can be expressed as Here, DAG<Y, E> refers to a directed acyclic graph composed of nodes (variables) Y and edges E, and Prob is a transition probability corresponding to each edge E.

Also in FIG. 1, the state generator 162 of the alternative state generation/evaluation model 16 is
(a) Receiving the observed state s and the corresponding action a outputted (from the action determining unit 152),
(b) Based on the collection of intention models CBN _∪ (integrated causal relationship information) received from the CBN collection 161,
Generate and output an alternative state s ^al as a possible state candidate.

Here, the alternative state s ^al is a new state that the user H did not predict/expect, or did not think could occur, for example, a more suitable alternative solution action for an unknown problem that has not been seen in the past. It is used to determine the

By the way, the state generator 162 corresponds to the generation part of a well-known generative adversarial network (GAN), but instead of generating fake data as in the past, it solves a new problem. It generates alternative ^states to generate new strategies for doing.

Similarly, in FIG. 1, the discriminator 163 of the alternative state generation/evaluation model 16 prevents the alternative state s ^al generated by the state generator 162 from being so fictitious that it cannot exist in the context where user H is present. Determine whether or not there is. In other words, the discriminator 163 determines whether the generated new alternative state s ^al can be useful for solving an actual problem, and generates the state so that the alternative state s ^al becomes such a state. This is to encourage training of the vessel 162.

Specifically, in this embodiment, the discriminator 163 is configured with a deep neural network (DNN) algorithm including a plurality of fully connected layers, and the discriminator 163 is configured with a deep neural network (DNN) algorithm that includes a plurality of fully connected layers. A loss ALoss representing the difference from the desired state sd is generated from the state s' and the alternative state s ^a (generated by the state generator 162), and with this loss ALoss, (a) (b) Train itself (discriminator 163).

Here, in a general generative adversarial network (GAN), the adversarial loss is the degree of difference between the current state s' and the generated state sg, that is, max _ψ (Ex _s~μ [ψ( s')]-Ex _sg~ug [ψ(sg)]) is calculated. Here, Ex is the expected value, and μ and ug are the sample probability distribution of the current state and the sample probability distribution of the generated state, respectively.

On the other hand, in this embodiment, the discriminator 163 outputs the scalar adversarial loss ALoss itself, and unlike the conventional method,
(a) ALoss(s'): as the loss between the current state s' and the desired state sd, and (b) AL(s ^al ): as the loss between the alternative state s ^al and the desired state sd. , ALoss(s') and AL(s ^al ).

Also in FIG. 1, the evaluator 164 of the alternative state generation/evaluation model 16 generates a predicted reward that is a reward corresponding ^to the alternative state s a generated by the state generator 162 (trained by the adversarial loss ALoss). do. Here, this predicted reward no longer includes the reward calculated by the behavior a output from (the behavior determining unit 152 of) the behavior model 15.

The evaluator 164 also causes the behavior model 15 (the behavior determining unit 152 thereof) to perform training on behavior determination using this predicted reward. This makes it possible to update the behavior decision processing in the behavior model 15 (the behavior decision unit 152 thereof) so that it can be applied to situations that have not been seen in the past, are not predicted, or occur only rarely. be.

Specifically, the predicted reward at time t can be a Q-learning value function Q(s ^al _t ) of reinforcement learning. This Q learning value function Q(s ^al _t ) is calculated by the following formula (4) Q(s ^al _t )=r(s ^al _t , a _t )+γQ(s ^al _t+1 )
will be updated as a discounted reward. Here, r(s ^al _t , at) is the reward for performing action a _t under state s ^al _t .

Next, in the present embodiment, the behavior determining unit 152 of the behavior model 15 uses this predicted reward (Q learning value function) Q(s ^al _t ) to convert π*(s) for deriving behavior a into r( We train to maximize s ^al _t , π*(s ^al _t ))+Q(ζ(s ^al _t , π*(s ^al _t ))). Here, the transition function ζ(s ^al _t , π*(s ^al _t )) becomes the alternative state s ^al _t+1 at time t+1. Such an alternative state s ^al _t+1 is created by sense of agency estimation model 1 when user H is in a predicament where it is impossible to answer the posed question due to the limit of his/her ability. It can also be seen as an answer presented. Furthermore, presenting such an answer to the user H also contributes to increasing the reliability of the user H's sense of agency estimation model (behavior change promotion model) 1.

<Sense of ownership model>
Similarly, in FIG. 1, in this embodiment, the sense of agency model 17 estimates and outputs real-time fluctuations in the sense of agency (SoA) level that depends on the dynamic context as a behavioral phenotype.

In a psychological experiment conducted in the past (Tapal, A. et al., Frontiers in Psychology, 8, Article 1552. 2017, <https://doi.org/10.3389/fpsyg.2017.01552>), certain Sense of agency is directly measured through self-reports (self-reports), which are the results of a person's direct evaluation of their sense of agency (SoA) regarding an event.

In the past, there have also been examples of direct measurement of a person's sense of agency, using the perceptual difference between a person's own measured value and an externally measured value of fluctuations in their sense of agency. However, both methods require a direct response from the person being measured regarding their sense of agency, which forces the person being measured to interrupt their behavior intermittently, resulting in significant fluctuations in the level of their sense of agency. It has been difficult to apply it in situations where

In contrast, the inventors of the present application have found that changes in the sense of agency level soa are influenced by physiological indicators (e.g., heart rate and breathing rate), posture, gestures, and speech prosody indicators ( For example, we have found that this phenomenon clearly appears in temporal changes in tone, accent, intonation, speech rate, speech pitch, speech volume, etc. (this is a hypothesis that works). Here, these measurement results have conventionally been effectively used to estimate emotional states (including emotions and moods).

In addition, in the field of so-called Affective Computing, AI is used to recognize a subject's emotions from information on behavioral and attitude expressions obtained from wearable sensors and environmental sensors, and to make adaptations to emotions. Research is being actively conducted to explore the response. Furthermore, there are some studies that have proven that people's sense of agency and emotions constantly interact in daily life (for example, Matthis Synofzik et al., Front. Psychol., 4( 127), 2013 <https://doi.org/10.3389/fpsyg.2013.00127> and Antje Gentsch1 and Matthis Synofzik, Front. Hum. Neurosci., 8:608, 2014 <https://doi.org/10.3389/ fnhum.2014.00608> etc.).

For example, sense of agency is influenced by emotional factors, such as whether the expectations for the emotional outcome of the action are positive or negative, whether the motivation for performing the current action is high or low, and whether the action is performed in a friendly environment. Research results show that it can be modulated depending on whether the person is in a hostile environment or in a hostile environment (Julia F Christensen et al., Exp Brain Res. 237(5), 1205-1212, 2019 <https://doi.org/10.1007/ s00221-018-5461-6>) is also disclosed.

In order to formulate the findings and findings explained above, the sense of agency model 17 uses a sense of agency recognition function Ω that outputs the current sense of agency level soa: ρ1×ρ2×ρ3×ρ4×...→S stipulates. Here, ρ1, ρ2, . , speech rate, speech pitch, speech amount, etc.).

After defining such a sense of agency recognition function Ω, the sense of agency model 17 is specifically constructed using a deep neural network (DNN) algorithm,
(a) A new state s' caused by the action a output from the action determining unit 152,
(b) receiving the feature amounts related to the predetermined feature ρ of the user H under this new state s' (affected by the new state s'), and changing these new state s' and the feature amounts; Based on ρ, the user H's sense of agency level (soa) is determined or updated and output. Furthermore, the sense of subjectivity level soa used in the belief model 11 is updated to the determined or updated sense of subjectivity level soa'. This makes it possible, for example, to estimate or anticipate the causes and characteristics of interruptions or discontinuities in the sense of agency.

Incidentally, the sense of agency level soa' output from the sense of agency model 17 updates the "belief information" of the belief model 11, and furthermore, this updated belief information is used as the "causal relationship information" (CBN configuration information) of the intention model 14. ) is exactly the embodiment of the conventional psychological theory that ``perceived control,'' which is related to a sense of agency, influences ``intention.'' It has become a thing.

Here, the sense of agency model 17 is
(c) Self-report (self-report) regarding user H's sense of agency received from value matching model 13 soa_r
It is also preferable to determine or update the sense of agency level (SOA) of the user H based on the information and output it. This is because the value integration model 13, as part of the processing flow of CIRL (Collaborative Inverse Reinforcement Learning), tells User H the level of sense of agency (when there is doubt about the perceived sense of agency). This process is performed when a self-report soa_r related to user H's sense of agency is obtained as a result of the inquiry.

The process of generating and updating the sense of agency level soa in the sense of agency model 17 has been explained above, but the sense of agency estimation model 1 can thereby, for example, accurately capture and understand the sense of agency level occurring in the user H. It is also possible to explain to the user H what the user is doing, and as a result, it becomes possible to further demonstrate to the user H the high level of empathy and reliability. Furthermore, sense of agency estimation model 1 can also convince user H that the proposed policy can only improve the level of user H's sense of agency, not reduce it. In this way, it is also possible to provide user H with a smart and sensible response.

[Sense of agency estimation device, sense of agency estimation program]
Hereinafter, returning to FIG. 1, a description will be given of a sense of agency estimation device 9 that is equipped with the sense of agency estimation model 1 as described above and that estimates the sense of agency of the user H who is the user to be estimated. Incidentally, with a similar configuration, this device can also be used as a behavior change promotion device equipped with a behavior change promotion model (sense of agency estimation model) 1.

The sense of agency estimation device 9 of this embodiment shown in the lower left part of FIG. 1 estimates the sense of agency of the user H, who is the estimation target, from the observed state in the environmental world using the installed sense of agency estimation model 1. Specifically, it is a device that determines the user H's sense of agency level.

Specifically, in FIG. 1, a user interface (IF) 91 of the sense of agency estimation device 9 includes an interface (IF) of the value matching model 13, an interface (IF) of the action planning unit 151, and an interface (IF) of the behavior model 15 (action determining unit). 152), and has the role of incorporating measurement results related to user H, inputting and outputting information related to various communications with user H, and furthermore, making determined actions act on the environmental world. fulfill. It also serves as an input unit that collects information necessary for training the sense of agency estimation model 1, such as the state of the environmental world, and information that is a condition for estimating a sense of agency.

The training unit 92 generates training data from the received information necessary for training the sense of agency estimation model 1, and uses this to train the sense of agency estimation model 1.

The sense of agency estimation unit 93 determines the sense of agency level of the user H using the trained sense of agency estimation model 1 based on the received information serving as a condition for estimation of a sense of agency. In this embodiment, it is possible to determine and output user H's real-time sense of agency level and dynamic fluctuations in user H's sense of agency level even in complex or ever-changing environmental world situations. It becomes.

The output unit 94 transmits information related to the determined sense of ownership level to an external information processing device (if equipped with a communication function) or displays it (if equipped with a display function). .

Here, the training unit 92 and the sense of agency estimation unit 93 are main functional components that implement an embodiment of the sense of agency estimation method according to the present invention, and also store an embodiment of the sense of agency estimation program according to the present invention. It can also be seen as a function of the processor and memory. Further, from this, the sense of agency estimation device 9 may be a dedicated device for estimating a sense of agency, but may be a cloud server, a non-cloud server device, a personal computer ( It is also possible to use a PC), a notebook or tablet computer, a smartphone, a wearable computer, or the like.

As explained in detail above, according to the present invention, it is possible to estimate a user's sense of agency, which has been difficult to estimate in the past (especially in a complex and constantly changing environment). Furthermore, by determining the user's behavior toward the environmental world in consideration of the user's appropriately updated sense of agency, it is also possible to encourage the user to change their behavior, that is, change their intentions or actions.

For example, by adopting an appropriate embodiment, it is possible to understand the actions that the user has taken of his or her own will (in response to suggestions and persuasion from the sense of agency estimation model according to the present invention) and the state of the environmental world that appears as a result. During this time, it is possible to make appropriate suggestions and persuasion by improving the user's sense of agency, so that the user feels connected and interlocked, without diminishing the user's sense of agency.

In addition, since the present invention has the effects described above, it is also useful for improving and developing the content of mutual understanding and collaborative activities between humans and AI that will be seen in various situations in the future. It is thought that it will also make a significant contribution.

Furthermore, in order to provide children with high-quality education that respects children's independence and motivation to study, for example, the sense of agency estimation model and behavior change promotion model of the present invention can be used to It is also possible to implement educational actions, including suggestions and guidance, that maintain and improve a sense of ownership. In other words, the present invention contributes to Goal 4 of the Sustainable Development Goals (SDGs) led by the United Nations: “Provide inclusive, equitable and quality education for all and promote lifelong learning opportunities.” It is also possible to do so.

Furthermore, in order to provide adults with decent work (jobs that are rewarding and humane), for example, the sense of agency estimation model and behavior change promotion model of the present invention can be used to maintain and improve adults' sense of agency. It is also possible to provide advice on how to get a job or on the job, such as encouraging adults to change their behavior at work. In other words, according to the present invention, it is also possible to contribute to Goal 8 of the SDGs led by the United Nations, "Promote inclusive and sustainable economic growth, employment and decent work for all." be.

Furthermore, in order to reliably and smoothly achieve the goals of automobile drivers driving in urban areas, for example, the sense of agency estimation model and behavioral change promotion model according to the present invention can be used to improve the driver's agency. It is also possible to implement navigation that does not reduce the feeling of navigation, that is, that is easy for drivers to understand. In other words, according to the present invention, it is also possible to contribute to Goal 11 of the SDGs led by the United Nations: "Making cities inclusive, safe, resilient and sustainable."

Furthermore, in order to provide consumers with sustainable consumption and lifestyles, for example, the sense of agency estimation model and behavior change promotion model of the present invention can be used to prevent consumers' sense of agency from diminishing. It is also possible to provide advice and suggestions on consumer behavior that are easy for consumers to understand. In other words, according to the present invention, it is also possible to contribute to Goal 12 of the SDGs led by the United Nations, "Ensure sustainable consumption and production patterns."

Regarding the various embodiments of the present invention described above, various changes, modifications, and omissions within the scope of the technical idea and viewpoint of the present invention can be easily made by those skilled in the art. The above description is merely an example and is not intended to be limiting in any way. The invention is limited only by the claims and their equivalents.

1 Sense of agency estimation model (behavioral change promotion model)
11 Belief model 12 Desire model 13 Value matching model 14 Intention model 15 Behavior model 151 Action planning unit 152 Behavior determining unit 16 Alternative state generation/evaluation model 161 CBN (Causal Bayesian Network) aggregate 162 State generator 163 Discriminator 164 Evaluation Device 17 Sense of agency model 9 Sense of agency estimation device 91 User interface (user IF)
92 Training section 93 Sense of agency estimation section 94 Output section

Claims (10)

A sense of agency estimation model that operates a computer to estimate a user's sense of agency while determining behavior regarding the state of the environmental world including the user using rewards,
a belief model that generates or updates belief information that is information that includes the probability that a new state will occur as a result of performing a certain action for a certain state under a certain sense of agency level of the user;
Receive value information related to value for the user and the corresponding reward, generate a desired state that is the state desired by the user, and determine a policy that is a set of actions that can bring about the desired state. Desire model and
an intention model that generates causal relationship information regarding a causal relationship between states, values, rewards, and actions based on the belief information, the value information, and the reward;
Based on the policy, the cause-and-effect relationship information , and the content of the communication including the predetermined questions conducted with the user , an optimal policy that is the optimal policy is generated, and using the optimal policy, a behavior model that determines and outputs actions to be taken in response to observed conditions;
Determining or updating the sense of agency level of the user based on the output new state caused by the action and the feature amount related to the predetermined characteristics of the user under the new state, and outputting the determined level. , a sense of agency estimation model characterized in that a computer functions as a sense of agency model that updates the sense of agency level used in the belief model to the determined or updated sense of agency level. Receive information related to the value information and information related to the reward corresponding to the information related to the value information from the user, and based on the information related to the value information and the information related to the reward, 2. The sense of agency estimation model according to claim 1, wherein the computer further functions as a value matching model that generates or updates the corresponding reward and outputs it to the desire model. 3. The sense of agency estimation model according to claim 2 , wherein the value matching model is constructed using an algorithm related to cooperative inverse reinforcement learning (CIRL). Receive the observed state and the corresponding output action, and select an alternative state as a possible state candidate based on integrated causal relationship information that integrates the causal relationship information for each of at least a plurality of users. a state generator that generates and outputs;
A loss representing the difference from the desired state is generated from the new state caused by the action and the alternative state, the state generator is trained using the loss, and the state generator is trained using the loss. A discriminator that performs
an evaluator that generates a predicted reward that is a reward corresponding to the alternative state generated by the trained state generator, and uses the predicted reward to train the behavior model to determine the behavior. 4. The sense of agency estimation model according to claim 1 , further comprising a computer functioning as an alternative state generation/evaluation model. 5. The sense of agency estimation model according to claim 4 , wherein the alternative state generation/evaluation model is constructed using an algorithm related to generative adversarial networks (GAN). The sense of agency estimation according to any one of claims 1 to 5 , wherein the belief model is constructed using an algorithm related to a Partially Observable Markov Decision Process (POMDP). model. 7. The sense of agency estimation model according to claim 1, wherein the causal relationship information is information related to a Bayesian network algorithm. 8. A sense of agency estimation device for estimating a sense of agency of the user from an observed state in the environmental world using the sense of agency estimation model according to any one of claims 1 to 7 . A sense of agency estimation method in which a computer estimates a user's sense of agency while determining behavior regarding the state of the environmental world including the user using rewards, the method comprising:
generating or updating belief information, which is information that includes a probability that a new state will occur as a result of performing a certain action on a certain state under a certain sense of agency level of the user;
Receive value information related to value for the user and the corresponding reward, generate a desired state that is the state desired by the user, and determine a policy that is a set of actions that can bring about the desired state. step and
a step of generating causal relationship information regarding a causal relationship between a state, a value, a reward, and an action based on the belief information, the value information, and the reward;
Based on the policy, the cause-and-effect relationship information , and the content of the communication including the predetermined questions conducted with the user , an optimal policy that is considered to be the optimal policy is generated, and the generated optimal policy is a step of determining and outputting an action to be taken for the observed state using the method;
Determining or updating the sense of agency level of the user based on the output new state caused by the action and the feature amount related to the predetermined characteristics of the user under the new state, and outputting the determined level. A method for estimating a sense of agency, comprising: updating the sense of agency level used in the step of generating or updating the belief information to the determined or updated sense of agency level . A behavior change promotion model that operates a computer that promotes behavior change in a user while determining behavior regarding the state of the environmental world including the user using rewards,
a belief model that generates or updates belief information that is information that includes the probability that a new state will occur as a result of performing a certain action for a certain state under a certain sense of agency level of the user;
Receive value information related to value for the user and the corresponding reward, generate a desired state that is the state desired by the user, and determine a policy that is a set of actions that can bring about the desired state. Desire model and
an intention model that generates causal relationship information regarding a causal relationship between states, values, rewards, and actions based on the belief information, the value information, and the reward;
Based on the policy, the cause-and-effect relationship information, and the content of the communication including the predetermined questions conducted with the user, an optimal policy that is the optimal policy is generated, and using the optimal policy, a behavior model that determines and outputs actions to be taken in response to observed conditions;
Based on the output new state caused by the action and the feature amount related to the predetermined characteristics of the user under the new state, the sense of agency level of the user is determined or updated, and the belief model is A behavioral change promotion model characterized in that a computer functions as a sense of agency model that updates a sense of agency level used in the above to a determined or updated sense of agency level.

Images