JP7139270B2 - Estimation device, estimation method and estimation program - Google Patents

Description

The present invention relates to an estimating device, an estimating method, and an estimating program.

In recent years, techniques for performing evaluation according to user's emotions have been known. As an example of such information distribution, there is known a technique of estimating a user's emotion from biometric information and evaluating a product or content viewed by the user based on the estimated emotion.

Japanese Unexamined Patent Application Publication No. 2006-012171 JP 2006-313483 A

However, it may not be possible to say that the user's emotion is properly estimated by the conventional technology described above.

For example, the user's emotion is only evaluated based on the momentary biometric information and the corresponding emotion. It is conceivable that the person may have the emotion indicated by the biometric information, or may not yet have the emotion indicated by the biometric information. As a result, when estimating instantaneous biological information and corresponding emotions, there is a risk that the user's emotions cannot be estimated appropriately.

This application was made in view of the above, and an object of this application is to estimate a user's emotion appropriately.

An estimating device according to the present application includes an acquisition unit that acquires behavior information indicating behavior of a predetermined user, and based on the behavior information acquired by the acquisition unit, determines a series of behaviors and a causal relationship of the predetermined user. and an estimating unit for estimating the emotion.

According to one aspect of the embodiment, it is possible to appropriately estimate the user's emotion.

FIG. 1 is a diagram illustrating a configuration example of a network system according to an embodiment. FIG. 2 is a diagram illustrating an example of estimation processing according to the embodiment. FIG. 3 is a diagram illustrating a configuration example of an estimation device according to the embodiment; 4 is a diagram illustrating an example of a learning data storage unit according to the embodiment; FIG. FIG. 5 is a flowchart showing an estimation processing procedure by the estimation device according to the embodiment. FIG. 6 is a diagram illustrating an example of a hardware configuration;

Embodiments for implementing the estimation device, estimation method, and estimation program according to the present application (hereinafter referred to as "embodiments") will be described in detail below with reference to the drawings. Note that the estimation device, estimation method, and estimation program according to the present application are not limited by this embodiment. Further, each embodiment can be appropriately combined within a range that does not contradict the processing contents. Further, in each of the following embodiments, the same parts are denoted by the same reference numerals, and overlapping descriptions are omitted.

[1. Network system configuration]
First, the configuration of a network system 1 according to an embodiment will be described with reference to FIG. FIG. 1 is a diagram showing a configuration example of a network system 1 according to an embodiment.

As shown in FIG. 1, a network system 1 according to the embodiment includes user devices 10 ₁ to 10 _n and an estimating device 100 (n is any natural number). In this specification, the user devices 10 ₁ to 10 _n are collectively referred to as "user devices 10" when there is no need to distinguish between the user devices 10 ₁ to 10 _n . The user device 10 and the estimation device 100 are each connected to the network N by wire or wirelessly. Although not shown in FIG. 1, the network system 1 may include multiple estimating devices 100 .

The user device 10 is an information processing device used by a user. The user device 10 may be any type of information processing device including a desktop PC (Personal Computer), a notebook PC, a smart phone, and a tablet PC.

Estimation device 100 is an information processing device that estimates a user's emotion. The estimation device 100 communicates with the user device 10 via the network N by wire or wirelessly.

[2. Estimation process]
Next, an example of estimation processing according to the embodiment will be described with reference to FIG. FIG. 2 is a diagram illustrating an example of estimation processing according to the embodiment.

Conventionally, various evaluation indices have been proposed for evaluating services on networks. For example, evaluation indexes such as CTR (Click Through Rate) and PV (Page Views) have been proposed. Since evaluation indices such as CTR and PV are clearly defined, service providers can use such evaluation indices to objectively evaluate the content of services on the network.

On the other hand, in recent years, something relatively undefined such as "emotion" has also been proposed as an evaluation index for evaluating services on networks. Emotions can be thought of as subjective conscious experiences such as happiness and disgust. Emotions of users using the network are, for example, sensor data (e.g., physiological reactions occurring in the user), text data (e.g., sentences written by the user), voice data (e.g., user's utterances), images, and so on. It may be estimated from data such as data (eg, user's facial expression).

With respect to emotion estimation above, generally the user's emotion is estimated from the data at "moments".

For example, a flag indicating the user's emotion is attached as a correct label to sensor data at a certain time. For example, an emotion (eg, "happiness") is associated with a physiological response (eg, heart rate, body temperature) at a certain time. Also, for example, other emotions (eg, "disgust") are associated with physiological responses at other times. In such a case, a model for estimating the user's emotion is generated by performing machine learning using learning data including sensor data and correct labels. When sensor data indicating a certain target user's heart rate and body temperature is input to the generated model, the generated model will receive data indicating this target user's emotion (for example, the user's emotion is "happiness"). probability) can be output.

In other words, data indicative of emotions at a given moment are generally created in order to generate a model that estimates the user's emotions. Such data is thus considered to be a snapshot of the user's emotions. In general, a model for estimating a user's emotions is generated based on such emotional snapshots.

Thus, in general emotion estimation, a user's emotion is defined as the conscious experience at a given moment. In this case, the user's emotion is associated with an instance at a certain time and a feature vector at a certain time. However, a user's emotions are not necessarily conscious experiences at a given moment. For example, a user's emotions can be thought of as a conscious experience in a certain "process." From the perspective of such effective theory of emotion and action, even if the user's emotion corresponds to an instance at "each time", the user's emotion also corresponds to an instance at a "predetermined time period". obtain.

As an illustrative example, assume that a user's behavior is a thick bundle of fine behavior lines. One thick bundle of fine action lines is a bundle of a plurality of action lines. For example, user actions correspond to actions such as "search" and "buy". In this case, the detailed action lines correspond to the content of the screen being viewed by the user (that is, the action of viewing) and actions such as user operations. In this example, the user's emotion is considered to be the conscious experience of the user's course of action (ie, the user's course of action).

In other words, it is considered inappropriate to define the user's emotion by the above-described "snapshot of emotion" from the viewpoint of the above-described "bundle of lines of behavior". On the other hand, from the viewpoint of the above-mentioned "effective theory of emotion and action", it is considered appropriate to define the user's emotion as a "bundle of lines of a plurality of actions". For example, a user's emotions can be thought of as the result of one big bundle of actions. More specifically, the user's emotions are considered to be determined according to a series of actions of the user.

For example, when a person asks a user, "How are you feeling at this moment?", this user may not be able to properly grasp his/her emotions. On the other hand, if a person asks a user, "How are you feeling at that time?" In this case, it can be said that such a user's emotion is close to the "atmosphere".

Thus, in the exemplary embodiment of FIG. 2, the estimating device 100 performs an estimating process described below to analyze the user's emotions.

In an example of the estimation process described below, the estimation device 100 performs a series of actions of the user (for example, contents of the screen being viewed by the user (that is, action of browsing), web actions such as user operations). and the causal relationship with this user's emotion. In addition, in this example of the estimation process, after the estimation device 100 analyzes the causal relationship, the estimation device 100 uses the causal relationship analysis result to provide the content to the user.

Regarding the analysis of the causal relationship described above, the estimation device 100 unravels the series of actions of the user as the above-described “bundling of a plurality of behavior lines”, Based on presence or absence, the causal relationship between the user's series of actions and the user's emotions is examined. In this way, the estimating apparatus 100 separates individual user behavior (for example, viewing behavior regarding individual products, purchasing behavior regarding individual products) under a situation in which the user has taken a series of actions, and determines the emotions of the user. (i.e. emotions that are close to mood).

In the example of FIG. 2, first, the estimation device 100 acquires user behavior information (step S11).

Although not shown in FIG. 2, the estimating device 100 provides services (for example, web services) on the network such as search, portal, SNS (Social Network Service), auction service, map information service, etc. to the user. may provide. Then, the estimation device 100 may receive the user's behavior information from the user's user device 10 via a service on the network. Thus, the estimation device 100 can acquire the user's behavior information.

The obtained user information indicates user behavior on the network. For example, the acquired behavior information includes the click history, the history of browsed content, the selection history of advertisements, the posting history of SNS, the exhibition history of auctions, the winning bid history of auctions, the purchase history, the location history (for example, GPS (Global Positioning System) history, beacon detection history), etc.

Next, the estimating device 100 identifies the timing at which the user purchased a predetermined transaction object (for example, a product or service) based on the acquired behavioral information, and identifies the identified timing from the acquired behavioral information. Behavior information indicating the behavior of the user during a predetermined period of time is determined (step S12).

In the example of FIG. 2, the determined action information is shown as data indicating actions in a predetermined period, such as action UA11, action UA12, action UA13, action UA14. Also, the determined behavior information indicates that the user has purchased a predetermined product in a predetermined period.

For example, the determined action information may indicate that the user took action UA11, action UA12, action UA13, action UA14, and then purchased a predetermined product. Further, for example, the determined action information may indicate that the user took action UA11, action UA12, purchased a predetermined product, action UA13, and then action UA14.

Next, the estimating apparatus 100 estimates the user's emotion during the predetermined period described above, and generates learning data (that is, training data) including the estimated user's emotion and the determined behavioral information (step S13). ).

For illustrative purposes, the example of FIG. 2 assumes that the user is utilizing a sensing device, such as a wearable device. In this example, the estimating apparatus 100 uses the sensing device to identify the user's physiological response (for example, changes in heart rate or body temperature). The sensing device may be the user equipment 10 . Estimation device 100 can then estimate the user's emotion during the above-described predetermined period based on the identified physiological response. For example, the estimation device 100 estimates emotions such as “happiness”, “surprise”, and “fear” (for example, six basic emotions) as the emotions of the user during a predetermined period based on the specified physiological reaction. do. Such emotions may be emotions associated with the purchase of a transactional object. For example, the estimation device 100 may estimate positive emotions such as happiness, joy, passion, fun, etc., as the user's emotions during a predetermined period.

In this way, the estimation device 100 can associate the estimated user's emotion with the determined behavioral information. As a result, estimation device 100 can generate learning data including the estimated user's emotion and the determined behavioral information.

It should be noted that the user's emotion during the predetermined period described above is, for example, the user's emotion during the predetermined period. The user's emotion during the predetermined time period described above may be the user's emotion after the predetermined time period. That is, the user's emotion in the above-described predetermined period may be the emotion immediately after the series of actions.

Estimation apparatus 100 then performs MIL (Multiple Instance Learning) based on the generated learning data to generate a prediction model that predicts the user's emotion (step S14).

MIL, described above, is a method of supervised learning. MIL predicts the label of "an instance with an unknown label" or "one or more instances with an unknown label" using learning data in which one or more instances are associated with one correct label. refers to the method. An instance is also called a feature vector.

For example, in MIL, one correct label is associated with one or more tuples corresponding to one or more instances. More specifically, one or more instances are included in a positive bag or a negative bag. A positive bag contains one or more positive instances. A negative bag, on the other hand, contains only one or more negative instances. In this regard, in general supervised learning, one correct label is associated with one instance (ie, one tuple).

In MIL, the dimensions of instances are set in advance, as in general supervised learning. That is, an instance is defined on an N-dimensional instance space (N is the number of dimensions). However, as noted above, in MIL one correct label is associated with one or more instances. Therefore, in MIL, the number of instances (that is, the number of tuples) used as input for training data is not fixed. In other words, the cardinality can be different for different bags (ie positive or negative bags).

Regarding the generation of the prediction model, MIL generates a prediction model that predicts labels using learning data in which one correct label is associated with one or more of the instances described above.

For example, an EMDD (Expectation Maximization Diverse Density) technique generates a probabilistic model that outputs the posterior probability that a given instance is a positive instance. Such probabilistic models can also be generated, for example, by the MIL boost technique. Also, for example, a classification model that classifies a certain instance into either a positive instance or a negative instance is generated by a SVM (Support Vector Machine) technique (eg, miSVM, MISVM).

Also, for example, the SVM approach can generate a classification model that classifies a bag into one of a positive bag or a negative bag. More specifically, an SVM approach using a distance function based on the distance between bags in bag space (eg, the Hausdorff distance) can generate such a classification model.

Note that in MIL, labels can be predicted without creating a prediction model to predict the labels. For example, in MIL labels can be predicted by pattern classification algorithms. For example, according to the Citation k Nearest Neighbor (CKNN) approach, a bag may be classified as one of a positive bag or a negative bag.

In the example of FIG. 2, one user's action corresponds to one instance described above.

For example, the eleventh component of an instance may correspond to a scroll operation. Also, for example, the twelfth component of an instance may correspond to a gesture. Also, for example, the thirteenth component of the instance may correspond to sensor data (eg, motion of the user device 10).

Also, for example, the 21st component of the instance may correspond to the viewed content. Also, for example, the 22nd component of the instance may correspond to the selected content. Also, for example, the 23rd component of the instance may correspond to the retrieved content.

Also, for example, the 31st component of an instance may correspond to a purchased trade object. Also, for example, the 32nd component of an instance may correspond to the cost paid (eg, the amount paid by the user to the service provider). Also, for example, the 33rd component of the instance may correspond to the service used.

Also, in the example of FIG. 2, one emotion is associated with one or more user actions. In other words, this one emotion corresponds to the above label. Also, one or more user actions are included in this one emotional bag (eg, positive bag or negative bag).

For example, action UA11, action UA12, action UA13, and action UA14 may be associated with the emotion "happiness." Also, for example, the action UA21 and the action UA22 (not shown) may be associated with the emotion "passion". Also, for example, the action UA31, the action UA32, and the action UA33 (not shown) may be associated with the emotion "enjoyment".

In other words, action UA11, action UA12, action UA13 and action UA14 may be included in the positive bag of the emotion "happiness". In this case, for example, action UA41 and action UA42 (not shown) may be included in the negative bag of the emotion "happiness". The positive bag for the emotion Happiness contains one or more actions corresponding to the emotion Happiness. On the other hand, the negative bag for the emotion Happiness contains only behaviors that do not correspond to one or more emotions Happiness.

That is, the positive bag of the emotion “happiness” indicates that the user experienced the emotion “happiness” during the period in which the user took action UA11, action UA12, action UA13, and then action UA14. . On the other hand, the negative bag for the emotion “happiness” indicates that the user did not experience the emotion “happiness” during the period in which the user took action A41 and then took action A42. In this way, MIL is applied to the domain of sentiment analysis.

For illustration purposes, in the example of FIG. 2, it is assumed that the estimating apparatus 100 performs MIL based on the generated learning data and generates an SVM that classifies the user's emotion as a prediction model that predicts the user's emotion. do. In this example, when behavior information indicative of one or more user behaviors is input into the generated SVM, the generated SVM may be Classify user sentiment.

For example, the generated SVM classifies one or more user actions into actions corresponding to the emotion "happiness". Also, for example, the generated SVM classifies the actions of another one or more users into actions that do not correspond to the emotion "happiness".

Next, the estimation device 100 acquires behavior information of the target user (step S15).

The above-mentioned target user is a user who is the target of emotion estimation using the above-mentioned MIL. Although not shown in FIG. 2, for example, the estimation device 100 may receive the target user's behavior information from the target user's user device 10 via the service on the network described above. In this way, the estimation device 100 can acquire behavior information of the target user.

Next, the estimating device 100 inputs the acquired behavior information of the target user into the generated prediction model, thereby estimating the emotion of the target user and a series of behaviors of the target user having a causal relationship with this emotion. (step S16).

For illustration purposes, in the example of FIG. 2 , the estimating device 100 generates a first Assume that the target user's behavior TUA11 and behavior TUA12 are classified into behaviors corresponding to the emotion "happiness". Further, assume that the behavior TUA11 and the behavior TUA12 of the first target user correspond to the behavior "investigate trading object". In this example, the estimating device 100 estimates the action of “examining the transaction target” as a series of actions having a causal relationship with the emotion “happiness”.

Furthermore, in the example of FIG. 2 , the estimation device 100 inputs a plurality of instances respectively corresponding to the behavior TUA21 and the behavior TUA22 of the second target user to the generated prediction model, thereby Assume that action TUA21 and action TUA22 are classified into actions corresponding to the emotion "enjoyment". Further assume that the behavior TUA21 and the behavior TUA22 of the second target user correspond to the behavior "purchase the transaction object". In this example, the estimation device 100 estimates the action of "purchasing a transaction target" as a series of actions having a causal relationship with the emotion of "enjoyment".

Thereafter, the estimating apparatus 100 determines whether a combination of the estimated "target user's emotions" and the estimated "target user's series of actions" satisfies a predetermined condition, and the combination satisfies the predetermined condition. is provided to the target user (step S17).

For purposes of illustration, in the example of FIG. 2, the predetermined condition for providing advertisements to the target user is "the combination of the estimated target user's emotion and the estimated target user's set of behaviors is happiness. , positive emotions such as joy, passion, and enjoyment; the behavior of encountering transaction objects; the behavior of examining transaction objects; the behavior of purchasing transaction objects; It is assumed that it is a condition that it is a combination with behavior. In this example, the estimating device 100 determines that the combination of the first target user's action of "examining transaction objects" and the first target user's emotion of "happiness" is a predetermined value for providing advertisements to the target user. It is determined that the conditions are satisfied. Further, the estimation device 100 determines that the combination of the second target user's action of "purchasing the transaction object" and the second target user's emotion of "enjoyment" is a predetermined condition for providing the target user with the advertisement. is determined to be satisfied. Accordingly, the estimating device 100 provides advertisements to the first target user and the second target user.

As described above, the estimation device 100 according to the embodiment analyzes the causal relationship between the user's series of actions and the user's emotions. Then, the estimation device 100 uses the causal relationship analysis result to provide the user with content (for example, an advertisement). Thereby, the estimation device 100 enables the service provider to promote the use of the service.

For example, the service cycle includes the process of encountering a transaction object (e.g., goods or services), examining the transaction object, purchasing the transaction object, paying a cost to the service provider, and using the service. . If the user has an unpleasant experience during any of these processes, the user may leave the service. In this regard, the estimating device 100 enables service providers to use emotions that have a causal relationship with a series of user actions as an evaluation index for evaluating services. Thereby, the estimation device 100 enables the service provider to improve the user experience from the viewpoint of the user who uses the service. Therefore, the estimating apparatus 100 can reduce the possibility that the user will withdraw from the service in any of the processes described above. The estimation device 100 that implements such estimation processing will be described in detail below.

[3. Configuration of estimation device]
Next, a configuration example of the estimation device 100 according to the embodiment will be described with reference to FIG. FIG. 3 is a diagram illustrating a configuration example of the estimation device 100 according to the embodiment. As shown in FIG. 3 , estimation device 100 includes communication unit 110 , storage unit 120 , and control unit 130 . Note that the estimation apparatus 100 has an input unit (for example, keyboard, mouse, etc.) that receives various operations from an administrator or the like who uses the estimation apparatus 100, and a display unit (liquid crystal display, etc.) for displaying various information. may

(Communication unit 110)
The communication unit 110 is realized by, for example, a NIC (Network Interface Card) or the like. The communication unit 110 is connected to a network by wire or wirelessly, and transmits and receives information to and from the user device 10 via the network.

(storage unit 120)
The storage unit 120 is realized by, for example, a semiconductor memory device such as a RAM (Random Access Memory) or flash memory, or a storage device such as a hard disk or an optical disk. As shown in FIG. 3 , the storage section 120 has a learning data storage section 121 .

(Learning data storage unit 121)
FIG. 4 is a diagram showing an example of the learning data storage unit 121 according to the embodiment. The learning data storage unit 121 stores sets of learning data (that is, data sets).

For example, the learning data storage unit 121 stores a set of learning data for learning the relationship between a user's emotion and a series of actions of the user that have a causal relationship with this emotion. More specifically, the learning data storage unit 121 stores learning data for learning the relationship between the user's emotion and the user's series of actions that have a causal relationship with the user's emotion using the MIL technique. memorize the set.

The learning data storage unit 121 stores learning data received by the receiving unit 131, for example. In the example of FIG. 4, the learning data storage unit 121 stores "learning data" for each "learning data ID". Illustratively, the “learning data” includes the item “bag” and the item “label”.

“Learning data ID” indicates an identifier for identifying learning data. "Bag" indicates a bag included in the training data set. "Label" indicates the correct label associated with the bag.

As noted above, in MIL, one correct label is associated with one or more tuples corresponding to one or more instances. More specifically, one or more instances are included in either the positive bag or the negative bag. A positive bag contains one or more positive instances. A negative bag, on the other hand, contains only one or more negative instances.

For example, in FIG. 4, the learning data bag identified by the learning data ID "D1" includes a plurality of instances respectively corresponding to "action UA11", "action UA12", "action UA13" and "action UA14". It shows that it is a "positive bag".

Also, for example, FIG. 4 shows that the correct label associated with the learning data bag identified by the learning data ID “D1” is the emotion “happiness”.

That is, FIG. 4 shows that the user experienced the emotion "happiness" during a predetermined period of time during which the user took action UA11, action UA12, action UA13, and action UA14.

Also, for example, FIG. 4 shows that the bag of learning data identified by the learning data ID “D2” is a “negative bag” that includes a plurality of instances respectively corresponding to “action UA21” and “action UA22”. showing.

Also, for example, FIG. 4 shows that the correct label associated with the learning data bag identified by the learning data ID “D2” is the emotion “passion”.

That is, FIG. 4 shows that a certain user did not experience the emotion “passion” during a predetermined period of time during which the user took action UA21 and took action UA22.

Note that the user's emotion during the predetermined period is, for example, the user's emotion during the predetermined period, but is not limited to this. The user's emotion during the predetermined time period described above may be the user's emotion after the predetermined time period. That is, the user's emotion in the above-described predetermined period may be the emotion immediately after the series of actions. Also, the user's emotion during the predetermined period may be the user's emotion before the predetermined period. That is, the user's emotion in the above-described predetermined period may be the emotion immediately before the series of actions.

That is, the user's emotion over a predetermined period of time may be the emotion over the entire predetermined period of time. Also, the user's emotion during the predetermined period may be the emotion immediately after the predetermined period. Also, the user's emotion during the predetermined period may be the emotion immediately before the predetermined period.

Regarding the relationship between actions and emotions, certain emotions may arise while the user is performing a series of actions. On the other hand, a series of actions of the user may produce certain emotions. There are also cases in which the user takes a series of actions due to an emotion.

(control unit 130)
The control unit 130 is a controller. For example, a processor such as a CPU (Central Processing Unit) or an MPU (Micro Processing Unit) executes various programs (estimation programs) stored in a storage device inside the estimation device 100. (equivalent to one example) is implemented by executing the RAM or the like as a work area. Also, the control unit 130 is a controller, and may be realized by an integrated circuit such as an ASIC (Application Specific Integrated Circuit) or an FPGA (Field Programmable Gate Array).

As shown in FIG. 3, the control unit 130 includes a receiving unit 131, an acquiring unit 132, an estimating unit 133, a first providing unit 134, and a second providing unit 135, and performs information processing described below. realize or perform the function or action of Note that the internal configuration of the control unit 130 is not limited to the configuration shown in FIG. 3, and may be another configuration as long as it performs information processing described later.

(Receiver 131)
The receiving unit 131 receives learning data. For example, the receiving unit 131 receives learning data from a predetermined information processing device. For example, the receiving unit 131 receives learning data from a device of an entity (eg, user, company) related to the estimating device 100 . The receiving unit 131 may store the received learning data in the learning data storage unit 121 .

The receiving unit 131 may receive learning data from the user of the estimation device 100 via a user interface. The receiving unit 131 may store the received learning data in the learning data storage unit 121 .

The receiving unit 131 receives behavior information of a user (eg, user, target user). For example, the receiving unit 131 receives behavior information from a predetermined information processing device. For example, the receiving unit 131 receives behavior information from devices of entities (for example, users, companies) related to the estimation device 100 . The receiving unit 131 may store the received behavior information in a behavior information storage unit (not shown), which is a predetermined storage area within the storage unit 120 .

The receiving unit 131 may receive behavior information from the user of the estimation device 100 via the user interface. The receiving unit 131 may store the received behavior information in the behavior information storage unit described above.

The receiving unit 131 may receive the behavior information of the user (for example, the user, the target user) from the user's user device 10 via the service provided by the third providing unit 136 . The services provided by the third providing unit 136 may be, for example, network services such as search, portal, SNS, auction service, and map information service.

The receiving unit 131 may receive behavior information of a user (eg, user, target user) via content provided by the third providing unit 136 . Contents provided by the third providing unit 136 include search, portal, shopping, auction, news, finance, travel, restaurant introduction, sports, television, video, game, map, route information, cuisine, real estate, automobile, blog. , marriage, comics, fortune-telling, local information, and the like.

(Acquisition unit 132)
The acquisition unit 132 acquires behavior information indicating behavior of a predetermined user (for example, target user).

The acquisition unit 132 acquires the behavior information received by the reception unit 131, for example. The acquisition unit 132 may acquire behavior information from a predetermined storage device. For example, the acquisition unit 132 may acquire behavior information from the behavior information storage unit described above. The acquisition unit 132 may store the acquired behavior information in the behavior information storage unit described above.

In one example, the acquisition unit 132 acquires user behavior information.

The user information acquired by the acquisition unit 132 indicates user behavior on the network. For example, the acquired behavior information includes the click history, the history of browsed content, the selection history of advertisements, the posting history of SNS, the exhibition history of auctions, the successful bid history of auctions, the purchase history, the location history (for example, the history of GPS , beacon detection history), etc., on the network.

In one example, the acquisition unit 132 acquires behavior information of the target user.

(Estimation unit 133)
Based on the action information acquired by the acquisition unit 132, the estimation unit 133 estimates an emotion having a causal relationship with a series of actions by a predetermined user (for example, a target user).

For example, the estimating unit 133 uses a model that has learned the relationship between a user's (for example, user's) emotion and a series of behaviors of this user that have a causal relationship with this emotion. Based on the behavioral information of (for example, the target user), an emotion having a causal relationship with the series of behaviors of this predetermined user is estimated.

Further, for example, when behavior information indicating a plurality of behaviors of a user is input, the estimation unit 133 determines a series of behaviors included in the plurality of behaviors, Predict the emotion of a given user using a model that has been trained to output the emotion of the user.

Also, for example, the estimating unit 133 uses a model that has learned the relationship between the user's emotion and a series of actions of the user that have a causal relationship with the user's emotion, using MIL technology, to obtain a predetermined Estimate user sentiment.

Also, for example, the estimating unit 133 estimates, as an emotion having a causal relationship with a series of actions, an emotion when a predetermined user is performing this series of actions.

Also, for example, the estimating unit 133 estimates, as an emotion having a causal relationship with a series of actions, an emotion after a predetermined user performs this series of actions.

Also, for example, the estimating unit 133 estimates, as an emotion having a causal relationship with a series of actions, an emotion before a predetermined user performs this series of actions.

Also, for example, the estimating unit 133 estimates an emotion that is a series of behaviors of a predetermined user and has a causal relationship with a series of behaviors related to a predetermined transaction target.

Also, for example, the estimating unit 133 estimates an emotion having a causal relationship with a series of actions included in actions from discovering a predetermined transaction object to purchasing the predetermined transaction object.

In one example, first, the estimating unit 133 identifies the timing at which the user purchased a predetermined transaction object (for example, a product or service) based on the behavior information acquired by the acquiring unit 132, and determines the timing acquired by the acquiring unit 132. Action information indicating the action of the user in a predetermined period including the specified timing is determined from the action information obtained.

Next, the estimation unit 133 estimates the user's emotion during the above-described predetermined period, and generates learning data (that is, training data) including the estimated user's emotion and the determined behavioral information.

For purposes of illustration, assume a user is utilizing a sensing device such as a wearable device. In this example, the estimation unit 133 uses the sensing device to identify the user's physiological response (for example, changes in heart rate or body temperature). The sensing device may be the user equipment 10 . Then, the estimating unit 133 can estimate the user's emotion during the above-described predetermined period based on the specified physiological reaction. For example, the estimation unit 133 estimates emotions such as “happiness”, “surprise”, and “fear” (for example, six basic emotions) as the emotions of the user during a predetermined period based on the specified physiological reaction. do. Such emotions may be emotions associated with the purchase of a transactional object. For example, the estimation unit 133 may estimate positive emotions such as happiness, joy, passion, and enjoyment as the user's emotions during a predetermined period of time.

In this way, the estimation unit 133 can associate the estimated user's emotion with the determined behavioral information. As a result, the estimation unit 133 can generate learning data including the estimated user's emotion and the determined behavioral information.

Next, the estimation unit 133 performs MIL based on the generated learning data to generate a prediction model that predicts the user's emotion.

For illustration purposes, it is assumed that the estimator 133 performs MIL based on the generated learning data and generates an SVM that classifies the user's emotion as a prediction model that predicts the user's emotion. In this example, when behavior information indicative of one or more user behaviors is input into the generated SVM, the generated SVM may be Classify user sentiment.

After that, the estimating unit 133 inputs the target user's behavior information acquired by the acquiring unit 132 into the generated prediction model, thereby obtaining a series of emotions of the target user and a target user having a causal relationship with this emotion. Estimate behavior and.

(First providing unit 134)
The first providing unit 134 provides a predetermined user with information of content corresponding to the emotion estimated by the estimating unit 133 .

For example, the first provision unit 134 inputs a plurality of instances corresponding to a series of behaviors of the target user to the prediction model generated by the estimation unit 133, thereby converting the series of behaviors of the target user into a predetermined emotion. Categorize into corresponding actions. Then, the first providing unit 134 provides the target user with information corresponding to the predetermined emotion.

For example, a series of actions of the target user corresponds to a part of a series of actions of encountering the transaction object, researching the transaction object, purchasing the transaction object, paying the cost to the service provider, and using the service. In this case, the first providing unit 134 may provide the target user with information on transaction objects and services as information corresponding to predetermined emotions.

As an example, the sequence of actions of encountering a transaction object and examining the transaction object may correspond to the predetermined emotion "happiness". In this example, the first providing unit 134 may provide the target user with an advertisement regarding this trading object.

As another example, the course of action of encountering a transaction object and examining the transaction object may correspond to the predetermined emotion "lethargy." In this example, the first providing unit 134 may not provide the target user with the advertisement related to this transaction object. In this case, the first providing unit 134 may provide the target user with detailed information on the transaction object so that the target user can easily search for information on the transaction object.

As yet another example, the sequence of actions of encountering a transaction object, researching the transaction object, and purchasing the transaction object may correspond to a predetermined emotion "upset." In this example, the first providing unit 134 may provide the target user with a coupon that can be used for the service.

(Second providing unit 135)
The second providing unit 135 may provide information of predetermined content to a predetermined user at a timing corresponding to the emotion estimated by the estimating unit 133 .

In one example, the second providing unit 135 determines whether the combination of the "target user's emotion" estimated by the estimation unit 133 and the "target user's series of actions" estimated by the estimation unit 133 satisfies a predetermined condition. If it is determined that the combination satisfies a predetermined condition, the advertisement is provided to the target user.

[4. Estimation process flow]
Next, a procedure of estimation processing by the estimation device 100 according to the embodiment will be described. FIG. 5 is a flowchart showing an estimation processing procedure by the estimation device 100 according to the embodiment.

As shown in FIG. 5, first, the estimating device 100 acquires behavior information indicating behavior of a predetermined user (step S101).

Next, the estimation device 100 estimates an emotion having a causal relationship with a series of behaviors of a given user based on the acquired behavior information (step S102).

For example, the estimating device 100 uses a model that has learned the relationship between the user's emotion and a series of actions of the user that have a causal relationship with this emotion, and based on predetermined user behavior information , to estimate an emotion that has a causal relationship with this series of actions of the given user.

Further, for example, when behavior information indicating a plurality of behaviors of a user is input, the estimating apparatus 100 can determine a series of behaviors included in the plurality of behaviors and a series of behaviors of the user having a causal relationship with the series of behaviors. Predict the emotion of a given user using a model that has been trained to output the emotion of the user.

Also, for example, the estimation apparatus 100 creates a model that has learned the relationship between the user's emotion and the user's series of behaviors that have a causal relationship with the user's emotion, using MIL (Multiple Instance Learning) technology. is used to estimate the emotion of a given user.

Also, for example, the estimating device 100 estimates, as an emotion having a causal relationship with a series of actions, an emotion when a predetermined user is performing this series of actions.

Also, for example, the estimating device 100 estimates an emotion after a predetermined user performs this series of actions as an emotion having a causal relationship with the series of actions.

Also, for example, the estimating device 100 estimates, as an emotion having a causal relationship with a series of actions, the emotion of a predetermined user before performing this series of actions.

Also, for example, the estimation device 100 estimates an emotion that is a series of actions by a predetermined user and that has a causal relationship with a series of actions regarding a predetermined transaction target.

Also, for example, the estimation device 100 estimates an emotion having a causal relationship with a series of actions included in actions from discovering a predetermined transaction object to purchasing the predetermined transaction object.

Next, the estimating device 100 provides a predetermined user with information of content corresponding to the estimated emotion (step S103). The estimating device 100 may provide information of predetermined content to a predetermined user at a timing according to the estimated emotion.

[5. Modification]
The estimation apparatus 100 according to the above embodiments may be implemented in various different forms other than the above embodiments. Therefore, other embodiments of the estimation device 100 will be described below.

[5-1. Estimation of changes in emotions]
The estimating unit 133 may estimate changes in emotions having a causal relationship with a series of actions.

For example, the estimating unit 133 inputs a plurality of instances corresponding to a series of actions of the target user during the first predetermined period into the prediction model generated by the estimating unit 133, thereby obtaining A set of actions of the target user may be grouped into actions corresponding to a first predetermined emotion. Similarly, the estimating unit 133 inputs a plurality of instances corresponding to a series of behaviors of the target user during the second predetermined period to the prediction model generated by the estimating unit 133, so that the second predetermined period may be classified into behaviors corresponding to a second predetermined emotion. In this case, the estimation unit 133 may estimate the transition of emotion from the first predetermined emotion to the second predetermined emotion as the transition of emotion having a causal relationship with a series of actions.

Note that the first predetermined period described above may be a period after the second predetermined period described above. Also, the first predetermined period described above may be a period before the second predetermined period described above. Also, the first predetermined period described above and the second predetermined period described above may overlap.

As for changes in emotions, the target user's emotions are considered to change from moment to moment. In this regard, for example, when the target user's emotion changes from negative emotion to positive emotion, the second providing unit 135 may provide the target user with information of predetermined content. For example, the second providing unit 135 may provide the advertisement to the target user when the target user's emotion changes from the emotion "nervous" to the emotion "enjoyment". Further, for example, when the target user's emotion changes from negative emotion to positive emotion, the second providing unit 135 may provide information of predetermined content to the target user. For example, when the target user's emotion changes from emotion "happiness" to emotion "anger," the second providing unit 135 may provide the target user with a coupon that can be used for the service.

As an example, a series of actions of encountering a transaction target and investigating the transaction target may correspond to an emotion that changes from the emotion "nervous" to the emotion "enjoyment". In this example, the second providing unit 135 may provide the target user with an advertisement regarding this transaction object.

As another example, a series of actions of examining a transaction object and purchasing a transaction object may correspond to an emotion that transitions from the emotion "happiness" to the emotion "anger." In this example, the second providing unit 135 may provide the target user with a coupon that can be used for the service.

[5-2. others〕
Also, among the processes described in the above embodiments, some of the processes described as being automatically performed can also be performed manually. Alternatively, all or part of the processes described as being performed manually can be performed automatically by known methods. In addition, information including processing procedures, specific names, various data and parameters shown in the above documents and drawings can be arbitrarily changed unless otherwise specified. For example, the various information shown in each drawing is not limited to the illustrated information.

Also, each component of each device illustrated is functionally conceptual, and does not necessarily need to be physically configured as illustrated. In other words, the specific form of distribution and integration of each device is not limited to the one shown in the figure, and all or part of them can be functionally or physically distributed and integrated in arbitrary units according to various loads and usage conditions. Can be integrated and configured.

For example, part or all of the storage unit 120 shown in FIG. 3 may be held in a storage server or the like instead of being held by the estimation device 100 . In this case, the estimation device 100 acquires various information such as learning data by accessing the storage server.

[5-3. Hardware configuration]
Also, the estimation device 100 according to the above-described embodiments is implemented by a computer 1000 configured as shown in FIG. 6, for example. FIG. 6 is a diagram illustrating an example of a hardware configuration; A computer 1000 is connected to an output device 1010 and an input device 1020, and an arithmetic device 1030, a primary storage device 1040, a secondary storage device 1050, an output IF (Interface) 1060, an input IF 1070, and a network IF 1080 are connected via a bus 1090. have

The arithmetic device 1030 operates based on programs stored in the primary storage device 1040 and the secondary storage device 1050, programs read from the input device 1020, and the like, and executes various processes. The primary storage device 1040 is a memory device such as a RAM that temporarily stores data used by the arithmetic device 1030 for various calculations. The secondary storage device 1050 is a storage device in which data used for various calculations by the arithmetic device 1030 and various databases are registered, and is implemented by a ROM (Read Only Memory), HDD, flash memory, or the like.

The output IF 1060 is an interface for transmitting information to be output to the output device 1010 that outputs various types of information such as a monitor and a printer. It is realized by a connector conforming to a standard such as HDMI (registered trademark) (High Definition Multimedia Interface). Also, the input IF 1070 is an interface for receiving information from various input devices 1020 such as a mouse, keyboard, scanner, etc., and is realized by, for example, USB.

Note that the input device 1020 includes, for example, optical recording media such as CDs (Compact Discs), DVDs (Digital Versatile Discs), PDs (Phase change rewritable discs), magneto-optical recording media such as MOs (Magneto-Optical discs), and tapes. It may be a device that reads information from a medium, a magnetic recording medium, a semiconductor memory, or the like. Also, the input device 1020 may be an external storage medium such as a USB memory.

Network IF 1080 receives data from other devices via network N and sends the data to arithmetic device 1030, and also transmits data generated by arithmetic device 1030 via network N to other devices.

The arithmetic device 1030 controls the output device 1010 and the input device 1020 via the output IF 1060 and the input IF 1070 . For example, arithmetic device 1030 loads a program from input device 1020 or secondary storage device 1050 onto primary storage device 1040 and executes the loaded program.

For example, when the computer 1000 functions as the estimation device 100 , the arithmetic device 1030 of the computer 1000 implements the functions of the control unit 130 by executing a program loaded on the primary storage device 1040 .

[6. effect〕
As described above, the estimation device 100 according to the embodiment has the acquisition unit 132 and the estimation unit 133 . The acquisition unit 132 acquires behavior information indicating behavior of a predetermined user. Based on the action information acquired by the acquisition unit 132, the estimation unit 133 estimates an emotion having a causal relationship with a series of actions of a predetermined user.

In addition, in the estimation device 100 according to the embodiment, the estimation unit 133 uses a model that has learned the relationship between the user's emotion and the user's series of actions that have a causal relationship with this emotion, Based on the action information of a given user, an emotion having a causal relationship with a series of actions of this given user is estimated.

Further, in the estimating apparatus 100 according to the embodiment, when behavior information indicating a plurality of behaviors of the user is input, the estimating unit 133 generates a series of behaviors included in the plurality of behaviors, and a series of behaviors. A model trained to output this user's emotion with a causal relationship is used to estimate a given user's emotion.

In addition, in the estimation device 100 according to the embodiment, the estimation unit 133 uses MIL (Multiple Instance Learning) technology to determine the relationship between the user's emotion and the user's series of actions that have a causal relationship with the user's emotion. A model that has learned relationships is used to estimate the emotion of a given user.

In addition, in the estimation device 100 according to the embodiment, the estimation unit 133 estimates the emotion when the predetermined user is performing the series of actions as the emotion having a causal relationship with the series of actions.

Also, in the estimation device 100 according to the embodiment, the estimation unit 133 estimates the emotion after the predetermined user performs the series of actions as the emotion having a causal relationship with the series of actions.

Further, in the estimation device 100 according to the embodiment, the estimation unit 133 estimates the emotion before the predetermined user performs the series of actions as the emotion having a causal relationship with the series of actions.

In addition, in the estimation device 100 according to the embodiment, the estimation unit 133 estimates changes in emotions having a causal relationship with a series of actions.

In addition, in the estimation device 100 according to the embodiment, the estimation unit 133 estimates an emotion having a causal relationship with a series of behaviors of a predetermined user regarding a predetermined transaction target.

In addition, in the estimation device 100 according to the embodiment, the estimation unit 133 estimates an emotion having a causal relationship with a series of actions included in actions from discovering a predetermined transaction object to purchasing the predetermined transaction object. .

Further, the estimation apparatus 100 according to the embodiment has a first providing unit 134 that provides a predetermined user with information of content corresponding to the emotion estimated by the estimating unit 133 .

The estimation apparatus 100 according to the embodiment also has a second providing unit 135 that provides information of predetermined content to a predetermined user at a timing according to the emotion estimated by the estimating unit 133 .

Through the processes described above, estimation device 100 can appropriately estimate the user's emotion.

As described above, some of the embodiments of the present application have been described in detail based on the drawings. It is possible to carry out the invention in other forms with modifications.

In addition, the above-described estimating apparatus 100 may be implemented by a plurality of server computers, and depending on the function, may be implemented by calling an external platform or the like using an API (Application Programming Interface), network computing, or the like. Flexible to change.

Also, the above-mentioned "section, module, unit" can be read as "means" or "circuit". For example, the receiving section can be read as receiving means or a receiving circuit.

1 network system 10 user device 100 estimation device 110 communication unit 120 storage unit 121 learning data storage unit 130 control unit 131 reception unit 132 acquisition unit 133 estimation unit 134 first provision unit 135 second provision unit

Claims (13)

an acquisition unit that acquires behavior information indicating behavior of a predetermined user;
Between a user's single emotion associated with a predetermined period and a series of actions by the user during the predetermined period that have a causal relationship with the emotion to the action information acquired by the acquisition unit, the emotion of the predetermined user during the period in which the predetermined user took a series of actions, an estimation device for estimating an emotion having a causal relationship with a series of actions by the predetermined user during a period of time. When behavior information indicating a plurality of behaviors of a user is input, the estimating unit determines a series of behaviors included in the plurality of behaviors and emotions of the user having a causal relationship with the series of behaviors. The estimating device according to claim 1, wherein the emotion of the predetermined user is estimated using a model that has been trained to output. The estimating unit uses a model that has learned the relationship between the user's emotion and a series of actions of the user that have a causal relationship with the user's emotion by MIL (Multiple Instance Learning) technology, 3. The estimating device according to claim 1, wherein the emotion of the predetermined user is estimated. 4. The estimating unit estimates, as the emotion having a causal relationship with the series of actions, the emotion of the predetermined user performing the series of actions. or the estimating device according to one. 4. The estimating unit estimates, as an emotion having a causal relationship with the series of actions, an emotion after the predetermined user performs the series of actions. 1. The estimating device according to 1. 4. Any one of claims 1 to 3, wherein the estimating unit estimates, as an emotion having a causal relationship with the series of actions, an emotion before the predetermined user performs the series of actions. The estimating device according to 1. The estimating device according to any one of claims 1 to 6, wherein the estimating unit estimates changes in emotion having a causal relationship with the series of actions. The estimating unit estimates an emotion having a causal relationship with a series of actions by the predetermined user, which are related to a predetermined transaction object. The estimating device according to 1. 9. The method according to claim 8, wherein the estimating unit estimates an emotion having a causal relationship with a series of actions included in actions from discovering the predetermined transaction object to purchasing the predetermined transaction object. Estimation device as described. The estimating device according to any one of claims 1 to 9, further comprising: a first providing unit that provides the predetermined user with information corresponding to the emotion estimated by the estimating unit. . 11. Any one of claims 1 to 10, further comprising a second providing unit that provides information of predetermined content to the predetermined user at a timing corresponding to the emotion estimated by the estimating unit. The estimating device described in . An estimation method executed by an estimation device,
an acquisition step of acquiring behavior information indicating behavior of a predetermined user;
Between a user's single emotion associated with a predetermined period and a series of actions by the user during the predetermined period that have a causal relationship with the emotion By applying a model that has learned the relationship between the an estimating step of estimating an emotion having a causal relationship with a series of actions by the given user over a period of time. an acquisition procedure for acquiring behavior information indicating behavior of a predetermined user;
Between a user's single emotion associated with a predetermined period and a series of actions by the user during the predetermined period that have a causal relationship with the emotion to the action information acquired by the acquisition procedure, the emotion of the predetermined user during the period in which the predetermined user took a series of actions, an estimation program for causing a computer to execute: an estimation procedure for estimating a series of behaviors and emotions having a causal relationship by the predetermined user during a period;

Images