WO2018207619A1

WO2018207619A1 - Data collection apparatus and learning apparatus

Info

Publication number: WO2018207619A1
Application number: PCT/JP2018/016704
Authority: WO
Inventors: Tanichi Ando
Original assignee: Omron Corporation
Priority date: 2017-05-11
Filing date: 2018-04-25
Publication date: 2018-11-15
Also published as: JP2018190318A; JP6911505B2

Abstract

A technique by which factors eliciting a target person's emotions can be estimated is provided. A data collection apparatus according to an aspect of the invention includes: a first obtainment unit configured to obtain first data expressing an emotion of a person; a second obtainment unit configured to, in parallel with the obtainment of the first data, obtain second data pertaining to a factor that can influence the emotion of the person; an emotion specifying unit configured to specify a type of the emotion of the person at a given time on the basis of the obtained first data; and a classification processing unit configured to classify the second data for each type of the emotion of the person by saving the second data, obtained in an elicitation time related to the eliciting of the emotion of the person at the given time, in association with the type of the emotion of the person at the specified given time.

Description

DATA COLLECTION APPARATUS AND LEARNING APPARATUS

The present invention relates to a data collection apparatus and a learning apparatus.

JP 2008-146318A proposes an emotion estimation apparatus that estimates an unspecified person's emotion by recognizing that person's facial expression. Specifically, JP 2008-146318A proposes creating a facial expression map using images of facial expressions of specified persons associated with predetermined emotions, and then estimating the unspecified person's emotion on the basis of the created facial expression map and an image of the unspecified person's facial expression.

JP 2008-146318A

Techniques for estimating a target person's emotion from an image of the target person’s facial expressions have existed for some time, as exemplified by JP 2008-146318A. However, the inventor of the present invention found a problem with the past technique in that even if a target person’s emotion can be estimated, the factors that elicited that emotion cannot be estimated.

Having been achieved in light of such circumstances, an object of an aspect of the present invention is to provide a technique that makes it possible to estimate factors that have elicited a target person’s emotion.

The present invention employs the following configuration to solve the above-described problem.

A data collection apparatus according to an aspect of the present invention includes: a first obtainment unit configured to obtain first data expressing an emotion of a person; a second obtainment unit configured to , in parallel with the obtainment of the first data, obtain second data pertaining to a factor that can influence the emotion of the person; an emotion specifying unit configured to specify a type of the emotion of the person at a given time on the basis of the obtained first data; and a classification processing unit configured to classify the second data for each type of the emotion of the person by saving the second data, obtained in an elicitation time related to the eliciting of the emotion of the person at the given time, in association with the type of the emotion of the person at the specified given time.

The data collection apparatus according to the configuration described above obtains the first data expressing the person’s emotion and the second data pertaining to a factor that can influence the person’s emotion, in parallel and associated with the same time series. The first data may be any data that can express the person’s emotion, such as image data showing the person’s face. The second data may be any data pertaining to factors that can influence the person’s emotion, such as the person’s biological data or environment data indicating the person’s surrounding environment.

The data collection apparatus according to the configuration described above specifies the type of a target person's emotion on the basis of the first data, and then saves the second data obtained within an elicitation time related to the eliciting of that emotion in association with the specified type of emotion. Thus with the data collection apparatus according to the configuration described above, the second data pertaining to factors than can influence the target person’s emotion can be classified and collected for each type of the target person's emotion. In other words, a database in which data of influencing factors that can act as factors in eliciting the target person’s emotion (the second data) is classified for each type of emotion can be created. Factors that elicit the target person’s emotions can be analyzed by using the data of influencing factors classified for each type of emotion. Thus according to the configuration described above, a technique that makes it possible to estimate factors eliciting a target person's emotions can be provided.

In the data collection apparatus according to the above-described aspect, the emotion specifying unit may be configured to specify the emotion of the person at the given time using a trained learning device trained so that when first data is inputted, the learning device outputs a value indicating the type of the emotion of the person expressed by the first data. The learning device may be configured as a neural network or the like, for example. According to this configuration, the type of the target person’s emotion expressed by the first data can be easily and appropriately specified.

In the data collection apparatus according to the above-described aspect, the elicitation time may be set for each of types of the second data. According to this configuration, an appropriate elicitation time can be set in accordance with the type of the second data. Through this, the amount of the second data classified for each type of emotion can be set to an appropriate data amount in accordance with the type of influencing factor.

In the data collection apparatus according to the above-described aspect, the classification processing unit may further associate attributes of the person with the second data classified for each type of the emotion of the person. According to this configuration, the second data pertaining to factors than can influence the target person’s emotion can be classified and collected for each type of emotion and each attribute of the person. The emotion elicited in a person may differ greatly depending on that person’s attributes. In response to this, according to this configuration, the searchability of the data used to estimate the factors that elicit emotions can be improved, and the accuracy of estimating the eliciting factors can be improved, by associating the second data with the target person's attributes.

In the data collection apparatus according to the above-described aspect, the attributes of the person may be specified by at least one of sex, height, weight, blood type, age, date of birth, place of birth, nationality, and profession. According to this configuration, the person’s attributes can be defined appropriately.

The data collection apparatus according to the above-described aspect may further include a factor estimating unit configured to estimate a factor eliciting the emotion by analyzing the second data classified for each type of the emotion. According to this configuration, factors that elicit the target person’s emotion can be estimated by using the second data classified and collected for each type of emotion.

In the data collection apparatus according to the above-described aspect, the classification processing unit may be configured to delete the second data not classified for any type of emotion. According to this configuration, unnecessary data is deleted, which makes it possible to efficiently use a storage unit that saves the data.

In the data collection apparatus according to the above-described aspect, the first data may include at least one of image data showing the face of the person, data obtained by polygraph, audio data containing a recording of the voice of the person, and response data from the person. According to this configuration, first data that can easily be used to specify the person’s emotion can be obtained.

In the data collection apparatus according to the above-described aspect, the second data may include at least one of biological data obtained from the person, environment data indicating an environment in the surroundings of the person, and event data indicating an event that has occurred for the person. The biological data may indicate at least one of heart rate, pulse rate, breathing rate, body temperature, blood pressure, brain waves, posture, and myoelectricity. The environment data may indicate at least one of temperature, humidity, brightness, weather, atmospheric pressure, noise, vibrations, and odors. According to this configuration, data pertaining to influencing factors that can act as factors in eliciting a target emotion in the target person can be collected appropriately.

In the data collection apparatus according to the above-described aspect, the second data may include at least two of the biological data, the environment data, and the event data; and the elicitation time may be different for the biological data, the environment data, and the event data. According to this configuration, an appropriate elicitation time can be set in accordance with the type of the second data. Through this, the amount of the second data classified for each type of emotion can be set to an appropriate data amount in accordance with the type of influencing factor.

A learning apparatus according to an aspect of the present invention includes: a data obtainment unit configured to obtain influencing factor data, the influencing factor data pertaining to a factor that can influence an emotion of a person and being classified for each of types of the emotion of the person; and a learning processing unit configured to train a learning device so that when the influencing factor data is inputted, the learning device outputs a value indicating the type of the emotion associated with the inputted influencing factor data. According to this configuration, a trained learning device that can estimate factors eliciting a target person's emotions can be constructed. Note that the data obtainment unit of the learning apparatus according to this configuration may obtain influencing factor data classified for each type of emotion from the data collection apparatus according to the above-described aspects.

In the learning apparatus according to the above-described aspect, the data obtainment unit may further obtain attribute data indicating attributes of the person; and the learning processing unit may train the learning device so that when the influencing factor data and the attribute data are inputted, the learning device outputs a value indicating the type of the emotion associated with the inputted influencing factor data. According to this configuration, a trained learning device that can estimate factors eliciting a target person's emotions in accordance with the target person's attributes can be constructed.

A learning apparatus according to an aspect of the present invention includes: a data obtainment unit configured to obtain emotion expression data expressing an emotion of a person; and a learning processing unit configured to train a learning device so that when the emotion expression data is inputted, the learning device outputs a value indicating a type of the emotion of the person expressed by the inputted emotion expression data. According to this configuration, a trained learning device that can estimate the type of a target person's emotion can be constructed.

Note that the data collection apparatus and learning apparatuses according to the aspects described above may also be realized as information processing methods that realize the above-described configurations, as programs, and as recording media (storage media) in which such programs are recorded and that can be read by a computer or other apparatus or machine. Here, a recording medium that can be read by a computer or the like is a medium that stores information of the programs or the like through electrical, magnetic, optical, mechanical, or chemical effects.

For example, a data collection method according to an aspect of the present invention is an information processing method in which a computer executes: a step of obtaining first data expressing an emotion of a person; a step of, in parallel with the obtainment of the first data, obtaining second data pertaining to a factor that can influence the emotion of the person; a step of specifying a type of the emotion of the person at a given time on the basis of the obtained first data; and a step of classifying the second data for each type of the emotion of the person by saving the second data, obtained in an elicitation time related to the eliciting of the emotion of the person at the given time, in association with the type of the emotion of the person at the specified given time.

For example, a data collection program according to an aspect of the present invention is a program that causes a computer to execute: a step of obtaining first data expressing an emotion of a person; a step of, in parallel with the obtainment of the first data, obtaining second data pertaining to a factor that can influence the emotion of the person; a step of specifying a type of the emotion of the person at a given time on the basis of the obtained first data; and a step of classifying the second data for each type of the emotion of the person by saving the second data, obtained in an elicitation time related to the eliciting of the emotion of the person at the given time, in association with the type of the emotion of the person at the specified given time.

For example, a learning method according to an aspect of the present invention is an information processing method in which a computer executes: a step of obtaining influencing factor data, the influencing factor data pertaining to a factor that can influence an emotion of a person and being classified for each of types of the emotion of the person; and a step of training a learning device so that when the influencing factor data is inputted, the learning device outputs a value indicating the type of the emotion associated with the inputted influencing factor data.

For example, a learning program according to an aspect of the present invention is a program that causes a computer to execute: a step of obtaining influencing factor data, the influencing factor data pertaining to a factor that can influence an emotion of a person and being classified for each of types of the emotion of the person; and a step of training a learning device so that when the influencing factor data is inputted, the learning device outputs a value indicating the type of the emotion associated with the inputted influencing factor data.

For example, a learning method according to an aspect of the present invention is an information processing method in which a computer executes: a step of obtaining emotion expression data expressing an emotion of a person; and a step of training a learning device so that when the emotion expression data is inputted, the learning device outputs a value indicating a type of the emotion of the person expressed by the inputted emotion expression data.

For example, a learning program according to an aspect of the present invention is a program that causes a computer to execute: a step of obtaining emotion expression data expressing an emotion of a person; and a step of training a learning device so that when the emotion expression data is inputted, the learning device outputs a value indicating a type of the emotion of the person expressed by the inputted emotion expression data.

According to the present invention, a technique that makes it possible to estimate factors eliciting a target person's emotions can be provided.

FIG. 1 schematically illustrates an example of a situation in which a data collection apparatus and each of learning apparatuses according to an embodiment are applied. FIG. 2 schematically illustrates an example of the hardware configuration of the data collection apparatus according to the embodiment. FIG. 3 schematically illustrates an example of the hardware configuration of a first learning apparatus according to the embodiment. FIG. 4 schematically illustrates an example of the hardware configuration of a second learning apparatus according to the embodiment. FIG. 5 schematically illustrates an example of the functional configuration of the data collection apparatus according to the embodiment. FIG. 6 schematically illustrates an example of a situation in which influencing factor data obtained within an elicitation time is associated with a type of identified emotion. FIG. 7 schematically illustrates an example of the functional configuration of the first learning apparatus according to the embodiment. FIG. 8 schematically illustrates an example of the functional configuration of the second learning apparatus according to the embodiment. FIG. 9 illustrates an example of a processing sequence carried out by the data collection apparatus according to the embodiment. FIG. 10 illustrates an example of a processing sequence carried out by the first learning apparatus according to the embodiment. FIG. 11 illustrates an example of a processing sequence carried out by the second learning apparatus according to the embodiment. FIG. 12 schematically illustrates an example of the functional configuration of a data collection apparatus according to a variation. FIG. 13 schematically illustrates an example of the functional configuration of a first learning apparatus according to a variation. FIG. 14 schematically illustrates an example of the functional configuration of a second learning apparatus according to a variation. FIG. 15 schematically illustrates an example of the functional configuration of a data collection apparatus according to a variation.

An embodiment according to an aspect of the present invention (also called “the present embodiment” below) will be described next with reference to the drawings. However, the present embodiment described below is in all senses merely an example of the present invention. It goes without saying that many improvements and changes can be made without departing from the scope of the present invention. In other words, specific configurations based on the embodiment can be employed as appropriate in carrying out the present invention. Note that although the data mentioned in the present embodiment is described with natural language, the data is more specifically defined by quasi-language, commands, parameters, machine language, and so on that can be recognized by computers.

§1 Application Example
First, an example of a situation in which the present invention is applied will be described using FIG. 1. FIG. 1 schematically illustrates an example of a situation in which a data collection apparatus 1, a first learning apparatus 2, and a second learning apparatus 3 according to the present embodiment are applied.

As illustrated in FIG. 1, the data collection apparatus 1 is an information processing apparatus that collects influencing factor data 123 pertaining to factors than can influence the emotions of a person, for each of types of emotions. Specifically, the data collection apparatus 1 uses a first sensor 51 to obtain emotion expression data 122 expressing a person’s emotion. The emotion expression data 122 corresponds to “first data” according to the present invention, and may be any data that can express a person’s emotion. For example, the emotion expression data 122 may be image data showing a person’s face, data obtained by polygraph, audio data containing a recording of a person’s voice, response data from a person, and so on. The first sensor 51 selects the emotion expression data 122 as appropriate in accordance with the type of the data obtained. The first sensor 51 may be, for example, a camera, a polygraph, a microphone, an input device for making responses, or the like.

Parallel with the obtainment of the emotion expression data 122, the data collection apparatus 1 obtains the influencing factor data 123 pertaining to factors that can influence the person’s emotions. That is, the data collection apparatus 1 obtains the emotion expression data 122 and the influencing factor data 123 associated with the same time series. The influencing factor data 123 corresponds to “second data” according to the present invention, and may be any data pertaining to factors that can influence a person’s emotions. For example, the influencing factor data 123 may be biological data obtained from the target person, environment data expressing the target person’s surrounding environment, event data indicating an event that has occurred with respect to the target person, and so on. A second sensor 52 selects the influencing factor data 123 as appropriate in accordance with the type of the data obtained. For example, the second sensor 52 may be a biological sensor that obtains a heart rate or the like from a person’s body, an environment sensor that obtains a temperature or the like from the person’s surroundings, or the like.

Next, the data collection apparatus 1 specifies a type of emotion of the target person at a given time on the basis of the emotion expression data 122. The data collection apparatus 1 then saves the influencing factor data 123, obtained in an elicitation time related to the eliciting of the target person’s emotion at the given time, in association with the type of the target person’s emotion at the specified given time. As a result, the data collection apparatus 1 classifies and collects the influencing factor data 123 for each type of emotion. In the example illustrated in FIG. 1, the emotion types are indicated as unique states, such as “joy”, “anger”, “sadness”, “fun”, and so on. However, the expressions of the types of emotions need not be limited to this example. For example, expressions of emotions that form pairs may be used, such as “stable” and “unstable”, “like” and “dislike”, and so on. When using expressions of emotions that form pairs, expressions indicating intermediate states of the pairs of emotions may be used as well, such as “somewhat unstable”, which indicates a state between “stable” and “unstable”. One or more expressions for multiple emotions may be combined to express the target person’s emotions as well. Note that the type of emotion may be indicated by a numerical expression using one or more parameters. Specifying the type of an emotion on the basis of the emotion expression data 122 can be carried out, for example, by using a trained learning device (a trained neural network 6) obtained through machine learning (described later).

The first learning apparatus 2 is an information processing apparatus that uses the influencing factor data classified for each type of the target person’s emotions to construct a trained learning device that estimates the type of emotion elicited in the target person on the basis of influencing factors. Specifically, the first learning apparatus 2 obtains the influencing factor data classified for each type of the target person’s emotions. The first learning apparatus 2 obtains the influencing factor data 123 classified for each type of emotion from the data collection apparatus 1 over a network or the like, for example. Then, the first learning apparatus 2 trains a learning device (a neural network 7, described later) so that when the influencing factor data 123 is inputted, the learning device outputs a value indicating the type of the emotion associated with the inputted influencing factor data 123. Through this, the first learning apparatus 2 constructs the trained learning device that estimates the type of emotion elicited in the target person on the basis of influencing factors.

Note that the first learning apparatus 2 may be configured to use the constructed learning device to estimate the type of emotion elicited in the target person on the basis of data having the same format as the influencing factor data 123. Alternatively, a first learning result usage apparatus (an elicited emotion estimation apparatus, described later) that obtains the learning device constructed by the first learning apparatus 2 or a copy thereof and estimates the type of emotion elicited in the target person on the basis of data having the same format as the influencing factor data 123 may be provided.

The second learning apparatus 3 is an information processing apparatus that uses emotion expression data expressing the emotions of the target person to construct a trained learning device that identifies the type of emotion expressed by the emotion expression data. Specifically, the second learning apparatus 3 obtains the emotion expression data expressing the target person’s emotion. Then, the second learning apparatus 3 trains a learning device (a neural network 8, described later) so that when the emotion expression data is inputted, the learning device outputs values indicating the type of the target person’s emotion expressed by the inputted emotion expression data. Through this, the second learning apparatus 3 constructs a trained learning device that specifies the type of emotion expressed by the emotion expression data.

The trained learning device created by the second learning apparatus 3 can be used by the data collection apparatus 1 in processing for specifying the type of the target person's emotion at a given time on the basis of the emotion expression data 122. Accordingly, the data collection apparatus 1 according to the present embodiment obtains the trained learning device created by the second learning apparatus 3 over a network or the like. The data collection apparatus 1 according to the present embodiment then uses the trained learning device created by the second learning apparatus 3 to specify the target person’s emotion expressed by the emotion expression data 122.

As described thus far, with the data collection apparatus 1 according to the present embodiment, the influencing factor data 123 pertaining to factors than can influence the target person’s emotions can be classified and collected for each type of emotion. Factors that elicit the target person’s emotions can be analyzed by using the influencing factor data 123 classified for each type of emotion. For example, the factors eliciting the emotions can be analyzed by subjecting the influencing factor data 123 classified for each emotion to a statistical method, a method using machine learning, or the like. The data collection apparatus 1 according to the present embodiment thus makes it possible to estimate factors eliciting the target person's emotions. Additionally, using the influencing factor data 123 classified for each type of emotion makes it possible to construct a trained learning device that estimates the type of emotion elicited in the target person on the basis of influencing factors, as with the first learning apparatus 2. Thus according to the present embodiment, emotions elicited in a target person can be estimated from data pertaining to influencing factors.

§2 Configuration Example
≪Hardware Configuration≫
<Data Collection Apparatus>
An example of the hardware configuration of the data collection apparatus 1 according to the present embodiment will be described next using FIG. 2. FIG. 2 schematically illustrates an example of the hardware configuration of the data collection apparatus 1 according to the present embodiment.

As illustrated in FIG. 2, the data collection apparatus 1 according to the present embodiment is a computer in which a control unit 11, a storage unit 12, an external interface 13, an input device 14, an output device 15, and a drive 16 are electrically connected to each other. Note that the external interface is denoted as “external I/F” in FIG. 2.

The control unit 11 includes a central processing unit (CPU), which is a hardware processor, random access memory (RAM), ROM (read-only memory), and so on, and controls the various constituent elements in accordance with information processing. The storage unit 12 is an auxiliary storage device such as a hard disk drive or a solid-state drive, and stores a collection program 121 executed by the control unit 11, the emotion expression data 122, the influencing factor data 123, expressed emotion learning result data 124, and so on.

The collection program 121 is a program for causing the data collection apparatus 1 to collect the influencing factor data 123 and execute a process for classifying the data for each type of emotion (described later; see FIG. 9). The expressed emotion learning result data 124 is data for configuring the trained learning device used in the process for specifying the type of the target person’s emotion expressed in the emotion expression data 122. Details will be given later.

The external interface 13 is an interface for connecting an external device, and is configured as appropriate in accordance with the external device to be connected. The external interface 13 may be a communication interface for connecting to another computer over a network. In the present embodiment, the data collection apparatus 1 is connected to the first sensor 51 and the second sensor 52 through the external interface 13. The first sensor 51 is, for example, a camera, a polygraph, a microphone, an input device for making responses, or the like, and is used to obtain the emotion expression data 122. On the other hand, the second sensor 52 is a biological sensor, an environment sensor, an input device, or the like, and is used to obtain the influencing factor data 123.

The types of the first sensor 51 and second sensor 52 may be selected as appropriate in accordance with the types of the obtained emotion expression data 122 and influencing factor data 123, respectively. Furthermore, a plurality of first sensors 51 and second sensors 52 may be connected to the data collection apparatus 1. In the case where image data is used as the emotion expression data 122, a night-vision camera, an infrared camera, or the like may be used as the first sensor 51. This makes it possible to obtain image data of a person’s face even in dark areas.

The input device 14 is a device for making inputs, such as a mouse or a keyboard. The output device 15 is a device for output, such as a display or speakers.

The drive 16 is a CD drive, a DVD drive, or the like, and is a drive device for loading programs stored in a storage medium 91. The type of the drive 16 may be selected as appropriate in accordance with the type of the storage medium 91. The above-described collection program 121, emotion expression data 122, influencing factor data 123, and expressed emotion learning result data 124 may be stored in the storage medium 91.

The storage medium 91 is a medium that stores information of programs or the like, recorded by the computer or other devices or machines, through electrical, magnetic, optical, mechanical, or chemical effects so that the program information can read. The data collection apparatus 1 may obtain at least one of the above-described collection program 121, emotion expression data 122, influencing factor data 123, and expressed emotion learning result data 124 from the storage medium 91.

FIG. 2 illustrates an example in which the storage medium 91 is a disk-type storage medium such as a CD or a DVD. However, the type of the storage medium 91 is not limited to a disk, and a type aside from a disk may be used instead. Semiconductor memory such as flash memory can be given as an example of a non-disk type storage medium.

With respect to the specific hardware configuration of the data collection apparatus 1, constituent elements can be omitted, replaced, or added as appropriate in accordance with the embodiment. For example, the control unit 11 may include a plurality of hardware processors. The data collection apparatus 1 may be constituted by a plurality of computers. Furthermore, rather than an information processing apparatus designed specifically for a service to be provided, the data collection apparatus 1 may use a generic server device, a personal computer (PC), or the like.

<First Learning Apparatus>
An example of the hardware configuration of the first learning apparatus 2 according to the present embodiment will be described next using FIG. 3. FIG. 3 schematically illustrates an example of the hardware configuration of the first learning apparatus 2 according to the present embodiment.

As illustrated in FIG. 3, the first learning apparatus 2 according to the present embodiment is a computer in which a control unit 21, a storage unit 22, an external interface 23, an input device 24, an output device 25, and a drive 26 are electrically connected to each other. Note that like in FIG. 2, the external interface is denoted as “external I/F” in FIG. 3.

The control unit 21 to the drive 26, and a storage medium 92, are the same as the control unit 11 to the drive 16, and the storage medium 91, of the above-described data collection apparatus 1. However, the storage unit 22 of the first learning apparatus 2 stores the influencing factor data 123, used to train the learning device, that has been classified for each type of emotion; a first learning program 221 (this may also be called an “elicited emotion learning program”), which is machine learning executed by the control unit 21 so that the learning device carries out learning for estimating an emotion elicited in a target person on the basis of the influencing factor data 123; elicited emotion learning result data 223 created by executing the first learning program 221; and so on. The first learning program 221 is a program for causing the first learning apparatus 2 to execute a learning process (FIG. 10) for the learning device, described later. Details will be given later.

Like the above-described data collection apparatus 1, the first learning program 221 and the influencing factor data 123 may be stored in the storage medium 92. As such, the first learning apparatus 2 may obtain at least one of the first learning program 221 and the influencing factor data 123 from the storage medium 92.

Also like the data collection apparatus 1, with respect to the specific hardware configuration of the first learning apparatus 2, constituent elements can be omitted, replaced, or added as appropriate in accordance with the embodiment. Rather than an information processing apparatus designed specifically for a service to be provided, the first learning apparatus 2 may use a generic server device, a PC, or the like.

<Second Learning Apparatus>
An example of the hardware configuration of the second learning apparatus 3 according to the present embodiment will be described next using FIG. 4. FIG. 4 schematically illustrates an example of the hardware configuration of the second learning apparatus 3 according to the present embodiment.

As illustrated in FIG. 4, the second learning apparatus 3 according to the present embodiment is a computer in which a control unit 31, a storage unit 32, an external interface 33, an input device 34, an output device 35, and a drive 36 are electrically connected to each other. Note that like in FIGS. 2 and 3, the external interface is denoted as “external I/F” in FIG. 4.

The control unit 31 to the drive 36, and a storage medium 93, are the same as the control unit 11 to the drive 16, and the storage medium 91, of the above-described data collection apparatus 1. However, the storage unit 32 of the second learning apparatus 3 stores learning data 322 used to train the learning device; a second learning program 321 (this may also be called an “expressed emotion learning program”), which is machine learning executed by the control unit 31 so that the learning device carries out learning for estimating an emotion of a target person on the basis of the learning data 322; the expressed emotion learning result data 124 created by executing the second learning program 321; and so on. The second learning program 321 is a program for causing the second learning apparatus 3 to execute a learning process (FIG. 11) for the learning device, described later. Details will be given later.

Like the above-described data collection apparatus 1, the second learning program 321 and the learning data 322 may be stored in the storage medium 93. As such, the second learning apparatus 3 may obtain at least one of the second learning program 321 and the learning data 322 from the storage medium 93.

Also like the data collection apparatus 1, with respect to the specific hardware configuration of the second learning apparatus 3, constituent elements can be omitted, replaced, or added as appropriate in accordance with the embodiment. Rather than an information processing apparatus designed specifically for a service to be provided, the second learning apparatus 3 may use a generic server device, a PC, or the like.

<Other>
As described above, in the present embodiment, a first learning result usage apparatus (an elicited emotion estimation apparatus, described later) that obtains the learning device constructed by the first learning apparatus 2 or a copy thereof and estimates the type of emotion elicited in the target person on the basis of data having the same format as the influencing factor data 123 may be provided. This first learning result usage apparatus (not illustrated) can be configured as a computer in which a control unit, a storage unit, an external interface, an input device, an output device, and a drive are electrically connected to each other, in the same manner as the above-described apparatuses 1 to 3.

The elicited emotion learning result data 223 obtained from the first learning apparatus 2 is saved in the storage unit. The first learning result usage apparatus is connected to a third sensor, of the same type as the second sensor 52, through the external interface. Through this, the first learning result usage apparatus can configure a trained learning device in which learning for estimating an elicited emotion has been carried out using the elicited emotion learning result data 223 saved in the storage unit. By obtaining data in the same format as the influencing factor data 123 from the third sensor and inputting the data obtained from the third sensor into the trained learning device that has been configured, the first learning result usage apparatus can obtain information indicating the type of emotion that can be elicited in a target person from the trained learning device.

≪Functional Configuration≫
<Data Collection Apparatus>
An example of the functional configuration of the data collection apparatus 1 according to the present embodiment will be described next using FIG. 5. FIG. 5 schematically illustrates an example of the functional configuration of the data collection apparatus 1 according to the present embodiment.

The control unit 11 of the data collection apparatus 1 loads the collection program 121 stored in the storage unit 12 into the RAM. The control unit 11 then controls the various constituent elements by using the CPU to interpret and execute the collection program 121 loaded into the RAM. As a result, as illustrated in FIG. 5, the data collection apparatus 1 according to the present embodiment functions as a computer including a first obtainment unit 111, a second obtainment unit 112, an emotion specifying unit 113, and a classification processing unit 114.

The first obtainment unit 111 uses the first sensor 51 to obtain the emotion expression data 122 expressing a person’s emotion. Parallel with the obtainment of the emotion expression data 122, the second obtainment unit 112 uses the second sensor 52 to obtain the influencing factor data 123 pertaining to factors that can influence the person’s emotions. Through this, the data collection apparatus 1 obtains the emotion expression data 122 and the influencing factor data 123 associated with the same time series.

Note that the emotion expression data 122 may include at least one of image data showing the target person’s face, data obtained by polygraph, audio data containing a recording of the target person’s voice, and response data from the target person. If image data is to be used as the emotion expression data 122, the faces of a plurality of target people may appear in the image data. In this case, the emotions of the plurality of people can be specified at once on the basis of the emotion expression data 122. The response data may be obtained, for example, by the target person entering his/her own emotion through the input device. However, the obtainment of the response data need not be limited to this example, and the target person’s emotion may be entered by a third party rather than by the target person him/herself. The influencing factor data 123 will be described later.

The emotion expression data 122 and the influencing factor data 123 being obtained in parallel does not mean that the periods in which the emotion expression data 122 and the influencing factor data 123 are obtained need to match. That is, it is sufficient for the periods in which the emotion expression data 122 and the influencing factor data 123 are obtained to at least partially overlap. In other words, it is sufficient for the influencing factor data 123 to be obtainable in a time period where an influencing factor eliciting the target person’s emotion, expressed by the emotion expression data 122, can be present. Accordingly, the starting times and ending times of obtaining the emotion expression data 122 and the influencing factor data 123 may differ. The timings at which the emotion expression data 122 and the influencing factor data 123 are obtained may differ as well.

The emotion specifying unit 113 specifies a type of emotion of the target person at a given time on the basis of the emotion expression data 122. As illustrated in FIG. 5, in the present embodiment, the emotion specifying unit 113 specifies the type of the target person’s emotion at a given time using a trained learning device (the neural network 6) trained so that when the emotion expression data 122 is inputted, values indicating the type of the target person’s emotion expressed by the emotion expression data 122 are outputted. The classification processing unit 114 classifies the influencing factor data 123 for each type of emotion by saving the influencing factor data 123, obtained in an elicitation time related to the eliciting of the target person’s emotion at the given time, in the storage unit 12 in association with the type of the target person’s emotion at the specified given time.

(Classification Process)
The process by which the influencing factor data 123 is classified for each type of emotion will be described further using FIG. 6. FIG. 6 schematically illustrates an example of a situation in which the influencing factor data 123 obtained within an elicitation time is associated with a type of identified emotion. In FIG. 6, the various data surrounded by dotted lines indicate data obtained within the elicitation time. In the examples illustrated in FIGS. 5 and 6, emotion types are indicated by unique states such as “joy”, “anger”, and so on, in the same manner as the example illustrated in FIG. 1. However, as described above, the expression of the types of emotions need not be limited to this example, and the types may be expressed as numerical expressions or the like using one or more parameters.

In the example illustrated in FIG. 6, the influencing factor data 123 includes biological data 1231, environment data 1232, and event data 1233. The biological data 1231 is data that can be obtained from a biological data 1231 attached to the target person’s body, and may be data indicating at least one of heart rate, pulse rate, breathing rate, body temperature, blood pressure, brain waves, posture, and myoelectricity, for example. The environment data 1232 is data pertaining to the target person’s surrounding environment, and may be data indicating at least one of temperature, humidity, brightness, weather, atmospheric pressure, noise, vibrations, and odors, for example. The event data 1233 is data indicating events occurring for the target person, and can be obtained by the target person inputting an event that has occurred through the input device him/herself. The event data 1233 may have any content as long as the data can indicate types of events that have occurred for the target person. Furthermore, the data is not limited to being inputted by the target person, and may instead be specified using the above-described biological data 1231 and environment data 1232. For example, an accelerometer may be attached to the target person, and a predetermined event may be determined to have occurred when a value obtained by the accelerometer exceeds a predetermined value. In this case, the predetermined event is an event that can be detected on the basis of an acceleration, such as the target person colliding with an object or the target person falling over, for example. This makes it possible to detect a predetermined event without inputs make through the input device. The event data 1233 may be inputted by a third party rather than the target person him/herself.

The target person’s emotion at the given time is assumed to be elicited by an influencing factor arising between that time and a time previous to that time. Accordingly, in the present embodiment, a time period related to the eliciting of the target person’s emotion at the given time, or in other words, a time period which an influencing factor that has elicited the target person’s emotion at the given time can be present, is set as the elicitation time. For example, the elicitation time may be set to a time period spanning from a time when the type of the target person's emotion is detected using the emotion expression data 122, to a time a predetermined amount of time before the stated time. However, the elicitation time need not be limited to this example, and may be set as appropriate in accordance with the embodiment. The end point of the elicitation time may be before or after the time at which the type of emotion has been specified. Note that the elicitation time may be set automatically on the basis of the process, executed by the emotion specifying unit 113, for specifying the type of emotion on the basis of the emotion expression data 122. For example, when the emotion specifying unit 113 has specified the type of a predetermined emotion on the basis of the emotion expression data 122 of a predetermined time (or period), a predetermined duration of time may be provided on the basis of that predetermined time (or period) and used as the elicitation time.

The elicitation time may be shared among a plurality of types of the influencing factor data 123. In other words, the classification processing unit 114 may save the biological data 1231, the environment data 1232, and the event data 1233 obtained within a common elicitation time in association with the specified type of emotion. However, it is assumed that the time period in which the influencing factor affects the eliciting of the emotion can differ depending on the type of the influencing factor. Specifically, it is assumed that an influencing factor that easily elicits an emotion has a shorter elicitation time, whereas an influencing factor that does not easily elicit an emotion has a longer elicitation time. The elicitation time may therefore be set for each type of the influencing factor data 123.

In the present embodiment, influencing factors indicated by the biological data 1231 have the most influence on eliciting emotions, influencing factors indicated by the environment data 1232 have the next most influence on eliciting emotions, and influencing factors indicated by the event data 1233 have the least influence on eliciting emotions. Accordingly, in the example illustrated in FIG. 6, the elicitation time set for the biological data 1231 is the shortest, the elicitation time set for the environment data 1232 is the next shortest, and the elicitation time set for the event data 1233 is the longest. However, the setting of the elicitation time need not be limited to this example. For example, the elicitation time set for the event data 1233 may be the shortest. Note that if the biological data 1231 includes a plurality of types of data selected from heart rate, pulse rate, breathing rate, body temperature, blood pressure, brain waves, posture, and myoelectricity, for example, the elicitation time may be different for each type of the biological data 1231. The same applies to the environment data 1232 and the event data 1233.

The biological data 1231 is assumed to be both data indicating influencing factors eliciting the target person's emotions and data that can vary as a result of the emotions arising (elicited) in the target person. For example, it is assumed that the blood pressure included in the biological data 1231 will rise as a result of the emotion of “anger” being elicited. Thus in order to obtain data before and after the emotion arising (elicited) in the target person, the end point of the elicitation time set for the biological data 1231 may be set to be relatively later than those of the environment data 1232 and the event data 1233.

The classification processing unit 114 classifies the influencing factor data 123 for each type of emotion by associating the biological data 1231, the environment data 1232, and the event data 1233 obtained in the respective elicitation times with the specified type of emotion. The classification processing unit 114 saves the influencing factor data 123 classified for each type of emotion in the storage unit 12. The association may be carried out as appropriate using a known method. For example, the association may be carried out through labeling such as adding tags. In other words, the classification processing unit 114 may add tags (identification data) indicating the type of the specified emotion to the respective pieces of data 1231 to 1233 obtains in the elicitation times. In this case, the type of emotion associated with the pieces of data 1231 to 1233 can be identified using the added tags.

In the present embodiment, there may be influencing factor data 123 not associated with any type of emotion, depending on the time at which the type of emotion is specified and the elicitation time. In the example illustrated in FIG. 6, the data not surrounded by dotted lines corresponds to such unassociated influencing factor data 123. Such unassociated influencing factor data 123 can also arise when an emotion specified by the emotion expression data 122 is an emotion not subjected to data collection, and when a target person’s emotion at a given time could not be specified using the emotion expression data 122. As such, the classification processing unit 114 according to the present embodiment is configured to delete influencing factor data 123 no classified into any type of emotion.

The specified emotion is assumed to arise not only at a predetermined time, but also continuously for a predetermined period. In this case, too, the elicitation time can be set in the same manner as when the specified emotion arises at a predetermined time. In other words, the elicitation time may be set to a time period spanning from the start time of the predetermined period in which the specified emotion arises to a time before that start time by an amount equivalent to a predetermined first time period. Additionally, the elicitation time may be set to a time period spanning from the end time of the predetermined period in which the specified emotion arises to a time after that end time by an amount equivalent to a predetermined second time period. Here, it is preferable that the first time period be longer than the second time period.

(Learning Device)
The learning device will be described next. As illustrated in FIG. 5, the data collection apparatus 1 according to the present embodiment uses the neural network 6 as a trained learning device that has been trained to specify the target person’s emotion expressed by the emotion expression data 122. The neural network 6 is a multilayer neural network used in what is known as deep learning, and has an input layer 61, an intermediate layer (hidden layer) 62, and an output layer 63, in order from the input side.

In FIG. 5, the neural network 6 includes one intermediate layer 62, such that the output of the input layer 61 is the input of the intermediate layer 62 and the output of the intermediate layer 62 is the input of the output layer 63. However, the number of intermediate layers 62 need not be limited to one, and the neural network 6 may include two or more intermediate layers 62.

Each of the layers 61 to 63 includes one or more neurons. For example, the number of neurons in the input layer 61 can be set in accordance with the emotion expression data 122. The number of neurons in the intermediate layer 62 can be set as appropriate in accordance with the embodiment. Additionally, the number of neurons in the output layer 63 can be set in accordance with the number of types of emotions subject to data collection.

The neurons in adjacent layers are connected to each other as appropriate, and a weight is set for each connection (a connection weight). In the example in FIG. 5, each neuron is connected to all of the neurons in the adjacent layers, but the connections of the neurons need not be limited to this example, and may be set as appropriate in accordance with the embodiment.

A threshold is set for each neuron, and the output of each neuron is basically determined on the basis of whether or not a sum of the products of the neurons and their weights exceeds the threshold. The emotion specifying unit 113 specifies the target person's emotion at a given time on the basis of output values obtained from the output layer 63 as a result of inputting the emotion expression data 122 obtained at that given time into the input layer 61 of the neural network 6.

Information indicating the configuration of the neural network 6 (for example, the number of layers in the neural network 6, the number of neurons in each layer, the connection relationships between neurons, and transfer functions of the neurons), the weights of the connections between the neurons, and the thresholds for the neurons is included in the expressed emotion learning result data 124. The data collection apparatus 1 configures the trained neural network 6 used in the process of specifying the type of the target person's emotion expressed by the emotion expression data 122 obtained at the given time by referring to the expressed emotion learning result data 124.

<First Learning Apparatus>
An example of the functional configuration of the first learning apparatus 2 according to the present embodiment will be described next using FIG. 7. FIG. 7 schematically illustrates an example of the functional configuration of the first learning apparatus 2 according to the present embodiment.

The control unit 21 of the first learning apparatus 2 loads the first learning program 221 stored in the storage unit 22 into the RAM. The control unit 21 then controls the various constituent elements by using the CPU to interpret and execute the first learning program 221 loaded into the RAM. As a result, as illustrated in FIG. 7, the first learning apparatus 2 according to the present embodiment functions as a computer including a data obtainment unit 211 and a learning processing unit 212.

The data obtainment unit 211 obtains the influencing factor data classified for each type of emotion as learning data 222. In the present embodiment, the data obtainment unit 211 obtains the influencing factor data 123 classified for each type of emotion from the data collection apparatus 1. The influencing factor data 123 serves as input data, and a value 1234 indicating the type of emotion associated with the influencing factor data 123, or in other words, a value 1234 indicating an emotion elicited by the influencing factor expressed by the influencing factor data 123, serves as training data. Then, the learning processing unit 212 trains the learning device so that when the influencing factor data 123 is inputted, the learning device outputs the value 1234 indicating the type of the emotion associated with the inputted influencing factor data 123. The training data may also be called “correct data”.

As illustrated in FIG. 7, in the present embodiment, the learning device whose capability for estimating elicited emotions from influencing factors is to undergo machine learning is the neural network 7. The neural network 7, which is an example of the learning device, has the same configuration as the neural network 6. In other words, the neural network 7 is a multilayer neural network used in what is known as deep learning, and has an input layer 71, an intermediate layer (hidden layer) 72, and an output layer 73, in order from the input side. The number of intermediate layers 72, the number of neurons in the layers 71 to 73, and the connections between neurons in adjacent layers may be set as appropriate in accordance with the embodiment.

Through the learning process for the neural network, the learning processing unit 212 constructs the neural network 7 so that when the influencing factor data 123 is inputted into the input layer 71, the value 1234 indicating the type of the emotion associated with the inputted influencing factor data 123 is outputted from the output layer 73. The learning processing unit 212 then stores the information indicating the configuration of the constructed neural network 7, the weights of the connections between the neurons, and the thresholds for the neurons in the storage unit 22 as the elicited emotion learning result data 223.

<Second Learning Apparatus>
An example of the functional configuration of the second learning apparatus 3 according to the present embodiment will be described next using FIG. 8. FIG. 8 schematically illustrates an example of the functional configuration of the second learning apparatus 3 according to the present embodiment.

The control unit 31 of the second learning apparatus 3 loads the second learning program 321 stored in the storage unit 32 into the RAM. The control unit 31 then controls the various constituent elements by using the CPU to interpret and execute the second learning program 321 loaded into the RAM. As a result, as illustrated in FIG. 8, the second learning apparatus 3 according to the present embodiment functions as a computer including a data obtainment unit 311 and a learning processing unit 312.

The data obtainment unit 311 obtains a plurality of pieces of the learning data 322 used to carry out machine learning on the capability of specifying the target person’s emotion expressed by the emotion expression data. Each piece of the learning data 322 is constituted of, for example, a combination of emotion expression data 3221 expressing a person’s emotion and a value 3222 indicating the type of the person’s emotion expressed by the emotion expression data 3221. The emotion expression data 3221 can be obtained using the first sensor 51, in the same manner as the above-described emotion expression data 122. The emotion expression data 3221 is input data. The value 3222 indicating the type of the person’s emotion expressed by the emotion expression data 3221 is training data. The learning processing unit 312 trains the learning device so that when the emotion expression data 3221 is inputted, the learning device outputs the value 3222 indicating the type of the person's emotion expressed by the inputted emotion expression data 3221.

As illustrated in FIG. 8, in the present embodiment, the learning device whose capability for specifying the target person's emotion expressed by the emotion expression data is to undergo machine learning is the neural network 8. The neural network 8, which is an example of the learning device, has the same configuration as the neural network 6. In other words, the neural network 8 is a multilayer neural network used in what is known as deep learning, and has an input layer 81, an intermediate layer (hidden layer) 82, and an output layer 83, in order from the input side. The number of intermediate layers 82, the number of neurons in the layers 81 to 83, and the connections between neurons in adjacent layers may be set as appropriate in accordance with the embodiment.

Through the learning process for the neural network, the learning processing unit 312 constructs the neural network 8 so that when the emotion expression data 3221 is inputted into the input layer 81, the value 3222 indicating the type of the person’s emotion expressed by the inputted emotion expression data 3221 is outputted from the output layer 83. The learning processing unit 312 then stores the information indicating the configuration of the constructed neural network 8, the weights of the connections between the neurons, and the thresholds for the neurons in the storage unit 32 as the expressed emotion learning result data 124.

<Other>
The various functions of the data collection apparatus 1, the first learning apparatus 2, and the second learning apparatus 3 will be described in detail later in an operation example. The present embodiment describes an example in which all of the functions of the data collection apparatus 1, the first learning apparatus 2, and the second learning apparatus 3 are realized by generic CPUs. However, some or all of the above-described functions may be realized by one or more dedicated processors. With respect to the functional configurations of the data collection apparatus 1, the first learning apparatus 2, and the second learning apparatus 3, functions may be omitted, replaced, or added as appropriate in accordance with the embodiment.

§3 Operation Example
≪Data Collection Apparatus≫
Next, an example of the operations of the data collection apparatus 1 will be described using FIG. 9. FIG. 9 is a flowchart illustrating an example of a processing sequence carried out by the data collection apparatus 1. Note that the processing sequence described hereinafter is merely an example, and the processes may be changed as possible. Furthermore, in the processing sequence described hereinafter, steps may be omitted, replaced, or added as appropriate in accordance with the embodiment.

(Startup)
First, a user starts up the data collection apparatus 1. In response to this, the control unit 11 of the data collection apparatus 1 that has been started up executes the collection program 121. Additionally, referring to the expressed emotion learning result data 124, the control unit 11 constructs the neural network 6, weights the connections between the neurons, and sets the thresholds for the neurons. The control unit 11 then carries out a process for collecting the influencing factor data 123 for each type of emotion, in accordance with the processing sequence described hereinafter.

(Step S101)
In step S101, functioning as the first obtainment unit 111, the control unit 11 continuously obtains the emotion expression data 122 expressing a person’s emotions. Additionally, functioning as the second obtainment unit 112, the control unit 11 continuously obtains the influencing factor data 123 pertaining to factors that can influence the person’s emotions, parallel with the continuous obtainment of the emotion expression data 122.

The emotion expression data 122 is, for example, at least one of image data showing the target person’s face, data obtained by polygraph, audio data containing a recording of the target person’s voice, and response data from the target person. The emotion expression data 122 can be obtained by using a camera, a polygraph, a microphone, or an input device for making responses as the first sensor 51.

Additionally, the influencing factor data 123 includes, for example, the biological data 1231, the environment data 1232, and the event data 1233. The biological data 1231 is data indicating at least one of heart rate, pulse rate, breathing rate, body temperature, blood pressure, brain waves, posture, and myoelectricity, for example. The environment data 1232 is data indicating at least one of temperature, humidity, brightness, weather, atmospheric pressure, noise, vibrations, and odors, for example. The event data 1233 is data indicating an event that has occurred for the target person. Such influencing factor data 123 can be obtained by using a biological sensor, an environment sensor, an input device, or the like as the second sensor 52.

Note that the obtainment of the emotion expression data 122 and the influencing factor data 123 may be carried out by one or more other information processing apparatuses aside from the data collection apparatus 1. In this case, the one or more other information processing apparatuses can obtain the emotion expression data 122 and the influencing factor data 123 by using the first sensor 51 and second sensor 52. The control unit 11 can then obtain the emotion expression data 122 and the influencing factor data 123 from the one or more other information processing apparatuses over a network, from the storage medium 91, or the like. In other words, the control unit 11 may obtain the emotion expression data 122 and the influencing factor data 123 indirectly rather than directly.

(Step S102)
Next, in step S102, functioning as the emotion specifying unit 113, the control unit 11 specifies a type of emotion of the target person at a given time on the basis of the emotion expression data 122. In the present embodiment, the type of the target person’s emotion at the given time, expressed by the emotion expression data 122, is specified using the trained neural network 6. In the present embodiment, the internal configuration of the trained neural network 6 is set on the basis of the expressed emotion learning result data 124 obtained as a result of machine learning carried out by the second learning apparatus 3, which will be described later.

Specifically, the control unit 11 inputs the emotion expression data 122 from the given time, obtained in step S101, to the input layer 61 of the neural network 6. The correspondence relationships between the inputted emotion expression data 122 and the neurons in the input layer 61 may be set as appropriate in accordance with the embodiment. Next, by determining whether each of the neurons in the layers 61 to 63 is firing along the downstream direction, the control unit 11 obtains an output value indicating the type of the target person’s emotion expressed by the emotion expression data 122 obtained at the given time from the output layer 63 of the neural network 6. Through this, the control unit 11 specifies the type of emotion of the target person at the given time as expressed in the emotion expression data 122. Assuming that the same type of emotion may be continuous, the “given time” can be replaced with a “predetermined period”. In other words, the control unit 11 may specify the type of emotion of the target person in a predetermined period as expressed in the emotion expression data 122.

(Step S103)
Next, in step S103, functioning as the classification processing unit 114, the control unit 11 saves the influencing factor data 123, obtained in the elicitation time related to the eliciting of the target person’s emotion at the given time, in association with the type of the target person’s emotion at the given time (or predetermined period) specified in step S102. Through this, the control unit 11 classifies the influencing factor data 123 for each type of emotion and stores that data in the storage unit 12.

As described above, in the present embodiment, elicitation times of different lengths are set for the biological data 1231, the environment data 1232, and the event data 1233, respectively. The control unit 11 adds a tag indicating the type of the emotion specified in step S102 to the biological data 1231, the environment data 1232, and the event data 1233 obtained in the elicitation times set for each thereof. Through this, the control unit 11 saves the influencing factor data 123, obtained in the elicitation time related to the eliciting of the target person’s emotion at the given time, in association with the type of the target person’s emotion at the given time (or predetermined period) specified in step S102. In other words, the emotion is specified on the basis of the emotion expression data 122 associated with the type of emotion, and the influencing factor data 123 obtained in the elicitation time of that emotion (that is, the biological data 1231, the environment data 1232, and the event data 1233) is associated with the specified type of emotion.

Note that the format in which the influencing factor data 123 is saved may be selected as appropriate in accordance with the embodiment. For example, the control unit 11 may use a known database technique to construct a database that, when a type of emotion is provided as a query, outputs the influencing factor data 123 associated with that type of emotion.

(Step S104)
Next, in step S104, functioning as the classification processing unit 114, the control unit 11 deletes the influencing factor data 123 not classified into any type of emotion in step S103. For example, the control unit 11 deletes the influencing factor data 123 not surrounded by dotted lines in the aforementioned example illustrated in FIG. 6. Once the processing of step S104 ends, the control unit 11 ends the processing according to the present operation example.

≪First Learning Apparatus≫
Next, an example of the operations of the first learning apparatus 2 will be described using FIG. 10. FIG. 10 is a flowchart illustrating an example of a processing sequence carried out by the first learning apparatus 2. Note that the processing sequence described hereinafter is merely an example, and the processes may be changed as possible. Furthermore, in the processing sequence described hereinafter, steps may be omitted, replaced, or added as appropriate in accordance with the embodiment.

(Step S201)
In step S201, functioning as the data obtainment unit 211, the control unit 21 obtains the influencing factor data classified for each type of emotion. For example, the control unit 21 obtains the influencing factor data 123 classified for each type of emotion from the data collection apparatus 1 over a network, the storage medium 92, or the like. By handling the influencing factor data 123 obtained within the elicitation time for each time the target person’s emotion has been specified as a single piece of data, the influencing factor data 123 classified for each type of emotion can be used as a plurality of pieces of the learning data 222.

Note that the control unit 21 may obtain the influencing factor data 123 classified for each type of emotion collected through a method aside from the method used by the data collection apparatus 1 described above. The collection of the influencing factor data 123 classified for each type of emotion can be carried out as follows, for example. In other words, the influencing factor data 123 is obtained by the second sensor 52 in a state where the emotion elicited in the target person can be specified. Then, a value indicating the type of the emotion elicited in the target person is associated with the influencing factor data 123 obtained by being inputted by an operator through the input device. The influencing factor data 123 classified for each type of emotion can be collected by repeating this series of processes.

The collection of the influencing factor data 123 classified for each type of emotion may be carried out by the first learning apparatus 2, or may be carried out by another information processing apparatus aside from the first learning apparatus 2. When the first learning apparatus 2 collects the influencing factor data 123 through the above-described method, the influencing factor data 123 classified for each type of emotion can be obtained by the control unit 21 executing the above-described process for collecting the influencing factor data 123 in step S201. However, when another information processing apparatus aside from the first learning apparatus 2 collects the influencing factor data 123 through the above-described method, the control unit 21 can obtain the influencing factor data 123 classified for each type of emotion collected by the other information processing apparatus over a network, from the storage medium 92, or the like.

(Step S202)
Next, in step S202, functioning as the learning processing unit 212, the control unit 21 subjects the neural network 7 to machine learning using the plurality of pieces of learning data 222 obtained in step S201.

Specifically, first, the control unit 21 prepares the neural network 7 to be subjected to the machine learning. The configuration of the prepared neural network 7, the default values of the weights on the connections between the neurons, and the default values of the thresholds for the neurons may be provided by a template, or may be provided by inputs made by the operator. If retraining is carried out, the control unit 21 may prepare the neural network 7 on the basis of the elicited emotion learning result data 223 subject to the retraining.

Next, the control unit 21 carries out the learning process for the neural network 7 using the influencing factor data 123 obtained in step S201 as input data and the value 1234 indicating the type of the emotion associated with the influencing factor data 123 as training data. Gradient descent, probabilistic gradient descent, or the like may be used in the learning process for the neural network 7.

For example, the control unit 21 carries out computational processing in the downstream direction of the neural network 7 using the influencing factor data 123 as the input to the input layer 71. As a result, the control unit 21 obtains output values from the output layer 73 of the neural network 7. Next, the control unit 21 calculates error between the output value obtained from the output layer 73 and the value 1234 indicating the type of emotion. Next, through differential reverse propagation, the control unit 21 calculates error in the weights of the connections between the neurons and in the thresholds for the neurons using the calculated error in the output values. Then, on the basis of the calculated errors, the control unit 21 updates the values of the weights of the connections between the neurons and the thresholds for the neurons.

The control unit 21 trains the neural network 7 by repeating this series of processes for each piece of the learning data 222 until the output value outputted from the output layer 73 matches the corresponding value 1234 indicating the type of emotion. Through this, the trained neural network 7 can be constructed so that when the influencing factor data 123 is inputted, the value 1234 indicating the type of the emotion associated with the inputted influencing factor data 123 is outputted.

(Step S203)
Next, in step S203, functioning as the learning processing unit 212, the control unit 21 stores the information indicating the configuration of the constructed neural network 7, the weights of the connections between the neurons, and the thresholds for the neurons in the storage unit 22 as the elicited emotion learning result data 223. Through this, the control unit 21 ends the process of training the neural network 7 according to this operation example.

Note that the control unit 21 may transfer the created elicited emotion learning result data 223 to an apparatus where the trained neural network 7 is desired to be used (called an “elicited emotion estimation apparatus” hereinafter) after the process of the above-described step S203 is complete. The control unit 21 may also periodically update the elicited emotion learning result data 223 by periodically executing the learning process of steps S201 to S203. The control unit 21 may periodically update the elicited emotion learning result data 223 held in the elicited emotion estimation apparatus by transferring the created elicited emotion learning result data 223 to the elicited emotion estimation apparatus each time the learning process is executed.

＜Second Learning Apparatus＞
Next, an example of the operations of the second learning apparatus 3 will be described using FIG. 11. FIG. 11 is a flowchart illustrating an example of a processing sequence carried out by the second learning apparatus 3. Note that the processing sequence described hereinafter is merely an example, and the processes may be changed as possible. Furthermore, in the processing sequence described hereinafter, steps may be omitted, replaced, or added as appropriate in accordance with the embodiment.

(Step S301)
In step S301, functioning as the data obtainment unit 311, the control unit 31 obtains a plurality of pieces of the learning data 322 used in machine learning. The learning data 322 is constituted of a combination of the emotion expression data 3221 expressing the person’s emotion and the value 3222 indicating the type of the person’s emotion expressed by the emotion expression data 3221.

This learning data 322 can be collected through the following method, for example. The first sensor 51 obtains a plurality of pieces of the emotion expression data 3221 expressing a variety of peoples’ emotions. For example, if image data is used as the emotion expression data 3221, the plurality of pieces of the emotion expression data 3221 expressing a variety of peoples’ emotions can be obtained by causing a variety of people to make expressions according to each type of emotion and then capturing images of the people with a camera so that the expressions appear in the images. Then, the value 3222 indicating the type of the emotion expressed by each piece of the emotion expression data 3221 is associated with each piece of the emotion expression data 3221 by an operator making inputs through the input device. A plurality of pieces of learning data 322 can be collected as a result.

The process of collecting the plurality of pieces of learning data 322 may be carried out by the second learning apparatus 3, or may be carried out by another information processing apparatus aside from the second learning apparatus 3. When the second learning apparatus 3 collects the multiple pieces of learning data 322 through the above-described method, the control unit 31 can obtain the plurality of pieces of the learning data 322 in step S301 by executing a process for collecting the learning data 322. However, when another information processing apparatus aside from the second learning apparatus 3 collects the plurality of pieces of learning data 322 through the above-described method, the control unit 31 can, in the present step S301, obtain the plurality of pieces of learning data 322 collected by the other information processing apparatus over a network, from the storage medium 93, or the like.

(Step S302)
Next, in step S302, functioning as the learning processing unit 312, the control unit 31 subjects the neural network 8 to machine learning using the plurality of pieces of learning data 322 obtained in step S301.

The machine learning of the neural network 8 can be carried out through the same method as in the above-described step S202. That is, for each piece of the learning data 322, the control unit 31 repeats a process for updating the values of the weights on the connections between neurons and the thresholds of the neurons until the output value outputted from the output layer 83 in response to the emotion expression data 3221 being inputted into the input layer 81 matches the value 3222 indicating the corresponding type of emotion. Through this, the control unit 31 can construct the trained neural network 8 so that when the emotion expression data 3221 is inputted, the value 3222 indicating the type of the person's emotion expressed by the inputted emotion expression data 3221 is outputted.

(Step S303)
Next, in step S303, functioning as the learning processing unit 312, the control unit 31 stores the information indicating the configuration of the constructed neural network 8, the weights of the connections between the neurons, and the thresholds for the neurons in the storage unit 32 as the expressed emotion learning result data 124. Through this, the control unit 31 ends the process of training the neural network 8 according to this operation example.

Note that the control unit 31 may transfer the created expressed emotion learning result data 124 to the data collection apparatus 1 after the process of step S303 is complete. The control unit 31 may also periodically update the expressed emotion learning result data 124 by periodically executing the learning process of steps S301 to S303. The control unit 31 may periodically update the expressed emotion learning result data 124 held in the data collection apparatus 1 by transferring the created expressed emotion learning result data 124 to the data collection apparatus 1 each time the learning process is executed.

＜Effects＞
As described thus far, with the data collection apparatus 1 according to the present embodiment, the influencing factor data 123 pertaining to factors than can influence the target person’s emotions can be classified and collected for each type of emotion through the above-described processing of steps S101 to S103. Factors that elicit the target person’s emotions can be analyzed by using the influencing factor data 123 classified for each type of emotion. The data collection apparatus 1 according to the present embodiment thus makes it possible to estimate factors eliciting the target person's emotions. Additionally, the obtained influencing factor data 123 is saved only for the predetermined elicitation time in which the emotion is expressed, and thus the amount of data that is saved can be suppressed.

Additionally, according to the present embodiment, in the above-described step S102, the trained neural network 6 is used to specify the type of the target person's emotion expressed by the emotion expression data 122. Through this, the type of the target person's emotion expressed by the emotion expression data 122 can be easily and appropriately specified, even when using emotion expression data 122 in which it is comparatively difficult to specify the emotion being expressed, such as image data showing a face or audio data in which a voice is recorded.

Additionally, according to the present embodiment, in the above-described step S103, the elicitation time defining the duration associated with the specified type of emotion is set for each type of the influencing factor data 123. Furthermore, in step S104, influencing factor data 123 not classified for any type of emotion is deleted. Thus according to the present embodiment, an appropriate amount of the influencing factor data 123 for expressing the influencing factors of elicited emotions can be collected, and as a result, the storage unit 12 that stores the influencing factor data 123 can be used efficiently.

The first learning apparatus 2 according to the present embodiment uses the influencing factor data 123 classified for each type of emotion to construct a learning device (the trained neural network 7) that estimates the type of emotion elicited in the target person on the basis of influencing factors. Thus with the trained neural network 7, emotions elicited in a target person can be estimated from the influencing factor data. Additionally, the learning device is constructed through machine learning using the influencing factor data 123 obtained only within the predetermined elicitation time and classified for each type of emotion, and thus the learning time can be reduced, and a learning device having an excellent accuracy of estimating eliciting factors can be constructed.

For example, when an automobile is driving autonomously, if the trained neural network 7 is used, emotions elicited in a driver and a passenger can be estimated on the basis of biological data obtained from the driver and the passenger and environment data obtained from the environment within the vehicle. Accordingly, the autonomous driving and the environment within the vehicle can be adjusted to make the driver and passenger more comfortable.

Additionally, for example, in a work site such as a factory, if the trained neural network 7 is used, emotions elicited in a worker can be estimated on the basis of biological data obtained from the worker and environment data obtained from the work environment within the factory. Accordingly, points of improvement in the work site can be found to make the worker more comfortable.

§4 Variations
Although an embodiment of the present invention has been described in detail thus far, the foregoing descriptions are intended to be nothing more than an example of the present invention in all senses. It goes without saying that many improvements and changes can be made without departing from the scope of the present invention. For example, variations such as those described below are also possible. In the following, constituent elements that are the same as those in the above-described embodiment will be given the same reference signs, and points that are the same as in the above-described embodiment will not be described. The following variations can also be combined as appropriate.

<4.1>
In the embodiment described above, in step S102, the trained neural network 6 is used to specify the type of the target person's emotion expressed by the emotion expression data 122. However, the method for specifying the type of the target person’s emotion expressed by the emotion expression data 122 need not be limited to this example, and the trained neural network 6 need not be used.

For example, if emotion expression data 122 that can express the type of emotion directly, such as data obtained by polygraph or response data from the target person, is used, the control unit 11 may specify the type of the target person’s emotion expressed by the emotion expression data 122 by, for example, comparing a value expressed by the emotion expression data 122 to a threshold in step S102. If, for example, image data in which the target person’s face appears is used as the emotion expression data 122, the control unit 11 may specify the type of the target person’s emotion expressed in the image data by subjecting the image data to known image analysis in step S102. Additionally, if, for example, audio data in which the target person’s voice is recorded is used as the emotion expression data 122, the control unit 11 may specify the type of the target person’s emotion expressed in the audio data by subjecting the audio data to known audio analysis in step S102.

<4.2>
In the embodiment described above, the influencing factor data 123 includes the biological data 1231, the environment data 1232, and the event data 1233. However, the configuration of the influencing factor data 123 need not be limited to this example, and at least one of the biological data 1231, the environment data 1232, and the event data 1233 may be omitted, for example. Additionally, the influencing factor data 123 may be configured to include at least two of the biological data 1231, the environment data 1232, and the event data 1233. In this case, too, a different elicitation time may be set for the biological data 1231, the environment data 1232, and the event data 1233, in the same manner as in the above-described embodiment.

<4.3>
In the embodiment described above, in step S103, the control unit 11 associates the influencing factor data 123 obtained in the elicitation time with the type of emotion specified at the given time. However, the data associated with the specified type of emotion need not be limited to the influencing factor data 123.

For example, the control unit 11 may associate the type of emotion specified in step S102 with the emotion expression data 122 from the given time used in that specification of the type of emotion. In this case, the first learning apparatus 2 may obtain the emotion expression data 122 and the influencing factor data 123 classified for each type of emotion from the data collection apparatus 1. Then, the first learning apparatus 2 may construct the trained neural network 7 that, when the emotion expression data 122 and the influencing factor data 123 are inputted, outputs values indicating the corresponding type of emotion.

Additionally, for example, the control unit 11 may associate attribute data indicating attributes of the target person with the type of emotion specified in step S102, as illustrated in FIGS. 12 to 14. FIGS. 12 to 14 schematically illustrate an example of the functional configuration of a data collection apparatus 1A, a first learning apparatus 2A, and a second learning apparatus 3A according to the present variation.

As illustrated in FIG. 12, the classification processing unit 114 of the data collection apparatus 1A according to the present variation is configured to further associate attribute data 125 indicating attributes of the target person with the influencing factor data 123 classified for each type of the target person’s emotion. A trained neural network 6A is constructed so that when the emotion expression data 122 and the attribute data 125 are inputted, values indicating the corresponding type of emotion are outputted. Aside from the attribute data 125 being able to be inputted to the input layer, the neural network 6A can have the same configuration as the above-described neural network 6. Expressed emotion learning result data 124A includes information indicating the configuration of the trained neural network 6A, the weights of the connections between neurons, and thresholds of the neurons. Aside from these points, the data collection apparatus 1A has the same configuration as the above-described data collection apparatus 1. Note that in the present variation, the trained neural network 6A may, like the above-described trained neural network 6, be constructed so that when the emotion expression data 122 is inputted, values indicating the corresponding type of emotion are outputted.

In the case of the present variation, the control unit 11 obtains the attribute data 125 of the target person as appropriate. For example, prior to the process of the above-described step S101, the control unit 11 accepts the input of a target person's attributes from an operator or the like and obtains the attribute data 125 on the basis of the accepted input. The target person's attributes can be defined by at least one of sex, height, weight, blood type, age, date of birth, place of birth, nationality, profession, and so on, for example.

Next, in the above-described step S102, the control unit 11 obtains values indicating the type of the target person's emotion expressed by the emotion expression data 122 from the output layer of the trained neural network 6A by inputting the emotion expression data 122 and the attribute data 125 into the input layer of the trained neural network 6A. The control unit 11 may then associate the obtained attribute data 125 with the influencing factor data 123 classified and saved for each type of emotion in the above-described step S103. Through this, the influencing factor data 123 pertaining to factors than can influence the target person’s emotions can be classified and saved for each of the target person's attributes and each type of emotion.

Additionally, as illustrated in FIG. 13, the data obtainment unit 211 of the first learning apparatus 2A according to the present variation is configured to obtain the influencing factor data 123 classified for each of the target person's attributes and each type of emotion. The learning processing unit 212 is configured to train a neural network 7A so that when the influencing factor data 123 and the attribute data 125 are inputted, the value 1234 indicating the corresponding type of emotion is outputted. Aside from the attribute data 125 further being able to be inputted to the input layer, the neural network 7A can have the same configuration as the above-described neural network 7. Aside from these points, the first learning apparatus 2A has the same configuration as the above-described first learning apparatus 2.

In the case of the present variation, the control unit 21 obtains the influencing factor data 123 classified for each of the target person's attributes and for each type of emotion from the data collection apparatus 1A, for example, in the above-described step S201. Next, through the learning process of step S202, the control unit 21 constructs the neural network 7A so that when the influencing factor data 123 and the attribute data 125 are inputted, the value 1234 indicating the corresponding type of emotion is outputted. Then, in step S203, control unit 21 stores the information indicating the configuration of the constructed neural network 7A, the weights of the connections between the neurons, and the thresholds for the neurons in the storage unit 22 as elicited emotion learning result data 223A. Through this, the trained neural network 7A that estimates the type of emotion elicited in the target person can be constructed on the basis of the target person’s attributes and the influencing factors. Thus according to the present variation, the trained neural network 7A having a high level of accuracy in estimating the type of emotion elicited in the target person can be constructed.

Additionally, as illustrated in FIG. 14, the data obtainment unit 311 of the second learning apparatus 3A according to the present variation is configured to obtain the learning data 322 associated with the attribute data 125 indicating the target person’s attributes. The learning processing unit 312 is configured to train a neural network 8A so that when the emotion expression data 3221 and the attribute data 125 are inputted, the value 3222 indicating the type of emotion expressed by the emotion expression data 3221 is outputted. Aside from the attribute data 125 being able to be inputted to the input layer, the neural network 8A can have the same configuration as the above-described neural network 8. Aside from these points, the second learning apparatus 3A has the same configuration as the above-described second learning apparatus 3.

In the case of the present variation, the control unit 31 obtains the learning data 322 associated with the attribute data 125 indicating the target person's attributes in step S301. Next, through the learning process of step S302, the control unit 31 constructs the neural network 8A so that when the emotion expression data 3221 and the attribute data 125 are inputted, the value 3222 indicating the type of emotion expressed by the emotion expression data 3221 is outputted. Then, in step S303, control unit 31 stores the information indicating the configuration of the constructed neural network 8A, the weights of the connections between the neurons, and the thresholds for the neurons in the storage unit 32 as the expressed emotion learning result data 124A. The expressed emotion learning result data 124A for configuring the trained neural network 6A used by the data collection apparatus 1A can be obtained as a result.

<4.4>
In the embodiment described above, the influencing factor data 123 pertaining to factors than can influence the target person’s emotions can be classified and collected for each type of emotion through the above-described processing of steps S101 to S103. Accordingly, the data collection apparatus 1 according to the above-described embodiment may be configured to analyze factors eliciting the target person’s emotions by using the influencing factor data 123 classified for each type of emotion.

FIG. 15 schematically illustrates an example of the functional configuration of a data collection apparatus 1B according to the present variation. As illustrated in FIG. 15, the data collection apparatus 1B according to the present variation is configured in the same manner as the above-described data collection apparatus 1, aside from further including a factor estimation unit 115 that estimates factors eliciting an emotion by analyzing the influencing factor data 123 classified for each type of emotion. In this case, after step S103, functioning as the factor estimation unit 115, the control unit 11 estimates factors eliciting each emotion by analyzing the influencing factor data 123 classified for each type of emotion. The analysis method may be selected as appropriate in accordance with the embodiment. For example, the control unit 11 may estimate the factors eliciting each emotion from the influencing factor data 123 classified for each type of emotion, through a statistical method that uses an appearance frequency or the like. Alternatively, the control unit 11 may estimate the factors eliciting each emotion from the influencing factor data 123 classified for each type of emotion, through a method using machine learning such as unsupervised learning or reinforcement learning. According to this variation, factors that elicit the target person’s emotions can be estimated by using the influencing factor data 123 classified and collected for each type of emotion.

<4.5>
In the embodiment described above, typical feed-forward neural networks having multilayer structures are used as the neural networks 6 to 8. However, the types of the neural networks 6 to 8 need not be limited to this example, and may be selected as appropriate in accordance with the embodiment. For example, convolutional neural networks including convolutional layers and pooling layers, recursive neural networks having recursive connections from the output side to the input side, such as from the intermediate layer to the input layer, or the like may be used as the neural networks 6 to 8. The internal structures of the neurons included in the neural networks 6 to 8 also need not be limited to the examples described in the embodiment. For example, spiking neurons that fire and generate pulses as a probability P on the basis of an internal potential h may be used as the neurons.

<4.6>
In the embodiment described above, each learning device is constituted by a neural network. However, the type of the learning device need not be limited to a neural network, and may be selected as appropriate in accordance with the embodiment. For example, a support vector machine, a self-organizing map, or a learning device that learns through reinforced learning may be used as each learning device.

<4.7>
In the embodiment described above, in step S104, influencing factor data 123 not classified for any type of emotion is deleted. However, the processing sequence carried out by the data collection apparatus 1 need not be limited to this example. The process of step S104 may be omitted if influencing factor data 123 not classified for any type of emotion is saved.

<4.8>
In the embodiment described above, the data collection apparatus 1, the first learning apparatus 2, the second learning apparatus 3, and the other information processing apparatuses (the elicited emotion estimation apparatus, for example) are constituted of individual computers. However, the configurations of the data collection apparatus 1, the first learning apparatus 2, the second learning apparatus 3, and the other information processing apparatuses need not be limited to this example. At least two of the data collection apparatus 1, the first learning apparatus 2, the second learning apparatus 3, and the other information processing apparatuses may be configured integrally. Note that if at least two of the data collection apparatus 1, the first learning apparatus 2, the second learning apparatus 3, and the other information processing apparatuses are configured integrally, the constituent elements included in the respective apparatuses configured integrally may be connected (a sum of sets). In other words, the information processing apparatuses configured integrally may be configured to include individual and unique constituent elements for each apparatus, such as respective programs, learning data, learning result data, and so on, and share a control unit, a storage unit, an input device, and output device, and so on.

<4.9>
In the embodiment described above, the expressed emotion learning result data 124 and the elicited emotion learning result data 223 include information indicating the configurations of the neural networks (6 and 7). However, the configuration of the learning result data (124 and 223) need not be limited to this example. For example, if the neural networks used have the same configuration from apparatus to apparatus, each piece of learning result data (124 and 223) need not include information indicating the configurations of the neural networks (6 and 7).

Claims

A data collection apparatus comprising:
a first obtainment unit configured to obtain first data expressing an emotion of a person;
a second obtainment unit configured to, in parallel with the obtainment of the first data, obtain second data pertaining to a factor that can influence the emotion of the person;
an emotion specifying unit configured to specify a type of the emotion of the person at a given time on the basis of the obtained first data; and
a classification processing unit configured to classify the second data for each type of the emotion of the person by saving the second data, obtained in an elicitation time related to the eliciting of the emotion of the person at the given time, in association with the type of the emotion of the person at the specified given time.
The data collection apparatus according to claim 1,
wherein the emotion specifying unit specifies the emotion of the person at the given time using a trained learning device trained so that when first data is inputted, the learning device outputs a value indicating the type of the emotion of the person expressed by the first data.
The data collection apparatus according to claim 1 or 2,
wherein the elicitation time is set for each of types of the second data.
The data collection apparatus according to any one of claims 1 to 3,
wherein the classification processing unit further associates attributes of the person with the second data classified for each type of the emotion of the person.
The data collection apparatus according to claim 4,
wherein the attributes of the person are specified by at least one of sex, height, weight, blood type, age, date of birth, place of birth, nationality, and profession.
The data collection apparatus according to any one of claims 1 to 5, further comprising:
a factor estimating unit configured to estimate a factor eliciting the emotion by analyzing the second data classified for each type of the emotion.
The data collection apparatus according to any one of claims 1 to 6,
wherein the classification processing unit is configured to delete the second data not classified for any type of emotion.
The data collection apparatus according to any one of claims 1 to 7,
wherein the first data includes at least one of image data showing the face of the person, data obtained by polygraph, audio data containing a recording of the voice of the person, and response data from the person.
The data collection apparatus according to any one of claims 1 to 8,
wherein the second data includes at least one of biological data obtained from the person, environment data indicating an environment in the surroundings of the person, and event data indicating an event that has occurred for the person.
The data collection apparatus according to claim 9,
wherein the second data includes at least two of the biological data, the environment data, and the event data; and
the elicitation time is different for the biological data, the environment data, and the event data.
The data collection apparatus according to claim 9 or 10,
wherein the biological data indicates at least one of heart rate, pulse rate, breathing rate, body temperature, blood pressure, brain waves, posture, and myoelectricity.
The data collection apparatus according to any one of claims 9 to 11,
wherein the environment data indicates at least one of temperature, humidity, brightness, weather, atmospheric pressure, noise, vibrations, and odors.
A learning apparatus comprising:
a data obtainment unit configured to obtain influencing factor data, the influencing factor data pertaining to a factor that can influence an emotion of a person and being classified for each of types of the emotion of the person; and
a learning processing unit configured to train a learning device so that when the influencing factor data is inputted, the learning device outputs a value indicating the type of the emotion associated with the inputted influencing factor data.
The learning apparatus according to claim 13,
wherein the data obtainment unit further obtains attribute data indicating attributes of the person; and
the learning processing unit trains the learning device so that when the influencing factor data and the attribute data are inputted, the learning device outputs a value indicating the type of the emotion associated with the inputted influencing factor data.
A learning apparatus comprising:
a data obtainment unit configured to obtain emotion expression data expressing an emotion of a person; and
a learning processing unit configured to train a learning device so that when the emotion expression data is inputted, the learning device outputs a value indicating a type of the emotion of the person expressed by the inputted emotion expression data.