JP6739805B2 - Information processing device, program - Google Patents

Description

The present invention relates to an information processing device and a program.

Generally, a person uses a sentence (for example, a diary) as a method of recording his/her feelings. A sentence represents a person's subjective feeling. In other words, the subjective feeling index related to the subjective feeling of the person can be obtained from the sentence created by the person.

On the other hand, Patent Document 1 discloses a technique for determining the emotion of a subject based on information obtained by an MRI (Magnetic Resonance Imaging) test. The information obtained by the MRI examination is a quantitative value and thus directly represents the objective emotion of the subject.

JP, 2013-176406, A

However, it is not easy to receive the MRI examination of Patent Document 1 because of the high cost and labor. Therefore, there are many people who have never undergone the MRI examination (that is, there is no data indicating objective emotions). Therefore, the emotion of a person who has never undergone the MRI examination cannot be determined with high accuracy.

As described above, in the related art, the accuracy of emotion estimation is not sufficient for a person who does not have data representing objective emotion.

An object of the present invention is to improve the accuracy of estimating the emotion of a person who does not have data representing the objective emotion.

One aspect of the present invention is
An information processing device for estimating a user's emotion,
Equipped with means for acquiring direct objective information that is directly related to the objective emotion of the subject,
A means for acquiring at least one of subject subjective information related to the subjective feeling of the subject and test indirect objective information indirectly related to the objective feeling of the subject,
A means for acquiring at least one of user subjective information related to the user's subjective feeling, and user indirect objective information indirectly related to the user's objective feeling;
A means for estimating user emotion information related to a user's emotion by referring to direct objective information, at least one of subject subjective information and subject indirect objective information, and at least one of user subjective information and user indirect objective information. Prepare,
It is an information processing device.

According to the present invention, it is possible to improve the accuracy of estimating the emotion of a person who does not have data representing objective emotion.

It is a block diagram which shows the structure of the information processing system of this embodiment. It is explanatory drawing of the outline of this embodiment. It is an explanatory view of one mode of an outline of this embodiment. It is a figure which shows the data structure of the subject information database of this embodiment. It is a figure which shows the data structure of a subject flow subjective information database of this embodiment. It is a figure which shows the data structure of a subject flow indirect objective information database of this embodiment. It is a figure which shows the data structure of the user information database of this embodiment. It is a figure which shows the data structure of the user flow subjective information database of this embodiment. It is a figure which shows the data structure of the user flow indirect objective information database of this embodiment. It is a figure which shows the whole flow of the information processing of this embodiment. FIG. 11 is a sequence diagram of a process of registering subject information in FIG. 10. It is a figure which shows the example of a screen displayed in the information processing of FIG. It is a figure which shows the example of a screen displayed in the information processing of FIG. FIG. 11 is a sequence diagram of a process of registering subject flow information in FIG. 10. It is a figure which shows the example of a screen displayed in the information processing of FIG. It is a figure which shows the example of a screen displayed in the information processing of FIG. It is a figure which shows the example of a screen displayed in the information processing of FIG. FIG. 11 is a sequence diagram of a process of user information registration process (OP3) of FIG. 10. It is a figure which shows the example of a screen displayed in the information processing of FIG. FIG. 11 is a sequence diagram of a process of registering user flow information of FIG. 10. 12 is a flowchart of the emotion visualization process of FIG. 11. FIG. 22 is a diagram showing an example of a screen displayed in the information processing of FIG. 21. It is a figure which shows the data structure of the user emotion information database of a modification. It is a flowchart of the process of visualization of emotions of a modification.

An embodiment of the present invention will be described in detail below with reference to the drawings. In addition, in the drawings for describing the embodiments, the same components are denoted by the same reference symbols in principle, and repeated description thereof will be omitted.

(1) Configuration of Information Processing System The configuration of the information processing system will be described. FIG. 1 is a block diagram showing the configuration of the information processing system of this embodiment.

As shown in FIG. 1, the information processing system 1 includes a client device 10 and a server 30.
The client device 10 and the server 30 are connected via a network (for example, the Internet or an intranet) NW.

The client device 10 is an example of an information processing device that transmits a request to the server 30. The client device 10 is, for example, a smartphone, a tablet terminal, or a personal computer.

The server 30 is an example of an information processing device that provides the client device 10 with a response according to the request transmitted from the client device 10. The server 30 is, for example, a web server.

(1-1) Configuration of Client Device The configuration of the client device 10 will be described with reference to FIG.

As shown in FIG. 1, the client device 10 includes a storage device 11, a processor 12, an input/output interface 13, and a communication interface 14.

The storage device 11 is configured to store programs and data. The storage device 11 is, for example, a combination of a ROM (Read Only Memory), a RAM (Random Access Memory), and a storage (for example, a flash memory or a hard disk).

The programs include, for example, the following programs.
-OS (Operating System) program-Application (for example, web browser) program that executes information processing

The data includes, for example, the following data.
-Database referred to in information processing-Data obtained by executing information processing (that is, execution result of information processing)

The processor 12 is configured to realize the function of the client device 10 by activating a program stored in the storage device 11. The processor 12 is an example of a computer.

The input/output interface 13 is configured to acquire a user instruction from an input device connected to the client device 10 and output information to an output device connected to the client device 10.
The input device is, for example, a keyboard, a pointing device, a touch panel, or a combination thereof.
The output device is, for example, a display.

The communication interface 14 is configured to control communication between the client device 10 and the server 30.

(1-2) Server Configuration The configuration of the server 30 will be described with reference to FIG.

As shown in FIG. 1, the server 30 includes a storage device 31, a processor 32, and a communication interface 34.

The storage device 31 is configured to store programs and data. The storage device 31 is, for example, a combination of a ROM, a RAM, and a storage (for example, a flash memory or a hard disk).

The programs include, for example, the following programs.
-OS program-Application program that executes information processing

The data includes, for example, the following data.
・Database referenced in information processing ・Results of information processing execution

The processor 32 is configured to realize the function of the server 30 by activating a program stored in the storage device 31. The processor 32 is an example of a computer.

The input/output interface 33 is configured to acquire a user instruction from an input device connected to the server 30 and output information to an output device connected to the server 30.
The input device is, for example, a keyboard, a pointing device, a touch panel, or a combination thereof.
The output device is, for example, a display.

The communication interface 34 is configured to control communication between the server 30 and the client device 10.

(2) Outline of the present embodiment An outline of the present embodiment will be described. FIG. 2 is an explanatory diagram of the outline of the present embodiment.

As shown in FIG. 2, the subject undergoes three types of tests (subjective test, indirect objective test, and direct objective test). That is, the subject is associated with three types of subject examination information (subjective information, indirect objective information, and direct objective information).
The subjective test is a test for measuring subject subjective information related to a subject's subjective feeling. The subjective test is performed by, for example, a Q&A type test. Subjective information is information that is determined by the subjectivity of the subject.
The indirect objective test is a test for measuring indirect objective information related to the subject's objective emotions. The indirect objective test is, for example, a subliminal test or a test using a wearable device. The indirect subject information does not directly reflect the reaction of the brain that controls the emotion of the subject, but indirectly reflects it. The subject indirect objective information is, for example, information indicating the reaction of the brain that controls the emotion of the subject, which is inferred from at least one of the pulse, the pupil size, and the sweat glands.
The direct objective test is a test for measuring direct objective information related to a subject's objective emotions. The direct objective examination is, for example, an examination performed using a brain image processing device (for example, an MRI device or an fMRI (functional Magnetic Resonance Imaging) device). The direct objective information directly reflects the reaction of the brain that controls the emotion of the subject and the brain function related to the emotion (for example, suppression of emotion). In other words, the direct objective information is the information that most objectively expresses the emotion of the subject in the examination information.

The user receives two types of inspections (subjective inspection and indirect objective inspection). That is, two types of user inspection information (subjective information and indirect objective information) are associated with the user.
Unlike the test subject, the user does not undergo an objective examination directly. Therefore, there is no direct objective emotion information of the user.

In the present embodiment, the correlation between the combination of the subjective information and the indirect objective information and the direct objective information is calculated from the three types of examination information (subjective information, indirect objective information, and direct objective information) associated with the subject. To do.
Further, by referring to the correlation between the combination of the subjective information and the indirect objective information associated with the subject and the direct objective information, the two types of examination information (subjective information and indirect objective information) associated with the user Emotion information (for example, the emotional condition, emotional tendency, and emotional factors that affect emotions) of the user is estimated.

As described above, according to the present embodiment, it is possible to estimate the direct objective information in which the reaction of the brain that controls the emotion of the user who has not undergone the direct objective test is directly reflected. As a result, it is possible to accurately estimate the emotion of the user who has not undergone the direct objective examination.

(2-1) One Aspect of Outline of Present Embodiment One aspect of an outline of the present embodiment will be described. FIG. 3 is an explanatory diagram of one aspect of the outline of the present embodiment.

As shown in FIG. 3, the subjective inspection is classified into two types (fixed subjective inspection and flowing subjective inspection).
The fixation subjective test is, for example, a Q&A type test. The questions in this test include a context for the subject to look back on past self-related events and experiences and to test self-awareness from patterns of those events and experiences. Therefore, the information obtained from the answers of this test is the sticky subjective information.
The flow subjective test is, for example, a Q&A type test. The questions in this test include context to remind the subject of one recent self event and experience. Therefore, the information obtained from the answer of this test is the floating subjective information. The difference between the plural pieces of flowing subjective information and the fixed subjective information is the difference in the ability of the individual to see himself and the metacognitive ability. This difference is one of the useful indexes in emotion observation ability.

The objective inspection is classified into two types (indirect objective inspection and direct objective inspection).

The indirect objective inspection is classified into a fixed indirect objective inspection and a flow indirect objective inspection.
The fixation indirect objective test is, for example, a subliminal test performed on a subject. In the subliminal test, content (image, sound, or a combination thereof) that gives a subliminal effect to a subject is presented, and the reaction of the subject to the content is measured. In general, brains that have been given a subliminal effect exhibit specific reactions that depend on the emotion of the individual. That is, the measured reaction reflects the emotions that are fixed to the brain of the subject. Therefore, the reaction of the subject is fixed indirect objective information.
The flow indirect objective test is, for example, a test using a wearable device worn by a subject. The data acquired by the wearable device varies depending on the timing of measurement. Therefore, this data is flow indirect objective information.

The direct objective inspection is classified into a fixed direct objective inspection and a flow direct objective inspection.
The fixation direct objective test is, for example, an MRI test performed on a subject. The image captured by the MRI device directly reflects the emotions that are fixed to the brain of the subject. Therefore, the image acquired by the MRI device is fixed indirect objective information.
The flow direct objective test is, for example, an EEG (Electroencephalogram) test or fMRI test performed on a subject. The result of the EEG test or fMRI test varies depending on the timing of the test. Therefore, the results of the EEG test and the fMRI test are flow direct objective information.

In one aspect of the present embodiment, the subjective information and the indirect objective are determined from five types of examination information (fixed subjective information, flow subjective information, fixed indirect objective information, flow indirect objective information, and direct objective information) associated with the subject. Calculate the correlation between information combinations and direct objective information.
Further, by referring to the correlation between the combination of the subjective information and the indirect objective information associated with the subject and the direct objective information, four types of inspection information associated with the user (fixed subjective information, flow subjective information, fixed indirect information) The emotional information of the user (for example, the emotional condition of the user, the emotional tendency, and the emotional factors that affect the emotion) is estimated from the objective information and the indirect objective information of the flow.

(3) Database The database of this embodiment will be described. The following databases are stored in the storage device 31.

(3-1) Subject Information Database The subject information database of this embodiment will be described. FIG. 4 is a diagram showing a data structure of the subject information database of this embodiment.

The subject information regarding the subject is stored in the subject information database of FIG.
The subject information database includes a “subject ID” field, a “subject name” field, a “subject attribute” field, a “fixed subjectivity” field, a “direct objective” field, and a “fixed indirect objective” field. ..
Each field is associated with each other.

A subject ID (an example of “subject identification information”) for identifying a subject is stored in the “subject ID” field.

The “subject name” field stores information (eg, text) related to the subject name. The subject name, for example, is arbitrarily determined by the subject when registering the subject information.

The “subject attribute” field stores subject attribute information related to the subject attribute. The subject attribute information is arbitrarily determined by the subject, for example, when registering the subject information.
The “subject attribute” field includes a “sex” field, an “age” field, and a “profession” field.

The “sex” field stores information about the sex of the subject.

Information regarding the age of the subject is stored in the "age" field.

The “profession” field stores information about the occupation of the subject.

The "fixation subjectivity" field stores the subject's fixation subjectivity information. The sticky subjective information is information determined by a combination of answers to a plurality of questions prepared in advance. The sticky subjective information is, for example, information on the type of personality of the subject (hereinafter referred to as “personal type”).

The "direct objective" field stores the direct objective information of the subject.
The "direct objective" field includes an "MRI" field and an "fMRI" field.

The "MRI" field stores an MRI image (an example of "direct objective information") relating to the MRI examination result. The MRI image is, for example, image data of a subject's brain.

In the “fMRI” field, fMRI information (an example of “direct objective information”) regarding the fMRI examination result is stored. The fMRI information is a hemodynamic response parameter (eg, amygdala activity, islet activity, dorsolateral prefrontal cortex inactivity, blood cortisol level) related to brain activity in each part of the brain for each of multiple stimuli. , Blood pressure fluctuation rate, and facial muscles).

In the “fixed indirect objective” field, fixed indirect objective information obtained from a fixed indirect objective test (for example, a subliminal test) on the subject is stored. The fixed indirect objective information includes, for example, the following.
・Details of reaction when presenting subliminal content ・Reaction speed when presenting subliminal content

(3-2) Subject Flow Subjectivity Information Database The subject subject flow information database of the present embodiment will be described. FIG. 5 is a diagram showing a data structure of the subject flow subjective information database of this embodiment.

The subject flow subjective information database of FIG. 5 stores the answers to the questions presented to the subject.
The subject flow subjective information database includes a "data ID" field, a "check-in date and time" field, a "check-in response" field, a "check-out date and time" field, and a "check-out response" field.
Each field is associated with each other.
The subject flow subjective information database is associated with the subject ID.

In the "data ID" field, a data ID for identifying a record in the subject flow subjective information database is stored.

The “check-in date and time” field stores information regarding the check-in date and time. Check-in is a process for acquiring an answer to a question presented to a subject before executing a certain task.

The “check-in answer” field stores check-in answer information regarding the answer acquired in the check-in. The information in the “check-in answer” field is stored separately for each question.

Information regarding the date and time of check-out is stored in the “check-out date and time” field. Checkout is a process for acquiring an answer to a question presented to a subject after completing a certain task.

The "checkout answer" field stores checkout answer information about the answer obtained in the checkout. The information in the “checkout answer” field is stored separately for each question.

(3-3) Subject Flow Indirect Objective Information Database The subject flow indirect objective information database of this embodiment will be described. FIG. 6 is a diagram showing a data structure of the subject flow indirect objective information database of this embodiment.

The subject flow indirect objective information of the subject is stored in the subject flow indirect objective information database of FIG.
The subject flow indirect objective information database includes a "data ID" field, a "measurement date and time" field, and a "wearable" field.
The subject flow indirect objective information database is associated with the subject ID.

In the "data ID" field, a data ID for identifying a record in the subject indirect objective information database is stored.

The “measurement date/time” field stores information about the date/time when the wearable device measured the data.

The "wearable" field stores data measured by the wearable device. The data measured by the wearable device includes at least one of the following, for example.
・Steps, pulse, heart rate, calories burned

(3-4) User Information Database The user information database of this embodiment will be described. FIG. 7 is a diagram showing a data structure of the user information database of this embodiment.

The user information database of FIG. 7 stores user information about users.
The user information database includes a “user ID” field, a “user name” field, a “user attribute” field, a “fixed subjectivity” field, and a “fixed indirect objective” field.
Each field is associated with each other.

A user ID (an example of “user identification information”) for identifying a user is stored in the “user ID” field.

Information (for example, text) related to the user name is stored in the “user name” field. The user name is arbitrarily determined by the user when registering the user information, for example.

The “user attribute” field stores user attribute information regarding the user attribute. The user attribute information is arbitrarily determined by the user when registering the user information, for example.
The "user attribute" field includes a "sex" field, an "age" field, and an "occupation" field.

Information regarding the gender of the user is stored in the “sex” field.

Information regarding the age of the user is stored in the "age" field.

Information regarding the occupation of the user is stored in the “occupation” field.

The "fixed subjectivity" field stores the fixed subjectivity information of the user. The sticky subjective information is information determined by a combination of answers to a plurality of questions prepared in advance. The sticky subjective information is, for example, information on the personality type of the user.

The "fixed indirect objective" field stores fixed indirect objective information obtained from a fixed indirect objective test (for example, a subliminal test) for the user. An example of the fixed indirect objective information is similar to the example of the “fixed indirect objective” field of the subject information database (FIG. 4).

(3-5) User Flow Subjective Information Database The user flow subjective information database of this embodiment will be described. FIG. 8 is a diagram showing a data structure of the user flow subjective information database of this embodiment.

The user flow subjective information database in FIG. 8 stores answers to questions presented to the user.
The user flow subjective information database includes a "data ID" field, a "check-in date and time" field, a "check-in response" field, a "check-out date and time" field, and a "check-out response" field.
Each field is associated with each other.
The user flow subjective information database is associated with the user ID.

In the "data ID" field, a data ID for identifying a record in the user flow subjective information database is stored.

The “check-in date and time” field stores information regarding the check-in date and time. Check-in is a process for acquiring an answer to a question presented to the user before executing a certain task.

The “check-in answer” field stores check-in answer information regarding the answer acquired in the check-in. The information in the “check-in answer” field is stored separately for each question.

Information regarding the date and time of check-out is stored in the “check-out date and time” field. Checkout is a process for obtaining an answer to a question presented to the user after completing a certain task.

The "checkout answer" field stores checkout answer information about the answer obtained in the checkout. The information in the “checkout answer” field is stored separately for each question.

(3-6) User Flow Indirect Objective Information Database The user flow indirect objective information database of this embodiment will be described. FIG. 9 is a diagram showing the data structure of the user flow indirect objective information database of this embodiment.

The user indirect objective information of the user's flow is stored in the user indirect objective information database of FIG.
The user flow indirect objective information database includes a "data ID" field, a "measurement date and time" field, and a "wearable" field.
The user flow indirect objective information database is associated with the user ID.

In the "data ID" field, a data ID for identifying a record of the user flow indirect objective information database is stored.

The “measurement date/time” field stores information about the date/time when the wearable device measured the data.

The "wearable" field stores data measured by the wearable device. An example of the data measured by the wearable device is the same as the example of the “wearable” field of the subject flow indirect objective information database (FIG. 6).

(4) Information processing Information processing of this embodiment will be described. FIG. 10 is a diagram showing an overall flow of information processing of this embodiment.

As shown in FIG. 10, in the information processing of the present embodiment, subject information registration (OP1), subject flow information registration (OP2), user information registration (OP3), and user flow information registration (OP4). ) And emotion visualization (OP5).
The registration of the subject information (OP1) and the registration of the subject flow information (OP2) are performed on the subject information database (FIG. 4), the subject flow subjective information database (FIG. 5), and the subject flow indirect objective information database (FIG. 6). This is a process for registering data. By registering the subject information (OP1) and registering the subject flow information (OP2), the subject test information and the emotion inference model are constructed in the server 30.
The registration of user information (OP3) and the registration of user flow information (OP4) are performed on the user information database (FIG. 7), the user flow subjective information database (FIG. 8), and the user flow indirect objective information database (FIG. 9). This is a process for registering data. By registering user information (OP3) and registering user flow information (OP4), the input information necessary for visualization of emotions (OP5) is given to the server 30.
The emotion visualization (OP5) is given in the inference model generated by the registration of the subject information (OP1) and the registration of the subject flow information (OP2), and the registration of the user information (OP3) and the registration of the user flow information (OP4). This is a process for visualizing the emotion of the user by using the input information thus obtained.

(4-1) Registration of Subject Information The process of registering subject information (OP1) according to this embodiment will be described. FIG. 11 is a sequence diagram of a process of registering subject information in FIG. 12 to 13 are diagrams showing examples of screens displayed in the information processing of FIG. 11.

As shown in FIG. 11, reception of subject information (S100) is executed.
Specifically, the processor 12 of the client device 10 displays the screen P100 (FIG. 12) on the display.

The screen P100 includes an operation object B100 and a field object F100.
The field object F100 is an object that receives input of subject information (for example, subject name, sex, age, and occupation).
The operation object B100 is an object that receives an instruction for transmitting the information input to the field object F100 to the server 30.

When the subject inputs the subject information to the field object F100 and operates the operation object B100, the processor 12 transmits the subject information registration request data to the server 30. The subject information registration request data includes the information input to the field object F100.

The processor 32 of the server 30 adds a new record to the subject information database (FIG. 4).
The following information is stored in each field of the new record.
A new subject ID is stored in the “subject ID” field.
The information included in the subject information registration request data (that is, the information input to the field object F100) is stored in the “subject name” field and the “subject attribute” field.

After step S100, the first Q&A test (S101) is executed.
Specifically, the processor 12 of the client device 10 displays the screen P101 (FIG. 12) on the display.

The screen P101 includes an operation object B101 and a field object F101.
In the field object F101, a question and options for answering the question are displayed. When the operator specifies at least one of the options, the field object F101 receives the option specified by the operator as the operator's answer.
The operation object B101 is an object that receives an instruction for fixing the answer received by the field object F101.

When the operation object B101 is operated, the processor 12 transmits the update request data to the server 30. The update request data includes the information input to the field object F101.

The storage device 31 of the server 30 stores a matching table indicating the relationship between the combination of information input to the field object F101 and the personality type of the input person.
The processor 32 refers to the matching table stored in the storage device 31 to determine the personality type corresponding to the information included in the update request data.
The processor 32 stores information on the determined personality type in the “fixed subjectivity” field of the new record added in step S100.
The processor 32 transmits the update response data to the client device 10. The update response data includes information about the personality type.

The processor 12 of the client device 10 displays the screen P102 (FIG. 12) on the display based on the update request data.
The personality type determined by the server 30 is displayed on the screen P102.

After step S101, a subliminal test (S102) is executed.
Specifically, the processor 12 of the client device 10 displays the screen P103 (FIG. 13) on the display.

The screen P103 includes a display object A103.
On the display object A 103, a message prompting the subject to perform an operation (for example, a text message “Please click the displayed triangle”) is displayed.

After a lapse of time t1 (for example, 4 seconds) from the display of the screen P103, the processor 12 displays the screen P104 (FIG. 13) on the display.

The screen P104 includes display objects A103 and A104.
The display object A104 includes a display area A104a and a display area A104b.
Image objects IMG104a to IMG104b are displayed in the display areas A104a to A104b, respectively. The image objects IMG104a to IMG104b are displayed side by side in a predetermined direction. The image objects IMG104a to IMG104b are, for example, rectangular images.

After a lapse of time t2 (<t1) from the display of the screen P104, the processor 12 displays the screen P105a or P105b (FIG. 13) on the display. The time t2 is, for example, 0.01 seconds. That is, the screen P104 is displayed for an extremely short time compared to the screen P103.

As described above, the display time t2 of the screen P104 is significantly shorter than the display time t1 of the screen P103. Therefore, the image objects IMG104a to IMG104b cannot be recognized by human vision, but normally, the brain recognizes them in a portion below the "consciousness" (so-called subconscious, or a boundary area between the consciousness and the subconscious). It is possible. Therefore, the subject unknowingly focuses on either the image object IMG104a or IMG104b. This phenomenon is called "subliminal effect". In step S102, the indirect objective fixation information of the subject is acquired by using this subliminal effect.

The screen P105a includes a display object A104.
The image object IMG105a is displayed in the display area A104a of the display object A104. The image object IMG105a is, for example, a triangular image. The image object IMG105a is an object that can be operated by the operator.

The screen P105b includes the display object A104.
The image object IMG105b is displayed in the display area A104b of the display object A104. The image object IMG105b is, for example, a triangular image. The image object IMG105b is an object that can be operated by the operator.

When the subject operates the image object IMG105a or IMG105b, the processor 12 stores the time (hereinafter referred to as “reaction time”) from the display of the screen P105a or P105b until the subject operates the image object IMG105a or IMG105b in the storage device 11. To do.
For example, when the subject is paying attention to the image object IMG104a on the screen P104, the reaction time for the operation of the image object IMG105a displayed in the same display area A104a as the image object IMG104a is displayed in the display area A104b different from the image object IMG104a. It becomes shorter than the reaction time for the operation of the displayed image object IMG105b.

The processor 12 repeatedly executes the above-described subliminal test a plurality of times.
When the predetermined number of subliminal tests are completed, the processor 12 transmits the update request data to the server 30. The update request data includes the following information.
Information about reaction time for image object IMG105a Information about reaction time for image object IMG105b

The processor 32 stores the information regarding the reaction time included in the update request data in the “fixed indirect objective” field of the new record added in step S100.
The processor 32 transmits the update response data to the client device 10.

After step S102, the MRI examination (S103) is executed.
Specifically, a computer connected to the magnetic resonance apparatus generates a brain activity parameter of the subject and a hemodynamic response parameter based on the signal obtained by the magnetic resonance apparatus.
The computer sends the update request data to the server 30.
The update request data includes the following information.
・Brain activity parameters ・Hemodynamic response parameters

The processor 32 of the server 30 stores the following information in the new record added in step S100.
The brain activity parameter included in the update request data is stored in the “MRI” field.
The hemodynamic response parameter included in the update request data is stored in the “fMRI” field.

(4-2) Registration of Subject Flow Information The process of registering subject flow information (OP2) according to the present embodiment will be described. FIG. 14 is a sequence diagram of a process of registering the subject flow information of FIG. 15 to 17 are diagrams showing examples of screens displayed in the information processing of FIG.

As shown in FIG. 14, acquisition of wearable information (S200) is executed.
The processor 12 of the client device 10 acquires wearable information (for example, at least one of the number of steps, a pulse, and a heart rate) from the wearable device worn by the subject.
The processor 12 sends the update request data to the server 30. The update request data includes the following information.
・Subject ID of the subject wearing the wearable device
・Wearable information ・Information about the execution date and time of step S200

The processor 32 of the server 30 adds a new record to the subject flow indirect objective information database (FIG. 6) associated with the subject ID included in the update request data.
The following information is stored in each field of the new record.
A new data ID is stored in the "data ID" field.
The "measurement date and time" field stores information about the execution date and time included in the update request data.
The wearable information included in the update request data is stored in the "wearable" field.

After step S200, check-in (S201) is executed.
Specifically, the processor 12 of the client device 10 displays the screen P200 (FIG. 15) on the display.

The screen P200 includes an operation object B200 and field objects F200a to F200e.

Questions regarding the task to be performed after step S201 are presented to the field objects F200a to F200e.
The field object F200a is presented with the question “What is the task content?”. When arbitrary text is input to the field object F200a, the processor 12 acquires the input text as information regarding the content of the task.
The field object F200b is presented with the question “What kind of task is it?”. When an arbitrary option is selected from the plurality of options of the field object F200b, the processor 12 acquires the selected option as information regarding the type of task.
The field object F200c is presented with the question “Who with?”. When an arbitrary option is selected from the plurality of options of the field object F200c, the processor 12 acquires the selected option as information regarding the person who will perform the task together.
The field object F200d is presented with the question "how often does the task occur?". When the slider of the field object F200d is moved, the processor 12 acquires a numerical value corresponding to the position of the slider as information regarding the frequency of tasks.
The field object F200e is presented with the question “How difficult is the task?”. When the slider of the field object F200e is moved, the processor 12 acquires a numerical value corresponding to the position of the slider as information regarding the difficulty level of the task.

When the operation object B200 is operated, a screen P201 (FIG. 15) is displayed.

The screen P201 includes an operation object B201 and field objects F201a and F201b.

Questions regarding the action performed before step S201 are presented to the field objects F201a to F201b.
The field object F201a is presented with the question "What were you doing in the last 30 minutes?". When an arbitrary option is selected from the plurality of options of the field object F201a, the processor 12 acquires the selected option as information regarding the action performed in the 30 minutes immediately before check-in.
The field object F201b is presented with the question "How was the last 30 minutes?" When the slider of the field object F201b is moved, the processor 12 acquires a numerical value according to the position of the slider as information regarding self-evaluation of the action performed during the 30 minutes immediately before check-in.

When the operation object B201 is operated, a screen P202 (FIG. 16) is displayed.

The screen P202 includes an operation object B202 and field objects F202a to F202g.

Questions related to the subjective feeling before performing the task are presented to the field objects F202a to F202g.
In the field object F202a, the question "How hungry?" is displayed. When the slider of the field object F202a is moved, the processor 12 acquires a numerical value according to the position of the slider as information regarding hunger.
In the field object F202b, the question "How sleepy?" is displayed. When the slider of the field object F202b is moved, the processor 12 acquires a numerical value according to the position of the slider as information regarding drowsiness.
In the field object F202c, the question “What is your stress?” is displayed. When the slider of the field object F202c is moved, the processor 12 acquires a numerical value according to the position of the slider as information regarding stress.
In the field object F202d, the question "What is the last task?" is displayed. When the slider of the field object F202d is moved, the processor 12 acquires a numerical value corresponding to the position of the slider as information regarding self-evaluation for the task performed before check-in.
In the field object F202e, the question "What is your motivation?" is displayed. When the slider of the field object F202e is moved, the processor 12 acquires a numerical value according to the position of the slider as information regarding motivation for the task.
In the field object F202f, the question “Are you feeling uneasy?” is displayed. When the slider of the field object F202f is moved, the processor 12 acquires a numerical value corresponding to the position of the slider as information regarding the anxiety about the task.
In the field object F202g, the question "What is the clarity?" is displayed. When the slider of the field object F202g is moved, the processor 12 acquires a numerical value corresponding to the position of the slider as information regarding the clarity of the task.

When the operation object B202 is operated, a screen P203 (FIG. 16) is displayed.

The screen P203 includes an operation object B203 and field objects F203a and F203b.

Questions related to the planned activity after performing the task are presented to the field objects F203a to F203b.
The field object F203a is presented with the question "What is the action after the task?". When an arbitrary option is selected from the plurality of options of the field object F203a, the selected option is acquired as information about the scheduled action after performing the task.
The field object F203b is presented with the question "How fun is the action after the task?". When the slider of the field object F203b is moved, the processor 12 acquires a numerical value corresponding to the position of the slider as information regarding the feeling of expectation for the action after the task.

The operation object B203 is an object that receives an instruction to complete the process of step S201.

When the operation object B203 is operated, the processor 12 of the client device 10 transmits check-in completion request data to the server 30.
The check-in completion request data includes the following information.
・Subject ID of the subject who answered
-Check-in time information indicating the execution date and time of step S201-Information input to field objects F200a to F200e, F201a to F201b, F202a to F202g, and F203a to F203b

The processor 32 of the server 30 adds a new record to the subject flow subjective information database (FIG. 5) associated with the subject ID included in the check-in request data.

The following information is stored in each field of the new record.
A new data ID is stored in the "data ID" field.
The check-in time information included in the check-in completion request data is stored in the "check-in date and time" field.
The answer information included in the check-in completion request data is stored for each question in the "check-in answer" field.

After step S201, the process of checkout (S202) is executed.
Specifically, the processor 12 of the client device 10 displays the screen P204 (FIG. 17) on the display.

The screen P204 includes an operation object B204 and field objects F204a to F204f.

Questions relating to completed tasks are presented in the field objects F204a to F204f.
In the field object F204a, the question "What did you do?" is displayed. When arbitrary text is input to the field object F204a, the processor 12 acquires the input text as information regarding the content of the completed task.
In the field object F204b, the question "What is the degree of completion?" is displayed. When the slider of the field object F204b is moved, the processor 12 acquires a numerical value according to the position of the slider as information regarding the completion degree of the task.
In the field object F204c, the question "What have you left over?" is displayed. When any text is input to the field object F204c, the processor 12 acquires the input text as information regarding the content of the task that has been left over.
In the field object F204d, the question "What is the nature of the task?" is displayed. When the slider of the field object F204d is moved, the processor 12 acquires a numerical value according to the position of the slider as information regarding the nature of the task (for example, workability/directivity ratio).
In the field object F204e, the question "What is the input amount?" is displayed. When the slider of the field object F204e is moved, the processor 12 acquires a numerical value corresponding to the position of the slider as information regarding the input amount of the task.
In the field object F204f, the question “What is the output amount?” is displayed. When the slider of the field object F204f is moved, the processor 12 acquires a numerical value according to the position of the slider as information regarding the output amount of the task.

When the operation object B204 is operated, a screen P205 is displayed.

The screen P205 includes an operation object B205 and field objects F205a to F205f.

In the field object F205a, the question "How are you feeling now?" is displayed. When an arbitrary option is selected from the plurality of options of the field object F205a, the processor 12 acquires the selected option as information regarding the mood after completing the task.
In the field object F205b, the question "What's the feeling of mood?" is displayed. When the slider of the field object F205b is moved, the processor 12 obtains a numerical value corresponding to the position of the slider as information regarding the feeling of misery for the task.
In the field object F205c, the question “What is your sense of accomplishment?” is displayed. When the slider of the field object F205c is moved, the processor 12 acquires a numerical value according to the position of the slider as the information regarding the feeling of achievement for the task.
In the field object F205d, the question "What is the degree of concentration?" is displayed. When the slider of the field object F205d is moved, the processor 12 acquires a numerical value according to the position of the slider as information regarding the degree of concentration on the task.
In the field object F205e, the question "How satisfied are you?" is displayed. When the slider of the field object F205e is moved, the processor 12 acquires a numerical value corresponding to the position of the slider as information regarding the degree of satisfaction with the task.
In the field object F205f, the question “How is your fatigue?” is displayed. When the slider of the field object F205f is moved, the processor 12 acquires a numerical value corresponding to the position of the slider as information regarding the feeling of fatigue for the task.

The operation object B205 is an object that receives an instruction to complete the process of step S202.

When the operation object B205 is operated, the processor 12 of the client device 10 transmits checkout completion request data to the server 30.
The check-out completion request data includes the following information.
・Subject ID of the subject who answered
-Check-in time information indicating the execution date and time of step S202-Information input to the field objects F204a to F204f and F205a to F205f

The processor 32 of the server 30 identifies the subject flow subjective information database (FIG. 5) associated with the subject ID included in the checkout request data.
The processor 32 identifies the latest record (that is, the new record added in step S201) in which the information in the “check-in date and time” field indicates the latest date and time in the identified subject flow subjective information database.

The following information is stored in each field of the latest record.
The checkout time information included in the checkout completion request data is stored in the “checkout date and time” field.
The answer information included in the checkout completion request data is stored in the “checkout answer” field for each question.

(4-3) User Information Registration Processing The user information registration processing of this embodiment will be described. FIG. 18 is a sequence diagram of the processing of the user information registration processing (OP3) of FIG. FIG. 19 is a diagram showing an example of a screen displayed in the information processing of FIG.

As shown in FIG. 18, registration of user information (S300) is executed.
Specifically, the processor 12 of the client device 10 displays the screen P300 (FIG. 19) on the display.

The screen P300 includes an operation object B300 and a field object F300.
The field object F300 is an object that receives input of user information (for example, user name, sex, age, and occupation).
The operation object B300 is an object that receives an instruction for transmitting the information input to the field object F300 to the server 30.

When the user inputs the user information to the field object F300 and operates the operation object B300, the processor 12 sends the user information registration request data to the server 30. The user information registration request data includes the information input to the field object F300.

The processor 32 of the server 30 adds a new record to the user information database (FIG. 7).
The following information is stored in each field of the new record.
A new user ID is stored in the "user ID" field.
The information included in the user information registration request data (that is, the information input to the field object F300) is stored in the “user name” field and the “user attribute” field.

After step S300, the first Q&A test (S301) is executed.
Step S301 differs from step S101 of FIG. 11 in the following points.
The processor 32 of the server 30 stores the information included in the update request data (that is, the information regarding the personality type of the user) in the “fixed subjectivity” field of the new record added in step S300.

After step S301, a subliminal test (S302) is executed.
Step S302 differs from step S102 of FIG. 11 in the following points.
The processor 32 of the server 30 stores the information included in the update request data (that is, the information regarding the reaction time for the user's image objects IMG105a to IMG105b) in the “fixed indirect objective” field of the new record added in step S300. ..

(4-4) Registration of User Flow Information The process of registering user flow information (OP4) according to this embodiment will be described. FIG. 20 is a sequence diagram of a process of registering the user movement information of FIG.

As shown in FIG. 20, wearable information acquisition (S400) is executed.
The processor 12 of the client device 10 acquires wearable information (for example, at least one of the number of steps, a pulse, and a heart rate) from the wearable device worn by the user.
The processor 12 sends the update request data to the server 30. The update request data includes the following information.
-User ID of the user wearing the wearable device
・Wearable information ・Information about the execution date and time of step S400

The processor 32 of the server 30 adds a new record to the user flow indirect objective information database (FIG. 9) associated with the user ID included in the update request data.
The following information is stored in each field of the new record.
A new data ID is stored in the "data ID" field.
The "measurement date and time" field stores information about the execution date and time included in the update request data.
The wearable information included in the update request data is stored in the "wearable" field.

After step S400, check-in (S401) is executed.
Step S401 differs from step S201 of FIG. 14 in that the check-in completion request data includes the following information.
-User ID of the user who responded
-Check-in time information indicating the execution date and time of step S401-Information input to field objects F200a to F200e, F201a to F201b, F202a to F202g, and F203a to F203b

After step S401, checkout (S402) is executed.
Step S402 differs from step S202 of FIG. 14 in that the checkout completion request data includes the following information.
-User ID of the user who responded
-Check-in time information indicating the execution date and time of step S402-Information input to the field objects F204a to F204f and F205a to F205f

(4-5) Emotion Visualization Processing The emotion visualization processing (OP5) according to this embodiment will be described. FIG. 21 is a flowchart of the emotion visualization process of FIG. FIG. 22 is a diagram showing an example of a screen displayed in the information processing of FIG.

As shown in FIG. 21, determination of target user information (S500) is executed.
Specifically, the processor 12 of the client device 10 acquires the user ID (hereinafter, referred to as “target user ID”) of the target user who is the target of the emotion to be visualized.
The processor 12 transmits the reference request data to the server 30. The reference request data includes the target user ID.

As a first example of step S500, the processor 32 of the server 30 refers to the user information database (FIG. 7 ), and the information in the “user attribute” field of the record associated with the target user ID included in the reference request data, The information in the “fixed subjectivity” field and the information in the “fixed indirect objective” field are specified.

As a second example of step S500, the processor 32 of the server 30 identifies the information in each field of the user flow subjective information database (FIG. 8) associated with the target user ID included in the reference request data.

As a third example of step S500, the processor 32 of the server 30 identifies the information in each field of the user flow indirect objective information database (FIG. 9) associated with the target user ID included in the reference request data.

As a fourth example of step S500, the processor 32 of the server 30 combines a part or all of the first to third examples.

After step S500, the determination of the reference subject ID (S501) is executed.
In step S501, at least one reference subject ID is identified by the following method.

As a first example of step S501, the processor 32 of the server 30 refers to the subject information database (FIG. 4), and the subject associated with the same or approximate information as the information in the “user attribute” field specified in step S500. An ID (that is, a subject who is the same as or similar to the target user in terms of user attributes) is specified. For example, the processor 32 identifies the following subject ID.
-Subject ID associated with the same gender as the target user ID
-Subject ID associated with an age whose difference from the age associated with the target user ID is within a predetermined range
-Subject ID associated with the same occupation as the target user ID

As a second example of step S501, the processor 32 of the server 30 refers to the subject information database (FIG. 4) and associates it with the subject fixation information that is the same as or similar to the information in the “fixation subjectivity” field identified in step S500. The subject ID (that is, the subject who is the same as or similar to the target user with respect to the fixed subjective information) is specified. For example, the processor 32 identifies the following subject ID.
-Subject ID associated with the same subject fixation subjective information as the information in the "fixation subjectivity" field associated with the target user ID
-Subject ID associated with subject fixation subjective information whose difference from the information in the "fixation subjectivity" field associated with the target user ID is within a predetermined range

As a third example of step S501, the processor 32 of the server 30 refers to the subject information database (FIG. 4) and is the same as or similar to the information in the “fixed indirect objective” field identified in step S500, and the subject fixed indirect objective information. The subject ID (that is, the subject who is the same as or close to the target user for the fixed indirect objective information) is identified. For example, the processor 32 identifies the following subject ID.
-Subject ID associated with the same subject fixed indirect objective information as the information in the "fixed indirect objective" field associated with the target user ID
-Subject ID associated with the subject fixed indirect objective information whose difference from the information in the "fixed indirect objective" field associated with the target user ID is within a predetermined range

As a fourth example of step S501, the processor 32 of the server 30 refers to the subject flow subjective information database (FIG. 5) and is the same as or approximate to the information of the user flow subjective information database (FIG. 8) specified in step S500. The subject ID associated with the subject flow subjective information (that is, the subject that is the same as or similar to the target user for the subject flow subjective information) is specified. For example, the processor 32 identifies the following subject ID.
-In the information of the user flow subjective information database associated with the target user ID, the information of the "check-in date and time" field is associated with the subject flow subjective information which is the same as or similar to the information of the "check-in answer" field of the latest record. Subject ID
-Among the information in the user flow subjective information database associated with the target user ID, the information in the "checkout date/time" field is associated with the subject flowable subjective information that is the same as or similar to the information in the "checkout response" field of the latest record. Subject ID
-Of the information in the user flow subjective information database associated with the target user ID, the information in the "check-in date and time" field is in the "check-in response" field of a plurality of records corresponding to a predetermined period (for example, the past month) Subject ID associated with subject flow subjective information having the same or similar tendency as the information tendency
-Of the information in the user flow subjective information database associated with the target user ID, the information in the "checkout date and time" field is in the "checkout response" field of a plurality of records corresponding to a predetermined period (for example, the past month) Subject ID associated with subject flow subjective information having the same or similar tendency as the information tendency

As a fifth example of step S501, the processor 32 of the server 30 refers to the subject flow indirect objective information database (FIG. 6) and is the same as the information of the user flow indirect objective information database (FIG. 9) specified in step S500 or A subject ID associated with the approximate subject flow indirect objective information (that is, a subject who is the same as or close to the target user for the flow indirect objective information) is specified. For example, the processor 32 identifies the following subject ID.
The subject ID associated with the subject flow indirect objective information in which the information in the “measurement date/time” field associated with the target user ID is the same as or similar to the information in the “wearable” field of the latest record
-Subject flow having the same or approximate tendency as the information tendency of the "wearable" field of a plurality of records in which the information of the "measurement date and time" field associated with the target user ID corresponds to a predetermined period (for example, the past month) Subject ID associated with indirect objective information

A sixth example of step S501 is a combination of at least two of the first to fifth examples. The larger the number of combinations, the more the subject ID of the subject having a high similarity to the target user is specified as the reference subject ID. This improves the estimation accuracy in the estimation of user emotion information (S502) described below.

After step S501, estimation of user emotion information (S502) is executed.

Specifically, the processor 32 of the server 30 refers to the subject information database (FIG. 4) and stores the information (that is, the brain) of the “direct objective” field associated with at least one reference subject ID identified in step S501. Activity parameters and hemodynamic response parameters).
The processor 32 identifies the information in the “check-in answer” field and the “check-out answer” field of the user flow subjective information (FIG. 8) associated with the target user ID (that is, the target user's flow subjective information). ..
The processor 43 identifies the information in the “wearable” field of the user indirect objective information (FIG. 9) associated with the target user ID (that is, the target user's indirect objective information).
The memory device 31 is constructed with an inference model of brain activity parameters, hemodynamic response parameters, subjective flow information, indirect objective flow information, and emotional conditions. The emotional condition is a level for each type of emotion in neuroscience, which is determined based on various factors including anatomical position of brain, expression of transmitter, receptor, and time.
The processor 32 uses the inference model stored in the storage device 31 to determine the emotional condition of the target user from the brain activity parameter, hemodynamic response parameter, subjective flow information, and indirect objective flow information of the target user. ..
The emotion type is, for example, at least one of the following.
・Fun, anxiety, tension

The processor 32 of the server 30 refers to the user information database (FIG. 7 ), and the information of the “fixed subjective” field associated with the target user ID (that is, the fixed subjective information of the target user) and the “fixed indirect objective” field. Information (that is, the fixed indirect objective information of the target user) is specified.
The processor 32 identifies the information in the “wearable” field of the user indirect objective information database (FIG. 9) associated with the target user ID (that is, the target user's indirect objective information).
In the storage device 31, the inference model regarding the positive correlation between the fixed subjective information, the fixed indirect objective information, and the indirect objective flowing information and the emotional tendency is built.
The processor 32 uses the inference model stored in the storage device 31 to determine the emotional tendency of the target user from the fixation subjective information of the target user, the fixation indirect objective information, and the flowing indirect objective information.
The emotional tendency includes at least one of the following, for example.
・Predominance of sympathetic nerves against parasympathetic nerves ・Predominance of parasympathetic nerves against sympathetic nerves ・Stress responsiveness ・Stress tolerance ・Activity of amygdala ・Utilization of salient networks

The processor 32 of the server 30 uses the subject information database (FIG. 4), the subject flow subjective information database (FIG. 5), the subject flow indirect objective information database (FIG. 6), and the user flow subjective information associated with the target user ID. (FIG. 8) and the user flow indirect objective information database (FIG. 9) are compared with the emotional condition and the emotional tendency to determine the emotional factor.
The emotion factor includes at least one of the following, for example.
-User behavior (exercise, work, study, or reading as an example)
・User external factors (for example, weather or temperature/humidity)
User's interpersonal factors (as one example, self-otherness, familiarity, relationship, level of likes and dislikes, or contact frequency)
・User objective factors (as an example, familiarity, relationships, likes and dislikes, contact frequency, etc.)
-Environmental factors of the user (location, weather, humidity, carbon dioxide concentration, atmospheric pressure, or rainfall, as an example)
-Condition factors of the user (as an example, sleeping state or eating state)

After step S502, presentation of user emotion information (S503) is executed.
Specifically, the processor 32 of the server 30 transmits the user emotion information estimated in step S502 to the client device 10.
The processor 12 of the client device 10 displays the screen P500 (FIG. 22) on the display based on the user emotion information.

The screen P500 includes display objects A500a to A500c.
The emotional condition is displayed on the display object A 500a. The emotional condition is, for example, a level for each emotional type (for example, fun, anxiety, and tension).
The emotional tendency is displayed on the display object A 500b.
The emotion factor is displayed on the display object A 500c.

According to the present embodiment, an inference model constructed by direct objective information, subject subjective information, and subject indirect objective information associated with a subject who has undergone MRI examination is used to associate with a user who has not undergone MRI examination. User emotion information is estimated from the obtained user subjective information and user indirect objective information. As a result, it is possible to improve the accuracy of estimating the emotion of the user who has not undergone the MRI examination.

(5) Modified Example A modified example of the present embodiment will be described.

(5-1) User Emotion Information Database A modified user emotion information database will be described. FIG. 23 is a diagram showing a data structure of the user emotion information database of the modified example.

The user emotion information database of FIG. 23 stores the user emotion information estimated in step S502 (FIG. 21).
The user emotion information database includes a “data ID” field, a “determination date and time” field, and an “emotion” field.
The user emotion information database is associated with the user ID.

In the “data ID” field, a data ID for identifying a record in the user emotion information database is stored.

Information regarding the execution date and time of step S502 is stored in the "determination date and time" field.

The processing result of step S502 is stored in the "emotion" field.
The "emotion" field includes an "emotion condition" field, an "emotion tendency" field, and an "emotion factor" field.

Information relating to emotional conditions is stored in the “emotional condition” field.

Information regarding emotional tendency is stored in the “emotional tendency” field.

Information regarding emotional factors is stored in the “emotional factor” field.

(5-2) Emotion Visualization Process A modified emotion visualization process (OP5) will be described. FIG. 24 is a flowchart of the emotion visualization process of the modified example.

As shown in FIG. 24, the emotion visualization process (OP5) of the modified example is different from the emotion visualization process (FIG. 21) of the present embodiment in that it includes determination of the reference user ID (S510).

After step S501, the reference user ID is determined (S510).
Specifically, the processor 32 of the server 30 determines the reference user ID of the reference user based on the target user information determined in step S500.

As a first example of step S510, the user information database (FIG. 7) is referenced to identify the information in the “user attribute” field associated with the target user ID.
The processor 32 refers to the user information database, refers to the “user attribute” field associated with the asymmetric user ID of the user other than the target user, and refers to the information in the “user attribute” field associated with the target user ID. Identify records that contain the same or similar information.
The processor 32 determines the user ID of the identified record as the reference user ID.

As a second example of step S510, the user information database (FIG. 7) is referenced to identify the information in the “fixed subjectivity” field associated with the target user ID.
The processor 32 refers to the user information database, refers to the “fixed subjectivity” field associated with the non-target user ID, and has a predetermined relationship with the information in the “fixed subjectivity” field associated with the target user ID. Identify records that contain.
The predetermined relationship is one of the following relationships.
A relationship in which the information in the "fixed subjectivity" field associated with the target user ID is the same as the information in the "fixed subjectivity" field associated with the asymmetric user ID ・ The "fixed subjectivity" field associated with the target user ID Information and the information in the “fixed subjectivity” field associated with the asymmetric user ID are a predetermined combination (for example, a combination having a good compatibility or a combination having a bad compatibility). The user ID of the record is determined as the reference user ID.

As a third example of step S510, the user information database (FIG. 7) is referenced to identify the information in the “fixed indirect objective” field associated with the target user ID.
The processor 32 refers to the user information database, refers to the “fixed indirect objective” field associated with the non-target user ID, and is the same as or similar to the information in the “fixed indirect objective” field associated with the target user ID. Specify the record that contains the information of.
The processor 32 determines the user ID of the identified record as the reference user ID.

As a fourth example of step S510, referring to the user flow subjective information database (FIG. 8), the user flow subjective information that is the same as or similar to the user flow subjective information associated with the target user ID from the non-target user IDs. The asymmetric user ID associated with is identified as the reference user ID.

As a fifth example of step S510, referring to the user flow indirect objective information database (FIG. 9), the user flow indirect information that is the same as or similar to the user flow indirect information associated with the target user ID among the non-target user IDs. The asymmetric user ID associated with the information is identified as the reference user ID.

After step S510, estimation of user emotion information (S502) is executed.
Specifically, the storage device 31 further has at least one of the following inference models built therein.
・Inference model of user attribute information and user emotion information ・Inference model of user fixation subjective information and user emotion information ・Inference model of user fixation indirect objective information and user emotion information ・User inference model of user flow subjective information and user emotion information ・User inference model of user flow indirect objective information and user emotion information

The processor 32 of the server 30 estimates the user emotion information using the inference model stored in the storage device 31 and the user inference model.

According to the modification, not only the subject information but also the reference user information other than the target user is referred to estimate the user emotion information. Thereby, the estimation accuracy of the emotion of the target user can be improved.

(6) Summary of the present embodiment The present embodiment will be summarized.

The first aspect of the present embodiment is
An information processing device (for example, a server 30) that estimates a user's emotion,
A means for acquiring direct objective information (for example, information that can be obtained by MRI or fMRI) directly related to the objective emotion of the subject (for example, the processor 32 that executes step S103),
A means for acquiring at least one of the subject subjective information related to the subject's subjective feeling and the subject indirect objective information indirectly related to the subject's objective feeling (for example, steps S101 to S102 and S200 to S202). A processor 32) executing at least one of
Means for acquiring at least one of the user subjective information related to the user's subjective feeling and the user indirect objective information indirectly related to the user's objective feeling (for example, steps S300 to S202 and S400 to S402). A processor 32) executing at least one of
A means for estimating user emotion information related to a user's emotion by referring to direct objective information, at least one of subject subjective information and subject indirect objective information, and at least one of user subjective information and user indirect objective information ( For example, comprising a processor 32) performing step S502,
It is an information processing device.

According to the first aspect, it is possible to infer the direct objective information that directly reflects the reaction of the brain that controls the emotion of the user who has not directly received the objective information. As a result, the emotion of the user can be estimated with high accuracy.

The second aspect of the present embodiment is
A means for presenting the estimated user emotion information (for example, the processor 32 that executes step S503),
It is an information processing device.

According to the second aspect, the emotion of the user can be visualized by the user.

The third aspect of the present embodiment is
The subjective information includes fixed subjective information related to subjective feelings fixed to the subject or the user, and flowing subjective information related to subjective feelings that are fluid to the subject or the user.
It is an information processing device.

According to the third aspect, the subjective information referred to for emotion estimation is divided into information fixed to the subject or the user and information that is fluid to the subject or the user. As a result, the accuracy of emotion estimation can be further improved.

The fourth aspect of the present embodiment is
The indirect objective information includes fixed objective information related to objective emotions fixed to the subject or the user, and fluid objective information related to objective emotions that are fluid to the subject or the user.
It is an information processing device.

According to the fourth aspect, indirect objective information referred to for emotion estimation is divided into information fixed to the subject or the user and information that is fluid to the subject or the user. As a result, the accuracy of emotion estimation can be further improved.

The fifth aspect of the present embodiment is
The direct objective information is at least one of MRI (Magnetic Resonance Imaging) examination, PET (Positron Emission Tomography) examination, fNIRS (functional Near Infrared Spectroscopy) examination, EEG (Electroencephalogram) examination, and MEG (Magnetoencephalography) examination for the subject. Information obtained by
It is an information processing device.

The sixth aspect of the present embodiment is
The means for estimating uses at least one of the subject subjective information and the subject indirect objective information, and the inference model configured by the direct objective information to extract the user emotion information from at least one of the user subjective information and the user indirect objective information. presume,
It is an information processing device.

A seventh aspect of the present embodiment is
A means for determining a reference user from a plurality of asymmetric users by referring to at least one of user subjective information and user indirect objective information of an asymmetric user different from the target user whose emotion should be estimated,
The estimating means includes at least one of the subject subjective information and the subject indirect objective information, at least one of the user subjective information and the user indirect objective information, at least one of the user subjective information and the user indirect objective information of the reference user, and the reference user. User emotion information, and estimates the user emotion information of the target user,
It is an information processing device.

According to the seventh aspect, not only the subject information but also the reference user information other than the target user is referred to estimate the user emotion information. Thereby, the estimation accuracy of the emotion of the target user can be improved.

The eighth aspect of the present embodiment is
It is a program for causing a computer (for example, the processor 32) to function as each of the above means.

(7) Other Modifications Other modifications will be described.

The storage device 11 may be connected to the client device 10 via the network NW. The storage device 31 may be connected to the server 30 via the network NW.

Each step of the above information processing can be executed by either the client device 10 or the server 30.

Steps S101 and S301 can be omitted. In this case, the emotion visualization (OP5) is performed using information other than the fixed subjective information.
Steps S102 and S302 can be omitted. In this case, emotion visualization (OP5) is executed.
Steps S200 and S400 can be omitted. In this case, emotion visualization (OP5) is executed.
Steps S201 and S202, and S401 and S402 can be omitted. In this case, emotion visualization (OP5) is executed.

The flowing subjective information may be a sentence created by the subject or the user. The flow subjective information includes at least one of the following, for example.
・Diary/Blog

The method of directly measuring objective information is not limited to MRI. The direct objective information can be acquired by at least one of the following measurement methods.
・PET (Positron Emission Tomography) inspection ・fNIRS (functional Near Infrared Spectroscopy) inspection ・EEG (Electroencephalogram) inspection ・MEG (Magnetoencephalography) inspection

In steps S201 to S202 and S401 to S402, not only the answer is accepted in the Q&A format, but also the option object of the subjective information is presented, and the subjective information corresponding to the operated option object is accepted as the flowing subjective information. Is also good.
The option object includes at least one of the following, for example.
・Icon for each emotion type ・Image object of color that represents emotion

In step S102 (FIG. 11), sound may be used instead of the image.
For example, the client device 10 includes a stereo speaker (as an example, a left speaker and a right speaker arranged on the left and right of the operator).
The processor 12 reproduces the subliminal sound from each of the left speaker and the right speaker.
The operator hears the subliminal sound reproduced from each of the left speaker and the right speaker, and gives the client device 10 a user instruction indicating which speaker reproduced the subliminal sound was easy to hear.
The processor 12 transmits the update request data including the user instruction (that is, the instruction indicating the speaker in which the subliminal sound is easy to hear) to the server 30.

Although the embodiments of the present invention have been described above in detail, the scope of the present invention is not limited to the above embodiments. In addition, the above-described embodiment can be variously modified and changed without departing from the gist of the present invention. Further, the above-described embodiments and modified examples can be combined.

1 :information processing system 10 :client device 11 :storage device 12 :processor 13 :input/output interface 14 :communication interface 30 :server 31 :storage device 32 :processor 33 :input/output interface 34 :communication interface

Claims (8)

An information processing device for estimating a user's emotion,
Equipped with means for acquiring direct objective information that is directly related to the objective emotion of the subject,
Subjects subjective information related to subjective feelings of the subject, and comprises means for obtaining at least one of the subjects indirect objective information indirectly related to the objective emotion of a subject,
And a means for acquiring at least one of user subjective information related to the user's subjective feeling and user indirect objective information indirectly related to the user's objective feeling,
A user emotion related to the emotion of the user with reference to the direct objective information, at least one of the subject subjective information and the subject indirect objective information, and at least one of the user subjective information and the user indirect objective information. Equipped with means for estimating information,
Information processing device. A means for presenting the estimated user emotion information,
The information processing apparatus according to claim 1. The subjective information includes fixed subjective information related to subjective feelings fixed to the subject or the user, and flowing subjective information related to subjective feelings that are fluid to the subject or the user.
The information processing apparatus according to claim 1. The indirect objective information includes fixed objective information related to objective emotions fixed to the subject or the user, and fluid objective information related to objective emotions that are fluid to the subject or the user,
The information processing device according to claim 1. The direct objective information is at least an MRI (Magnetic Resonance Imaging) test, a PET (Positron Emission Tomography) test, a fNIRS (functional Near Infrared Spectroscopy) test, an EEG (Electroencephalogram) test, and a MEG (Magnetoencephalography) test for the subject. Information obtained by one,
The information processing apparatus according to claim 1. The estimating unit uses at least one of the subject subjective information and the subject indirect objective information, and the inference model configured by the direct objective information, and uses at least one of the user subjective information and the user indirect objective information. Estimating the user emotion information from
The information processing apparatus according to claim 1. A means for determining a reference user from a plurality of asymmetric users by referring to at least one of user subjective information and user indirect objective information of an asymmetric user different from the target user whose emotion should be estimated,
The estimating means includes at least one of the subject subjective information and the subject indirect objective information, at least one of the user subjective information and the user indirect objective information, and at least the user subjective information and the user indirect objective information of the reference user. Estimating the user emotion information of the target user with reference to one and the user emotion information of the reference user,
The information processing device according to claim 1. A program for causing a computer to function as each unit according to claim 1.