JP2020048871A - Information processing device and program - Google Patents

Abstract

To improve the accuracy of estimating emotions of a person about whom data representing objective emotions do not exist.SOLUTION: An information processing device includes: means for acquiring direct objective information directly related to objective emotions of a subject; means for acquiring at least one of subject subjective information related to subjective emotions of the subject and subject indirect objective information indirectly related to objective emotions of the subject; means for acquiring at least one of user subjective information related to subjective emotions of a user and user indirect objective information indirectly related to objective emotions of the user; and means for estimating user emotion information related to emotions of the user with reference to the direct objective information, at least one of the subject subjective emotion information and the subject indirect objective information, and at least one of the user subjective information and the user indirect objective information.SELECTED DRAWING: Figure 2

Description

  The present invention relates to an information processing device and a program.

  Generally, a person uses a sentence (for example, a diary) as a method of recording his or her own emotion. Sentences express a person's subjective emotions. That is, a sentence created by a person can provide a subjective emotion index related to the subjective emotion of the person.

  On the other hand, Patent Literature 1 discloses a technique for determining a subject's emotion based on information obtained by an MRI (Magnetic Resonance Imaging) test. Since the information obtained by the MRI examination is a quantitative value, it directly represents the objective emotion of the subject.

JP 2013-176406 A

  However, it is not easy to receive the MRI test of Patent Document 1 because it is expensive and troublesome. Therefore, there are many people who have never undergone an MRI examination (that is, there is no data indicating objective emotion). Therefore, the emotion of a person who has never undergone an MRI test cannot be determined with high accuracy.

  As described above, in the related art, the accuracy of emotion estimation is not sufficient for a person for whom there is no data representing objective emotion.

  An object of the present invention is to improve the accuracy of estimating emotions of a person who does not have data representing objective emotions.

One embodiment of the present invention provides
An information processing apparatus for estimating a user's emotion,
Means for obtaining direct objective information directly related to the subject's objective emotions,
Subject subjective information related to the subject's subjective emotion, and means for acquiring at least one of the test indirect objective information indirectly related to the subject's objective emotion,
A means for acquiring at least one of user subjective information related to the user's subjective emotion, and user indirect objective information indirectly related to the user's objective emotion,
Means for estimating user emotion information related to the emotion of the user by referring to the direct objective information, at least one of the subject subjective information and the subject indirect objective information, and at least one of the user subjective information and the user indirect objective information; Prepare,
It is an information processing device.

  According to the present invention, it is possible to improve the accuracy of estimating the emotion of a person for whom there is no data representing objective emotion.

It is a block diagram showing the composition of the information processing system of this embodiment. It is an explanatory view of the outline of the present embodiment. It is an explanatory view of one mode of an outline of this embodiment. It is a figure showing the data structure of the subject information database of this embodiment. It is a figure showing the data structure of the subject flow subjective information database of this embodiment. It is a figure showing the data structure of the subject flow indirect objective information database of this embodiment. It is a figure showing the data structure of the user information database of this embodiment. It is a figure showing the data structure of the user flow subjective information database of this embodiment. It is a figure showing the data structure of the user flow indirect objective information database of this embodiment. It is a figure showing the whole flow of information processing of this embodiment. FIG. 11 is a sequence diagram of a process of registering subject information in FIG. 10. FIG. 12 is a diagram illustrating an example of a screen displayed in the information processing in FIG. 11. FIG. 12 is a diagram illustrating an example of a screen displayed in the information processing in FIG. 11. FIG. 11 is a sequence diagram of a process of registering subject flow information of FIG. 10. FIG. 15 is a diagram illustrating an example of a screen displayed in the information processing of FIG. 14. FIG. 15 is a diagram illustrating an example of a screen displayed in the information processing of FIG. 14. FIG. 15 is a diagram illustrating an example of a screen displayed in the information processing of FIG. 14. FIG. 11 is a sequence diagram of a process (OP3) of registering user information in FIG. FIG. 19 is a diagram illustrating an example of a screen displayed in the information processing in FIG. 18. FIG. 11 is a sequence diagram of a process of registering the user flow information of FIG. 10. It is a flowchart of the processing of visualization of the emotion of FIG. FIG. 22 is a diagram showing an example of a screen displayed in the information processing of FIG. 21. It is a figure showing the data structure of the user emotion information database of a modification. It is a flowchart of the processing of visualization of the emotion of the modification.

  Hereinafter, an embodiment of the present invention will be described in detail with reference to the drawings. In the drawings for describing the embodiments, the same components are denoted by the same reference numerals in principle, and the repeated description thereof will be omitted.

(1) Configuration of Information Processing System The configuration of the information processing system will be described. FIG. 1 is a block diagram illustrating the configuration of the information processing system according to the present embodiment.

As shown in FIG. 1, the information processing system 1 includes a client device 10 and a server 30.
The client device 10 and the server 30 are connected via a network (for example, the Internet or an intranet) NW.

  The client device 10 is an example of an information processing device that transmits a request to the server 30. The client device 10 is, for example, a smartphone, a tablet terminal, or a personal computer.

  The server 30 is an example of an information processing device that provides a response to a request transmitted from the client device 10 to the client device 10. The server 30 is, for example, a web server.

(1-1) Configuration of Client Device The configuration of the client device 10 will be described with reference to FIG.

  As shown in FIG. 1, the client device 10 includes a storage device 11, a processor 12, an input / output interface 13, and a communication interface 14.

  The storage device 11 is configured to store programs and data. The storage device 11 is, for example, a combination of a ROM (Read Only Memory), a RAM (Random Access Memory), and a storage (for example, a flash memory or a hard disk).

The programs include, for example, the following programs.
-OS (Operating System) program-Application (for example, web browser) program that executes information processing

The data includes, for example, the following data.
・ Database referred to in information processing ・ Data obtained by executing information processing (that is, execution result of information processing)

  The processor 12 is configured to realize a function of the client device 10 by activating a program stored in the storage device 11. Processor 12 is an example of a computer.

The input / output interface 13 is configured to acquire a user's instruction from an input device connected to the client device 10 and output information to an output device connected to the client device 10.
The input device is, for example, a keyboard, a pointing device, a touch panel, or a combination thereof.
The output device is, for example, a display.

  The communication interface 14 is configured to control communication between the client device 10 and the server 30.

(1-2) Configuration of Server The configuration of the server 30 will be described with reference to FIG.

  As shown in FIG. 1, the server 30 includes a storage device 31, a processor 32, and a communication interface.

  The storage device 31 is configured to store programs and data. The storage device 31 is, for example, a combination of a ROM, a RAM, and a storage (for example, a flash memory or a hard disk).

The programs include, for example, the following programs.
-OS program-Application program that executes information processing

The data includes, for example, the following data.
・ Database referenced in information processing ・ Execution result of information processing

  The processor 32 is configured to realize a function of the server 30 by activating a program stored in the storage device 31. Processor 32 is an example of a computer.

The input / output interface 33 is configured to acquire a user instruction from an input device connected to the server 30 and output information to an output device connected to the server 30.
The input device is, for example, a keyboard, a pointing device, a touch panel, or a combination thereof.
The output device is, for example, a display.

  The communication interface 34 is configured to control communication between the server 30 and the client device 10.

(2) Overview of the present embodiment An overview of the present embodiment will be described. FIG. 2 is an explanatory diagram of the outline of the present embodiment.

As shown in FIG. 2, the subject undergoes three types of tests (a subjective test, an indirect objective test, and a direct objective test). In other words, three types of subject test information (subjective information, indirect objective information, and direct objective information) are associated with the subject.
The subjective test is a test for measuring subject subjective information related to the subject's subjective emotion. The subjective test is performed by, for example, a Q & A type test. The subjective information is information determined by the subject's subjectivity.
The indirect objective test is a test for measuring indirect objective information related to a subject's objective emotion. The indirect objective inspection is, for example, an inspection using a subliminal test or a wearable device. Subject indirect objective information does not directly reflect the response of the brain that controls the emotions of the subject, but does reflect it indirectly. The subject-indirect objective information is, for example, information indicating a pulse, a pupil size, and a response of a brain governing a subject's emotions inferred from at least one of sweat glands.
The direct objective test is a test for measuring direct objective information related to a subject's objective emotion. The direct objective test is, for example, a test performed using a brain image processing device (for example, an MRI device or an fMRI (functional Magnetic Resonance Imaging) device). The direct objective information directly reflects the reaction of the brain that controls the emotions of the subject and the brain function related to the emotions (for example, suppression of emotions). In other words, the direct objective information is information that expresses the subject's emotion most objectively in the examination information.

The user receives two types of tests (a subjective test and an indirect objective test). That is, two types of user examination information (subjective information and indirect objective information) are associated with the user.
The user, unlike the subject, has not received a direct objective test. Therefore, there is no direct objective emotion information of the user.

In the present embodiment, the correlation between the combination of the subjective information and the indirect objective information and the direct objective information is calculated from three types of test information (subjective information, indirect objective information, and direct objective information) associated with the subject. I do.
In addition, referring to the correlation between the subjective information and the indirect objective information associated with the subject and the direct objective information, the user can obtain the user's information from the two types of test information (the subjective information and the indirect objective information) associated with the user. Estimate emotion information (eg, emotional condition, emotional tendency, and emotional factors affecting the emotions of the user).

  As described above, according to the present embodiment, it is possible to infer direct objective information directly reflecting a brain reaction that controls the emotions of a user who has not received an objective test. As a result, it is possible to highly accurately estimate the emotion of the user who has not received the objective test directly.

(2-1) One aspect of the outline of this embodiment One aspect of the outline of this embodiment will be described. FIG. 3 is an explanatory diagram of one aspect of the outline of the present embodiment.

As shown in FIG. 3, the subjective tests are classified into two types (fixed subjective tests and flow subjective tests).
The sticking subjective test is, for example, a Q & A type test. The test question includes context for reminding the subject of past self-related events and experiences and for examining self-awareness from patterns of the events and experiences. Therefore, the information obtained from the answer to this test is the sticking subjective information.
The flow subjective test is, for example, a Q & A type test. The test questions include context to remind the subject of one of his most recent events and experiences. Therefore, the information obtained from the answer of this test is the flow subjective information. The difference between the plural pieces of flowing subjective information and the fixed subjective information is a difference in the ability of the individual to see the self and the meta-cognitive ability. This difference is one of the useful indices for emotion observation.

  The objective inspection is classified into two types (indirect objective inspection and direct objective inspection).

The indirect objective inspection is classified into a sticky indirect objective inspection and a flow indirect objective inspection.
The sticking indirect objective test is, for example, a subliminal test performed on a subject. In the subliminal test, content (image, sound, or a combination thereof) that gives a subliminal effect to the subject is presented, and the subject's response to the content is measured. In general, a brain subjected to a subliminal effect exhibits a particular response that depends on the individual's emotions. In other words, the measured response reflects the emotion that has stuck to the subject's brain. Therefore, the subject's response is sticky indirect objective information.
The flow indirect objective test is, for example, a test using a wearable device worn by a subject. Data acquired by the wearable device fluctuates depending on the timing of measurement. Therefore, this data is flow indirect objective information.

The direct objective inspection is classified into a sticky direct objective inspection and a flow direct objective inspection.
The fixed direct objective test is, for example, an MRI test performed on a subject. The images acquired by the MRI apparatus directly reflect the emotions fixed on the brain of the subject. Therefore, the image acquired by the MRI apparatus is the fixed indirect objective information.
The flow direct objective test is, for example, an EEG (Electroencephalogram) test or fMRI test performed on a subject. The result of the EEG test or the fMRI test varies depending on the timing of the test. Therefore, the results of the EEG test and the fMRI test are flow direct objective information.

In one aspect of the present embodiment, subjective information and indirect objective information are obtained from five types of test information (fixed subjective information, fluid subjective information, fixed indirect objective information, fluid indirect objective information, and direct objective information) associated with the subject. Calculate the correlation between the information combination and the direct objective information.
Also, by referring to the correlation between the combination of the subjective information and the indirect objective information associated with the subject and the direct objective information, the four types of test information associated with the user (fixed subjective information, fluid subjective information, fixed indirect information, The user's emotion information (for example, the emotional condition, the emotional tendency, and the emotional factor affecting the emotional state of the user) is estimated from the objective information and the fluid indirect objective information.

(3) Database The database of the present embodiment will be described. The following databases are stored in the storage device 31.

(3-1) Subject Information Database The subject information database of the present embodiment will be described. FIG. 4 is a diagram showing a data structure of the subject information database of the present embodiment.

The subject information database of FIG. 4 stores subject information on the subject.
The subject information database includes a “subject ID” field, a “subject name” field, a “subject attribute” field, a “stick subjectivity” field, a “direct objective” field, and a “stick indirect objective” field. .
Each field is associated with one another.

  A subject ID (an example of “subject identification information”) for identifying a subject is stored in the “subject ID” field.

  In the “subject name” field, information (for example, text) on the subject name is stored. The subject is arbitrarily determined by the subject when registering the subject name, for example, subject information.

The “subject attribute” field stores subject attribute information regarding the attributes of the subject. The subject attribute information is arbitrarily determined by the subject when registering the subject information, for example.
The “subject attribute” field includes a “sex” field, an “age” field, and a “occupation” field.

  The “sex” field stores information on the gender of the subject.

  The “age” field stores information on the age of the subject.

  The “occupation” field stores information on the occupation of the subject.

  The subject's subjective fixation information is stored in the “fixation subjective” field. The fixed subjective information is information determined by a combination of answers to a plurality of questions prepared in advance. The sticking subjective information is, for example, information on the type of the subject's character (hereinafter, referred to as “personal type”).

The "direct objective" field stores direct objective information of the subject.
The “direct objective” field includes an “MRI” field and an “fMRI” field.

  The “MRI” field stores an MRI image (an example of “direct objective information”) related to an MRI examination result. The MRI image is, for example, image data of the brain of the subject.

  The “fMRI” field stores fMRI information (an example of “direct objective information”) on the fMRI test result. The fMRI information includes hemodynamic response parameters (eg, amygdala activity, islet activity, dorsolateral prefrontal cortex inactivity, blood cortisol level) related to brain activity of each part of the brain in response to each of a plurality of stimuli. , Blood pressure fluctuation rate, and facial muscles).

The “sticky indirect objective” field stores sticky indirect objective information obtained from a sticky indirect objective test (for example, a subliminal test) on the subject. The fixed indirect objective information includes, for example, the following.
・ Content of reaction when presenting subliminal content ・ Reaction speed when presenting subliminal content

(3-2) Subject flow subjective information database The subject flow subjective information database of the present embodiment will be described. FIG. 5 is a diagram showing a data structure of the subject flow subjective information database of the present embodiment.

The subject flow subjective information database of FIG. 5 stores answers to the questions presented to the subject.
The subject flow subjective information database includes a “data ID” field, a “check-in date and time” field, a “check-in response” field, a “check-out date and time” field, and a “check-out response” field.
Each field is associated with one another.
The subject flow subjective information database is associated with the subject ID.

  The “data ID” field stores a data ID for identifying a record in the subject flow subjective information database.

  The “check-in date and time” field stores information on the date and time of check-in. Check-in is a process for acquiring an answer to a question presented to a subject before executing a certain task.

  The “check-in response” field stores check-in response information relating to the response obtained at check-in. The information in the “check-in answer” field is stored separately for each question.

  The “checkout date and time” field stores information on the date and time of checkout. Checkout is a process for acquiring an answer to a question presented to a subject after completing a certain task.

  The “checkout answer” field stores checkout answer information on the answer obtained in the checkout. The information in the "checkout answer" field is stored separately for each question.

(3-3) Subject Flow Indirect Objective Information Database The subject flow indirect objective information database of the present embodiment will be described. FIG. 6 is a diagram showing the data structure of the subject flow indirect objective information database of the present embodiment.

The subject's flow indirect objective information of FIG. 6 stores the subject's flow indirect objective information.
The subject flow indirect objective information database includes a “data ID” field, a “measurement date and time” field, and a “wearable” field.
The subject flow indirect objective information database is associated with the subject ID.

  The “data ID” field stores a data ID for identifying a record in the subject flow indirect objective information database.

  The “measurement date and time” field stores information on the date and time when the data measured by the wearable device is acquired.

The “wearable” field stores data measured by the wearable device. The data measured by the wearable device includes, for example, at least one of the following.
・ Steps, pulse, heart rate, calories burned

(3-4) User Information Database The user information database of the present embodiment will be described. FIG. 7 is a diagram illustrating a data structure of the user information database according to the present embodiment.

The user information database of FIG. 7 stores user information on the user.
The user information database includes a "user ID" field, a "user name" field, a "user attribute" field, a "sticky subjective" field, and a "sticky indirect objective" field.
Each field is associated with one another.

  The “user ID” field stores a user ID for identifying a user (an example of “user identification information”).

  The “user name” field stores information (for example, text) related to the user name. The user name is arbitrarily determined by the user when registering the user information, for example.

The “user attribute” field stores user attribute information on the attribute of the user. The user attribute information is arbitrarily determined by the user when registering the user information, for example.
The “user attribute” field includes a “sex” field, an “age” field, and a “occupation” field.

  The “sex” field stores information about the gender of the user.

  The “age” field stores information on the age of the user.

  The “occupation” field stores information on the occupation of the user.

  The “fixed subjective” field stores the fixed subjective information of the user. The fixed subjective information is information determined by a combination of answers to a plurality of questions prepared in advance. The sticking subjective information is, for example, information on the personality type of the user.

  The “sticky indirect objective” field stores sticky indirect objective information obtained from a sticky indirect objective inspection (for example, a subliminal test) for the user. An example of the fixed indirect objective information is the same as the example of the “fixed indirect objective” field of the subject information database (FIG. 4).

(3-5) User flow subjective information database The user flow subjective information database of the present embodiment will be described. FIG. 8 is a diagram showing the data structure of the user flow subjective information database of the present embodiment.

The answer to the question presented to the user is stored in the user flow subjective information database of FIG.
The user flow subjective information database includes a “data ID” field, a “check-in date and time” field, a “check-in response” field, a “check-out date and time” field, and a “check-out response” field.
Each field is associated with one another.
The user flow subjective information database is associated with the user ID.

  The “data ID” field stores a data ID for identifying a record in the user flow subjective information database.

  The “check-in date and time” field stores information on the date and time of check-in. Check-in is a process for acquiring an answer to a question presented to a user before executing a certain task.

  The “check-in response” field stores check-in response information relating to the response obtained at check-in. The information in the “check-in answer” field is stored separately for each question.

  The “checkout date and time” field stores information on the date and time of checkout. Checkout is a process for acquiring an answer to a question presented to a user after completing a certain task.

  The “checkout answer” field stores checkout answer information on the answer obtained in the checkout. The information in the "checkout answer" field is stored separately for each question.

(3-6) User flow indirect objective information database The user flow indirect objective information database of the present embodiment will be described. FIG. 9 is a diagram showing the data structure of the user flow indirect objective information database of the present embodiment.

The user's flow indirect objective information of FIG. 9 stores the user's flow indirect objective information.
The user flow indirect objective information database includes a “data ID” field, a “measurement date and time” field, and a “wearable” field.
The user flow indirect objective information database is associated with the user ID.

  The “data ID” field stores a data ID for identifying a record in the user flow indirect objective information database.

  The “measurement date and time” field stores information on the date and time when the data measured by the wearable device is acquired.

  The “wearable” field stores data measured by the wearable device. An example of the data measured by the wearable device is the same as the example of the “wearable” field of the subject flow indirect objective information database (FIG. 6).

(4) Information Processing Information processing of the present embodiment will be described. FIG. 10 is a diagram illustrating an overall flow of the information processing according to the present embodiment.

As shown in FIG. 10, the information processing of this embodiment includes registration of subject information (OP1), registration of subject flow information (OP2), registration of user information (OP3), and registration of user flow information (OP4). ) And visualization of emotion (OP5).
Registration of subject information (OP1) and registration of subject flow information (OP2) are performed on the subject information database (FIG. 4), the subject flow subjective information database (FIG. 5), and the subject flow indirect objective information database (FIG. 6). This is a process for registering data. By registering the subject information (OP1) and registering the subject flow information (OP2), the server 30 constructs the subject test information and the emotion inference model.
Registration of user information (OP3) and registration of user flow information (OP4) are performed in the user information database (FIG. 7), the user flow subjective information database (FIG. 8), and the user flow indirect objective information database (FIG. 9). This is a process for registering data. By registering the user information (OP3) and registering the user flow information (OP4), the input information necessary for visualizing the emotion (OP5) is provided to the server 30.
The emotion visualization (OP5) is given in the inference model generated by the registration of the subject information (OP1) and the registration of the subject flow information (OP2), and the registration of the user information (OP3) and the registration of the user flow information (OP4). This is a process for visualizing the user's emotion using the input information obtained.

(4-1) Registration of Subject Information The processing of registration of subject information (OP1) of the present embodiment will be described. FIG. 11 is a sequence diagram of the registration processing of the subject information in FIG. 12 and 13 are diagrams illustrating examples of screens displayed in the information processing of FIG.

As shown in FIG. 11, reception of subject information (S100) is performed.
Specifically, processor 12 of client device 10 displays screen P100 (FIG. 12) on the display.

Screen P100 includes an operation object B100 and a field object F100.
The field object F100 is an object that receives input of subject information (for example, subject name, gender, age, and occupation).
The operation object B100 is an object that receives an instruction to cause the server 30 to transmit information input to the field object F100.

  When the subject inputs the subject information to the field object F100 and operates the operation object B100, the processor 12 transmits the subject information registration request data to the server 30. The subject information registration request data includes the information input to the field object F100.

The processor 32 of the server 30 adds a new record to the subject information database (FIG. 4).
The following information is stored in each field of the new record.
A new subject ID is stored in the “subject ID” field.
In the “subject name” field and the “subject attribute” field, information included in the subject information registration request data (that is, information input to the field object F100) is stored.

After step S100, an initial Q & A test (S101) is performed.
Specifically, processor 12 of client device 10 displays screen P101 (FIG. 12) on the display.

The screen P101 includes an operation object B101 and a field object F101.
The field object F101 displays a question and options for answering the question. When the operator specifies at least one of the options, the field object F101 receives the option specified by the operator as the operator's answer.
The operation object B101 is an object that receives an instruction for determining the answer received by the field object F101.

  When the operation object B101 is operated, the processor 12 transmits update request data to the server 30. The update request data includes the information input to the field object F101.

The storage device 31 of the server 30 stores a matching table indicating the relationship between the combination of information input to the field object F101 and the personality type of the input user.
The processor 32 determines the personality type corresponding to the information included in the update request data with reference to the matching table stored in the storage device 31.
The processor 32 stores information on the determined personality type in the “fixed subjectivity” field of the new record added in step S100.
The processor 32 transmits the update response data to the client device 10. The update response data includes information on the personality type.

The processor 12 of the client device 10 displays the screen P102 (FIG. 12) on the display based on the update request data.
The screen P102 displays the personality type determined by the server 30.

After step S101, a subliminal test (S102) is performed.
Specifically, processor 12 of client device 10 displays screen P103 (FIG. 13) on the display.

Screen P103 includes display object A103.
The display object A103 displays a message prompting the subject to perform an operation (for example, a text message “Click on the displayed triangle”).

  After a lapse of time t1 (for example, 4 seconds) from the display of the screen P103, the processor 12 displays the screen P104 (FIG. 13) on the display.

Screen P104 includes display objects A103 and A104.
The display object A104 includes a display area A104a to a display area A104b.
Image objects IMG104a to IMG104b are displayed in display areas A104a to A104b, respectively. The image objects IMG104a to IMG104b are displayed side by side in a predetermined direction. The image objects IMG104a to IMG104b are, for example, rectangular images.

  After a lapse of time t2 (<t1) from the display of the screen P104, the processor 12 displays the screen P105a or P105b (FIG. 13) on the display. The time t2 is, for example, 0.01 second. That is, the screen P104 is displayed for an extremely short time as compared with the screen P103.

  Thus, the display time t2 of the screen P104 is much shorter than the display time t1 of the screen P103. Therefore, although the image objects IMG104a to IMG104b cannot be recognized by human vision, the brain usually recognizes the part below the “consciousness” (so-called subconscious, or a boundary region between consciousness and subconscious). It is possible. Therefore, the test subject unconsciously focuses on either the image object IMG 104a or the IMG 104b. This phenomenon is called "subliminal effect". In step S102, the indirect objective information of the subject is acquired by using the subliminal effect.

The screen P105a includes a display object A104.
The image object IMG105a is displayed in the display area A104a of the display object A104. The image object IMG105a is, for example, a triangular image. The image object IMG 105a is an object that can be operated by the operator.

The screen P105b includes a display object A104.
The image object IMG105b is displayed in the display area A104b of the display object A104. The image object IMG105b is, for example, a triangular image. The image object IMG 105b is an object that can be operated by the operator.

When the subject operates the image object IMG105a or IMG105b, the processor 12 stores the time from when the screen P105a or P105b is displayed to when the subject operates the image object IMG105a or IMG105b (hereinafter referred to as "reaction time") in the storage device 11. I do.
For example, when the subject is paying attention to the image object IMG104a on the screen P104, the reaction time to the operation of the image object IMG105a displayed in the same display area A104a as the image object IMG104a is displayed in the display area A104b different from the image object IMG104a. The response time to the operation of the displayed image object IMG 105b is shorter.

The processor 12 repeatedly executes the above-described sabilimetric test a plurality of times.
When a predetermined number of subliminal tests are completed, the processor 12 transmits update request data to the server 30. The update request data includes the following information.
Information about the reaction time for the image object IMG105a Information about the reaction time for the image object IMG105b

The processor 32 stores information on the reaction time included in the update request data in the “sticky indirect objective” field of the new record added in step S100.
The processor 32 transmits the update response data to the client device 10.

After step S102, an MRI inspection (S103) is performed.
Specifically, a computer connected to the magnetic resonance apparatus generates a brain activity parameter and a hemodynamic response parameter of the subject based on a signal obtained by the magnetic resonance apparatus.
The computer transmits the update request data to the server 30.
The update request data includes the following information.
・ Brain activity parameters ・ Hemodynamic response parameters

The processor 32 of the server 30 stores the following information in the new record added in step S100.
-The brain activity parameter included in the update request data is stored in the "MRI" field.
-The hemodynamic response parameters included in the update request data are stored in the "fMRI" field.

(4-2) Registration of subject flow information The process of registering subject flow information (OP2) of the present embodiment will be described. FIG. 14 is a sequence diagram of the process of registering the subject flow information of FIG. 15 to 17 are diagrams illustrating examples of screens displayed in the information processing in FIG.

As shown in FIG. 14, the acquisition of wearable information (S200) is performed.
The processor 12 of the client device 10 acquires wearable information (for example, at least one of the number of steps, the pulse, and the heart rate) from the wearable device worn by the subject.
The processor 12 sends the update request data to the server 30. The update request data includes the following information.
・ Subject ID of the subject wearing the wearable device
-Wearable information-Information on the execution date and time of step S200

The processor 32 of the server 30 adds a new record to the subject flow indirect objective information database (FIG. 6) associated with the subject ID included in the update request data.
The following information is stored in each field of the new record.
-The new data ID is stored in the "data ID" field.
-The "measurement date" field stores information on the execution date and time included in the update request data.
-The "wearable" field stores wearable information included in the update request data.

After step S200, check-in (S201) is performed.
Specifically, processor 12 of client device 10 displays screen P200 (FIG. 15) on the display.

  Screen P200 includes an operation object B200 and field objects F200a to F200e.

In the field objects F200a to F200e, a question regarding a task performed after step S201 is presented.
The question "What is the content of the task?" Is presented to the field object F200a. When any text is input to the field object F200a, the processor 12 acquires the input text as information relating to the content of the task.
The question "What kind of task is it?" Is presented to the field object F200b. When an arbitrary option is selected from a plurality of options of the field object F200b, the processor 12 acquires the selected option as information on the task type.
The field object F200c is presented with a question "who is with whom?" When an arbitrary option is selected from a plurality of options of the field object F200c, the processor 12 acquires the selected option as information on a person who performs the task together.
The field object F200d is presented with a question "What is the frequency of tasks?" When the slider of the field object F200d is moved, the processor 12 acquires a numerical value corresponding to the position of the slider as information on the frequency of the task.
The question "How difficult is the task?" Is presented to the field object F200e. When the slider of the field object F200e is moved, the processor 12 acquires a numerical value corresponding to the position of the slider as information on the task difficulty.

  When the operation object B200 is operated, the screen transits to a screen P201 (FIG. 15).

  The screen P201 includes an operation object B201 and field objects F201a to F201b.

In the field objects F201a to F201b, a question regarding an action performed before step S201 is presented.
The field object F201a is presented with a question "What was you doing in the last 30 minutes?" When an arbitrary option is selected from a plurality of options of the field object F201a, the processor 12 acquires the selected option as information on an action performed 30 minutes immediately before check-in.
The field object F201b is presented with a question "What was the last 30 minutes?" When the slider of the field object F201b is moved, the processor 12 acquires a numerical value corresponding to the position of the slider as information relating to self-evaluation of the action performed 30 minutes immediately before check-in.

  When the operation object B201 is operated, the screen transits to a screen P202 (FIG. 16).

  The screen P202 includes an operation object B202 and field objects F202a to F202g.

In the field objects F202a to F202g, questions related to subjective emotions before performing a task are presented.
In the field object F202a, a question "What is the degree of hunger?" Is displayed. When the slider of the field object F202a is moved, the processor 12 acquires a numerical value corresponding to the position of the slider as information on the hunger degree.
The question “Do you sleepy?” Is displayed in the field object F202b. When the slider of the field object F202b is moved, the processor 12 acquires a numerical value corresponding to the position of the slider as information about drowsiness.
The question "What is stress?" Is displayed in the field object F202c. When the slider of the field object F202c is moved, the processor 12 acquires a numerical value corresponding to the position of the slider as information regarding stress.
The field object F202d displays a question "What was the previous task?" When the slider of the field object F202d is moved, the processor 12 acquires a numerical value corresponding to the position of the slider as information related to self-evaluation of a task performed before check-in.
In the field object F202e, a question “What is your motivation?” Is displayed. When the slider of the field object F202e is moved, the processor 12 acquires a numerical value corresponding to the position of the slider as information on motivation for the task.
In the field object F202f, a question "What is your anxiety?" Is displayed. When the slider of the field object F202f is moved, the processor 12 acquires a numerical value corresponding to the position of the slider as information relating to anxiety about the task.
The question "What is the degree of clarity?" Is displayed in the field object F202g. When the slider of the field object F202g is moved, the processor 12 acquires a numerical value corresponding to the position of the slider as information relating to the clarity of the task.

  When the operation object B202 is operated, the screen changes to a screen P203 (FIG. 16).

  The screen P203 includes an operation object B203 and field objects F203a to F203b.

In the field objects F203a to F203b, a question about a scheduled action after performing a task is presented.
In the field object F203a, a question "What is the action after the task?" Is presented. When an arbitrary option is selected from a plurality of options of the field object F203a, the selected option is acquired as information on the scheduled action after performing the task.
In the field object F203b, a question "What is the pleasure of the action after the task?" Is presented. When the slider of the field object F203b is moved, the processor 12 obtains a numerical value corresponding to the position of the slider as information relating to an expectation of an action after the task.

  The operation object B203 is an object that receives an instruction for completing the processing in step S201.

When the operation object B203 is operated, the processor 12 of the client device 10 sends check-in completion request data to the server 30.
Check-in completion request data includes the following information.
・ Subject ID of the subject who answered
-Check-in time information indicating the execution date and time of step S201-Information input to field objects F200a to F200e, F201a to F201b, F202a to F202g, and F203a to F203b

  The processor 32 of the server 30 adds a new record to the subject flow subjective information database (FIG. 5) associated with the subject ID included in the check-in request data.

The following information is stored in each field of the new record.
-A new data ID is stored in the "data ID" field.
-The "check-in date" field stores check-in time information included in the check-in completion request data.
In the "check-in answer" field, answer information included in the check-in completion request data is stored for each question item.

After step S201, a check-out (S202) process is executed.
Specifically, processor 12 of client device 10 displays screen P204 (FIG. 17) on the display.

  Screen P204 includes an operation object B204 and field objects F204a to F204f.

Questions relating to completed tasks are presented to the field objects F204a to F204f.
The question "What did you do?" Is displayed in the field object F204a. When any text is input to the field object F204a, the processor 12 acquires the input text as information on the content of the completed task.
In the field object F204b, a question "What is the degree of completion?" Is displayed. When the slider of the field object F204b is moved, the processor 12 acquires a numerical value corresponding to the position of the slider as information on the degree of completion of the task.
The question "What did you do?" Is displayed in the field object F204c. When an arbitrary text is input to the field object F204c, the processor 12 acquires the input text as information relating to the content of the task left behind.
The question "What is the nature of the task?" Is displayed in the field object F204d. When the slider of the field object F204d is moved, the processor 12 acquires a numerical value corresponding to the position of the slider as information on the nature of the task (for example, the ratio of workability and directivity).
The question "input amount?" Is displayed in the field object F204e. When the slider of the field object F204e is moved, the processor 12 acquires a numerical value corresponding to the position of the slider as information on the input amount of the task.
The question "What is the output amount?" Is displayed in the field object F204f. When the slider of the field object F204f is moved, the processor 12 acquires a numerical value corresponding to the position of the slider as information on the output amount of the task.

  When the operation object B204 is operated, the screen transitions to a screen P205.

  The screen P205 includes an operation object B205 and field objects F205a to F205f.

The question "How do you feel now?" Is displayed in the field object F205a. When an arbitrary option is selected from the plurality of options of the field object F205a, the processor 12 acquires the selected option as information on mood after completing the task.
In the field object F205b, a question "What is the feeling of frustration?" Is displayed. When the slider of the field object F205b is moved, the processor 12 obtains a numerical value corresponding to the position of the slider as information on the feeling of the task feeling.
In the field object F205c, a question "What is your sense of accomplishment?" Is displayed. When the slider of the field object F205c is moved, the processor 12 acquires a numerical value corresponding to the position of the slider as information on a sense of accomplishment for the task.
The question "What is the degree of concentration?" Is displayed in the field object F205d. When the slider of the field object F205d is moved, the processor 12 acquires a numerical value according to the position of the slider as information on the degree of concentration on the task.
In the field object F205e, a question "What is the degree of satisfaction?" Is displayed. When the slider of the field object F205e is moved, the processor 12 acquires a numerical value corresponding to the position of the slider as information on the degree of satisfaction with the task.
In the field object F205f, a question "How tired?" Is displayed. When the slider of the field object F205f is moved, the processor 12 acquires a numerical value corresponding to the position of the slider as information on the feeling of fatigue for the task.

  The operation object B205 is an object that receives an instruction for completing the processing in step S202.

When the operation object B205 is operated, the processor 12 of the client device 10 transmits checkout completion request data to the server 30.
Checkout completion request data includes the following information.
・ Subject ID of the subject who answered
-Check-in time information indicating the execution date and time of step S202-Information input to field objects F204a to F204f and F205a to F205f

The processor 32 of the server 30 specifies the subject flow subjective information database (FIG. 5) associated with the subject ID included in the checkout request data.
The processor 32 specifies the latest record (that is, the new record added in step S201) indicating the latest date and time in the information of the “check-in date and time” field in the specified subject flow subjective information database.

The following information is stored in each field of the latest record.
The “checkout date and time” field stores checkout time information included in the checkout completion request data.
In the "checkout answer" field, answer information included in the checkout completion request data is stored for each question item.

(4-3) User Information Registration Process The user information registration process according to the present embodiment will be described. FIG. 18 is a sequence diagram of the user information registration process (OP3) in FIG. FIG. 19 is a diagram showing an example of a screen displayed in the information processing of FIG.

As shown in FIG. 18, registration of user information (S300) is executed.
Specifically, processor 12 of client device 10 displays screen P300 (FIG. 19) on the display.

Screen P300 includes an operation object B300 and a field object F300.
The field object F300 is an object that receives input of user information (for example, user name, gender, age, and occupation).
The operation object B300 is an object that receives an instruction to cause the server 30 to transmit information input to the field object F300.

  When the user inputs user information to the field object F300 and operates the operation object B300, the processor 12 transmits user information registration request data to the server 30. The user information registration request data includes the information input to the field object F300.

The processor 32 of the server 30 adds a new record to the user information database (FIG. 7).
The following information is stored in each field of the new record.
-The "user ID" field stores a new user ID.
In the “user name” field and the “user attribute” field, information included in the user information registration request data (that is, information input to the field object F300) is stored.

After step S300, an initial Q & A test (S301) is performed.
Step S301 differs from step S101 in FIG. 11 in the following points.
The processor 32 of the server 30 stores information included in the update request data (that is, information on the personality type of the user) in the “stickiness subject” field of the new record added in step S300.

After step S301, a subliminal test (S302) is performed.
Step S302 differs from step S102 in FIG. 11 in the following points.
The processor 32 of the server 30 stores information included in the update request data (that is, information on the reaction time of the user with respect to the image objects IMG105a to IMG105b) in the “sticky indirect objective” field of the new record added in step S300. .

(4-4) Registration of User Flow Information The process of registering user flow information (OP4) of the present embodiment will be described. FIG. 20 is a sequence diagram of the process of registering the user flow information of FIG.

As shown in FIG. 20, the acquisition of wearable information (S400) is performed.
The processor 12 of the client device 10 acquires wearable information (for example, at least one of the number of steps, the pulse, and the heart rate) from the wearable device worn by the user.
The processor 12 sends the update request data to the server 30. The update request data includes the following information.
・ User ID of the user wearing the wearable device
-Wearable information-Information on the execution date and time of step S400

The processor 32 of the server 30 adds a new record to the user flow indirect objective information database (FIG. 9) associated with the user ID included in the update request data.
The following information is stored in each field of the new record.
-The new data ID is stored in the "data ID" field.
-The "measurement date" field stores information on the execution date and time included in the update request data.
-The "wearable" field stores wearable information included in the update request data.

After step S400, check-in (S401) is performed.
Step S401 differs from step S201 in FIG. 14 in that the check-in completion request data includes the following information.
・ User ID of the user who answered
-Check-in time information indicating the execution date and time of step S401-Information input to field objects F200a to F200e, F201a to F201b, F202a to F202g, and F203a to F203b

After step S401, check out (S402) is performed.
Step S402 differs from step S202 in FIG. 14 in that the checkout completion request data includes the following information.
・ User ID of the user who answered
-Check-in time information indicating the execution date and time of step S402-Information input to field objects F204a to F204f and F205a to F205f

(4-5) Emotion Visualization Processing The emotion visualization (OP5) processing of the present embodiment will be described. FIG. 21 is a flowchart of the emotion visualization process of FIG. FIG. 22 is a diagram showing an example of a screen displayed in the information processing of FIG.

As shown in FIG. 21, the target user information is determined (S500).
Specifically, the processor 12 of the client device 10 acquires the user ID (hereinafter, referred to as “target user ID”) of the target user who is the target of the emotion to be visualized.
The processor 12 transmits the reference request data to the server 30. The reference re-ext data includes the target user ID.

  As a first example of step S500, the processor 32 of the server 30 refers to the user information database (FIG. 7), and obtains information of a “user attribute” field of a record associated with the target user ID included in the reference request data; The information in the “fixed subjectivity” field and the information in the “fixed indirect objective” field are specified.

  As a second example of Step S500, the processor 32 of the server 30 specifies information of each field of the user flow subjective information database (FIG. 8) associated with the target user ID included in the reference request data.

  As a third example of Step S500, the processor 32 of the server 30 specifies information of each field of the user flow indirect objective information database (FIG. 9) associated with the target user ID included in the reference request data.

  As a fourth example of step S500, the processor 32 of the server 30 combines some or all of the first to third examples.

After step S500, a reference subject ID is determined (S501).
In step S501, at least one reference subject ID is specified by the following method.

As a first example of step S501, the processor 32 of the server 30 refers to the subject information database (FIG. 4) and associates the subject associated with the same or similar information as the information of the “user attribute” field specified in step S500. An ID (that is, a subject whose user attribute is the same as or similar to the target user) is specified. For example, the processor 32 specifies the following subject ID.
・ Subject ID associated with the same gender as the gender associated with the target user ID
A subject ID associated with an age whose difference from the age associated with the target user ID is within a predetermined range
・ Subject ID associated with the same occupation as the occupation associated with the target user ID

As a second example of step S501, the processor 32 of the server 30 refers to the subject information database (FIG. 4) and associates it with the subject fixation subjective information that is the same as or similar to the information of the “fixation subjectivity” field specified in step S500. The identified subject ID (that is, the subject who is the same as or similar to the target user with respect to the fixed subjective information) is specified. For example, the processor 32 specifies the following subject ID.
The subject ID associated with the same subject fixation subjective information as the information in the “fixation subjectivity” field associated with the target user ID
A subject ID associated with subject fixation subjective information whose difference from the information in the “fixation subjectivity” field associated with the target user ID is within a predetermined range;

As a third example of step S501, the processor 32 of the server 30 refers to the subject information database (FIG. 4), and has the same or similar subject fixed indirect objective information as the information in the “fixed indirect objective” field specified in step S500. (That is, the same or similar subject as the target user with respect to the fixed indirect objective information). For example, the processor 32 specifies the following subject ID.
The subject ID associated with the same subject fixed indirect objective information as the information in the “sticky indirect objective” field associated with the target user ID
The subject ID associated with the subject fixed indirect objective information whose difference from the information in the “fixed indirect objective” field associated with the target user ID is within a predetermined range

As a fourth example of step S501, the processor 32 of the server 30 refers to the subject flow subjective information database (FIG. 5), and has the same or similar information as the information of the user flow subjective information database (FIG. 8) specified in step S500. The subject ID associated with the subject flow subjective information (that is, the subject who is the same as or similar to the target user for the flow subject information) is specified. For example, the processor 32 specifies the following subject ID.
-Of the information in the user flow subjective information database associated with the target user ID, the information in the "check-in date" field is associated with the subject's flow subjective information that is the same as or similar to the information in the "check-in response" field of the most recent record Subject ID
-Of the information in the user flow subjective information database associated with the target user ID, the information in the "checkout date" field is associated with the subject's flow subjective information that is the same as or similar to the information in the "checkout answer" field of the most recent record Subject ID
-In the information of the user flow subjective information database associated with the target user ID, the information of the "check-in date" field is the information of the "check-in response" field of a plurality of records corresponding to a predetermined period (for example, the past month). Subject ID associated with subject flow subjective information having the same or similar tendency as the information tendency
-In the information of the user flow subjective information database associated with the target user ID, the information of the "checkout date" field is the information of the "checkout answer" field of a plurality of records corresponding to a predetermined period (for example, the past month). Subject ID associated with subject flow subjective information having the same or similar tendency as the information tendency

As a fifth example of step S501, the processor 32 of the server 30 refers to the subject flow indirect objective information database (FIG. 6), and is the same as or similar to the information in the user flow indirect objective information database (FIG. 9) specified in step S500. The subject ID associated with the approximate subject flow indirect objective information (that is, the subject who is the same as or similar to the target user in the flow indirect objective information) is specified. For example, the processor 32 specifies the following subject ID.
-The subject ID associated with the subject flow indirect objective information in which the information in the "measurement date" field associated with the target user ID is the same as or similar to the information in the "wearable" field of the newest record
A subject flow in which the information in the “measurement date and time” field associated with the target user ID has the same or similar tendency as the information in the “wearable” field of a plurality of records corresponding to a predetermined period (for example, the past month). Subject ID associated with indirect objective information

  A sixth example of Step S501 is a combination of at least two of the first to fifth examples. As the number of combinations increases, the subject ID of a subject having a high degree of similarity to the target user is specified as the reference subject ID. This improves the estimation accuracy in estimating user emotion information (S502) described later.

  After step S501, estimation of user emotion information (S502) is performed.

Specifically, the processor 32 of the server 30 refers to the subject information database (FIG. 4), and refers to the information (that is, the brain) of the “direct objective” field associated with at least one reference subject ID specified in step S501. Activity parameters and hemodynamic response parameters).
The processor 32 specifies information in the “check-in response” field and the “check-out response” field of the user flow subjective information (FIG. 8) associated with the target user ID (that is, the flow subjective information of the target user). .
The processor 43 specifies the information of the “wearable” field of the user flow indirect objective information (FIG. 9) associated with the target user ID (that is, the flow indirect objective information of the target user).
In the storage device 31, an inference model of a brain activity parameter, a blood flow dynamic response parameter, flow subjective information, flow indirect objective information, and an emotional condition is constructed. The emotional condition is a level for each type of emotional neuroscience determined based on various factors including the anatomical position of the brain, the expression of a transmitter, a receptor, and time.
The processor 32 uses the inference model stored in the storage device 31 to determine the emotional condition of the target user from the brain activity parameter, the blood flow dynamic response parameter, the flow subjective information, and the flow indirect objective information of the target user. .
The type of emotion is, for example, at least one of the following.
・ Pleasure ・ Anxiety ・ Tension

The processor 32 of the server 30 refers to the user information database (FIG. 7) and refers to the information of the “fixed subjectivity” field associated with the target user ID (that is, the fixed fixation subjective information of the target user) and the “fixed indirect objective” field. (That is, the fixed indirect objective information of the target user) is specified.
The processor 32 specifies the information of the “wearable” field of the user flow indirect objective information database (FIG. 9) associated with the target user ID (that is, the flow indirect objective information of the target user).
In the storage device 31, an inference model relating to the correlation between the sticky subjective information, the sticky indirect objective information, and the flow indirect objective information and the emotion tendency is constructed.
The processor 32 uses the inference model stored in the storage device 31 to determine the emotional tendency of the target user from the fixed subjective information, the fixed indirect objective information, and the fluid indirect objective information of the target user.
The emotion tendency includes, for example, at least one of the following.
・ Sympathetic superiority to parasympathetic nerve ・ Parasympathetic superiority to sympathetic nerve ・ Stress responsiveness ・ Stress tolerance ・ Amygdala activity ・ Salience network utilization

The processor 32 of the server 30 includes the subject information database (FIG. 4), the subject flow subjective information database (FIG. 5), the subject flow indirect objective information database (FIG. 6), and the user flow subjective information associated with the target user ID. The emotion factor is determined by comparing (FIG. 8) and the user flow indirect objective information database (FIG. 9) with the emotion condition and the emotion tendency.
The emotion factor includes, for example, at least one of the following.
-User behavior (eg, exercise, work, study, or reading)
-External factors of the user (for example, weather or temperature and humidity)
・ Interpersonal factors of the user (for example, self / other, affinity, relationship, level of likes / dislikes, or contact frequency)
-User objective factors (for example, closeness, relationship, likes and dislikes, contact frequency, etc.)
-Environmental factors of the user (for example, location, weather, humidity, carbon dioxide concentration, atmospheric pressure, or rainfall)
・ Condition factors of the user (for example, sleeping state or eating state)

After step S502, presentation of user emotion information (S503) is performed.
Specifically, the processor 32 of the server 30 transmits the user emotion information estimated in step S502 to the client device 10.
Processor 12 of client device 10 displays screen P500 (FIG. 22) on the display based on the user emotion information.

Screen P500 includes display objects A500a to A500c.
The emotional condition is displayed on the display object A500a. The emotion condition is, for example, a level for each type of emotion (for example, enjoyment, anxiety, and tension).
An emotional tendency is displayed on the display object A500b.
An emotion factor is displayed on the display object A500c.

  According to the present embodiment, using an inference model constructed based on direct objective information, subject subjective information, and subject indirect objective information associated with a subject who has undergone an MRI examination, the user is associated with a user who has not undergone an MRI examination. User emotion information is estimated from the obtained user subjective information and user indirect objective information. As a result, it is possible to improve the accuracy of estimating the emotion of the user who has not undergone the MRI examination.

(5) Modification A modification of the present embodiment will be described.

(5-1) User emotion information database A user emotion information database of a modification will be described. FIG. 23 is a diagram illustrating a data structure of a user emotion information database according to a modification.

The user emotion information database of FIG. 23 stores the user emotion information estimated in step S502 (FIG. 21).
The user emotion information database includes a “data ID” field, a “judgment date” field, and an “emotion” field.
The user emotion information database is associated with the user ID.

  The “data ID” field stores a data ID for identifying a record in the user emotion information database.

  Information regarding the date and time of execution of step S502 is stored in the “determination date and time” field.

The processing result of step S502 is stored in the “emotion” field.
The “emotion” field includes an “emotional condition” field, an “emotional tendency” field, and an “emotional factor” field.

  In the “emotional condition” field, information on the emotional condition is stored.

  The “emotional tendency” field stores information about the emotional tendency.

  The information about the emotion factor is stored in the “emotional factor” field.

(5-2) Emotion Visualization Processing The emotion visualization processing (OP5) of the modified example will be described. FIG. 24 is a flowchart of a process of visualizing emotions according to a modification.

  As shown in FIG. 24, the emotion visualization process (OP5) of the modification is different from the emotion visualization process (FIG. 21) of the present embodiment in that it includes determination of a reference user ID (S510).

After step S501, a reference user ID is determined (S510).
Specifically, the processor 32 of the server 30 determines the reference user ID of the reference user based on the target user information determined in step S500.

As a first example of step S510, the information of the “user attribute” field associated with the target user ID is specified with reference to the user information database (FIG. 7).
The processor 32 refers to the user information database, refers to the “user attribute” field associated with the asymmetric user ID of a user other than the target user, and obtains information of the “user attribute” field associated with the target user ID. Identify records that contain the same or similar information.
The processor 32 determines the user ID of the specified record as the reference user ID.

As a second example of step S510, referring to the user information database (FIG. 7), the information of the "fixed subjectivity" field associated with the target user ID is specified.
The processor 32 refers to the user information database, refers to the “fixed subjectivity” field associated with the non-target user ID, and obtains information having a predetermined relationship with the information in the “fixed subjectivity” field associated with the target user ID. Identify records that contain.
The predetermined relationship is any of the following relationships.
A relationship in which the information in the “sticky subject” field associated with the target user ID is the same as the information in the “sticky subjective” field associated with the asymmetric user ID; and the “sticky subjective” field associated with the target user ID. And the information in the “sticky subjectivity” field associated with the asymmetric user ID is a predetermined combination (for example, a combination that has good compatibility or a combination that has poor compatibility). The processor 32 specifies The user ID of the record is determined as a reference user ID.

As a third example of step S510, the information of the “fixed indirect objective” field associated with the target user ID is specified with reference to the user information database (FIG. 7).
The processor 32 refers to the user information database, refers to the “sticky indirect objective” field associated with the non-target user ID, and is the same as or similar to the information in the “sticky indirect objective” field associated with the target user ID. Identify the record that contains the information.
The processor 32 determines the user ID of the specified record as the reference user ID.

  As a fourth example of step S510, referring to the user flow subjective information database (FIG. 8), the user flow subjective information that is the same as or similar to the user flow subjective information associated with the target user ID is selected from the non-target user IDs. Is identified as the reference user ID.

  As a fifth example of step S510, referring to the user flow indirect objective information database (FIG. 9), from among the non-target user IDs, the same or similar user flow indirect information as the user flow indirect information associated with the target user ID is used. The asymmetric user ID associated with the information is specified as a reference user ID.

After step S510, estimation of user emotion information (S502) is performed.
Specifically, the storage device 31 is further configured with at least one inference model described below.
・ Inference model of user attribute information and user emotion information ・ Inference model of user fixation subjective information and user emotion information ・ Inference model of user fixation indirect objective information and user emotion information ・ User inference model of user flow subjective information and user emotion information ・User inference model of user flow indirect objective information and user emotion information

  The processor 32 of the server 30 estimates user emotion information using the inference model stored in the storage device 31 and the user inference model.

  According to the modification, the user emotion information is estimated with reference to not only the information of the subject but also the information of the reference user other than the target user. Thereby, the estimation accuracy of the emotion of the target user can be improved.

(6) Summary of this Embodiment This embodiment will be summarized.

A first aspect of the present embodiment is:
An information processing device (e.g., server 30) for estimating a user's emotion,
Means for obtaining direct objective information (for example, information that can be obtained by MRI or fMRI) directly related to the objective emotion of the subject (for example, the processor 32 executing step S103);
Means for acquiring at least one of subject subjective information related to the subject's subjective emotion and subject indirect objective information indirectly related to the subject's objective emotion (for example, steps S101 to S102 and S200 to S202) A processor 32) that performs at least one of the following:
Means for acquiring at least one of user subjective information related to the user's subjective emotion and user indirect objective information indirectly related to the user's objective emotion (for example, steps S300 to S202 and S400 to S402) A processor 32) that performs at least one of the following:
Means for estimating user emotion information related to the user's emotion with reference to the direct objective information, at least one of the subject subjective information and the subject indirect objective information, and at least one of the user subjective information and the user indirect objective information ( For example, the processor 32) that executes Step S502 is provided.
It is an information processing device.

  According to the first aspect, it is possible to guess direct objective information directly reflecting a reaction of the brain that controls the emotion of the user who has not received the objective information directly. As a result, the emotion of the user can be estimated with high accuracy.

A second aspect of the present embodiment is:
Including means for presenting the estimated user emotion information (for example, the processor 32 executing step S503);
It is an information processing device.

  According to the second aspect, the emotion of the user can be visualized to the user.

A third aspect of the present embodiment is:
Subjective information includes sticky subjective information related to the subjective emotion fixed to the subject or the user, and fluid subjective information related to the subjective emotion that is fluid to the subject or the user,
It is an information processing device.

  According to the third aspect, the subjective information to be referred to for estimating the emotion is handled separately as information fixed to the subject or the user and information fluid to the subject or the user. Thereby, the accuracy of emotion estimation can be further improved.

A fourth aspect of the present embodiment is:
Indirect objective information includes sticky objective information related to objective feelings fixed to the subject or the user, and fluid objective information related to objective feelings fluid to the subject or the user.
It is an information processing device.

  According to the fourth aspect, the indirect objective information referred to for estimating the emotion is handled separately as information fixed to the subject or the user and information fluid to the subject or the user. Thereby, the accuracy of emotion estimation can be further improved.

A fifth aspect of the present embodiment is:
The direct objective information is at least one of MRI (Magnetic Resonance Imaging) test, PET (Positron Emission Tomography) test, fNIRS (functional Near Infrared Spectroscopy) test, EEG (Electroencephalogram) test, and MEG (Magnetoencephalography) test. Information obtained by
It is an information processing device.

A sixth aspect of the present embodiment is:
The estimating means uses the inference model constituted by at least one of the subject subjective information and the subject indirect objective information and the direct objective information, and extracts the user emotion information from at least one of the user subjective information and the user indirect objective information. presume,
It is an information processing device.

A seventh aspect of the present embodiment is:
A means for determining a reference user from among a plurality of asymmetric users by referring to at least one of user subjective information and user indirect objective information of an asymmetric user different from a target user whose emotion is to be estimated,
The estimating means includes at least one of subject subjective information and subject indirect objective information; at least one of user subjective information and user indirect objective information; at least one of user subjective information and user indirect objective information of a reference user; With reference to the user emotion information, the user emotion information of the target user is estimated,
It is an information processing device.

  According to the seventh aspect, the user emotion information is estimated with reference to not only the information of the subject but also the information of the reference user other than the target user. Thereby, the estimation accuracy of the emotion of the target user can be improved.

An eighth aspect of the present embodiment is:
This is a program for causing a computer (for example, the processor 32) to function as each of the above units.

(7) Other Modification Examples Other modification examples will be described.

  The storage device 11 may be connected to the client device 10 via the network NW. The storage device 31 may be connected to the server 30 via the network NW.

  Each step of the above information processing can be executed by either the client device 10 or the server 30.

Steps S101 and S301 can be omitted. In this case, emotion visualization (OP5) is performed using information other than the fixed subjective information.
Steps S102 and S302 can be omitted. In this case, emotion visualization (OP5) is performed.
Steps S200 and S400 can be omitted. In this case, emotion visualization (OP5) is performed.
Steps S201 and S202, and S401 and S402 can be omitted. In this case, emotion visualization (OP5) is performed.

The flow subjective information may be a sentence created by the subject or the user. The flow subjective information includes, for example, at least one of the following.
・ Diary ・ Blog

The method for directly measuring objective information is not limited to MRI. Direct objective information can be obtained by at least one of the following measurement methods.
-PET (Positron Emission Tomography) inspection-fNIRS (functional Near Infrared Spectroscopy) inspection-EEG (Electroencephalogram) inspection-MEG (Magnetoencephalography) inspection

In steps S201 to S202 and S401 to S402, not only the answer is received in the Q & A format, but also the option object of the subjective information is presented, and the subjective information corresponding to the operated option object is received as the fluid subjective information. Is also good.
The option object includes, for example, at least one of the following.
-Icons for each type of emotion-Image objects of colors representing emotions

In step S102 (FIG. 11), sound may be used instead of an image.
For example, the client device 10 includes a stereo speaker (for example, a left speaker and a right speaker disposed on the left and right sides of the operator).
The processor 12 reproduces a subliminal sound from each of the left speaker and the right speaker.
The operator hears the subliminal sound reproduced from each of the left speaker and the right speaker, and gives a user instruction to the client device 10 indicating which of the speakers reproduces the subliminal sound.
The processor 12 transmits the update request data including the user instruction (that is, an instruction indicating a speaker that can easily hear the subliminal sound) to the server 30.

  As mentioned above, although embodiment of this invention was described in detail, the scope of this invention is not limited to said embodiment. Further, the above-described embodiment can be variously improved or changed without departing from the gist of the present invention. Further, the above embodiments and modified examples can be combined.

1: Information processing system 10: Client device 11: Storage device 12: Processor 13: Input / output interface 14: Communication interface 30: Server 31: Storage device 32: Processor 33: Input / output interface 34: Communication interface

Claims (8)

An information processing apparatus for estimating a user's emotion,
Means for obtaining direct objective information directly related to the subject's objective emotions,
Subject subjective information related to the subject's subjective emotion, and means for acquiring at least one of the test indirect objective information indirectly related to the subject's objective emotion,
The user subjective information related to the user's subjective emotion, and means for acquiring at least one of user indirect objective information indirectly related to the user's objective emotion,
A user emotion related to the emotion of the user with reference to the direct objective information, at least one of the subject subjective information and the subject indirect objective information, and at least one of the user subjective information and the user indirect objective information; Comprising means for estimating information;
Information processing device. Comprising means for presenting the estimated user emotion information,
The information processing device according to claim 1. The subjective information includes sticky subjective information related to the subjective emotion fixed to the subject or the user, and fluid subjective information related to a subjective emotion that is fluid for the subject or the user,
The information processing device according to claim 1. The indirect objective information, fixed objective information related to the objective emotion fixed to the subject or the user, and fluid objective information related to the objective emotion that is fluid for the subject or the user,
The information processing device according to claim 1. The direct objective information is at least one of an MRI (Magnetic Resonance Imaging) test, a PET (Positron Emission Tomography) test, a fNIRS (functional Near Infrared Spectroscopy) test, an EEG (Electroencephalogram) test, and a MEG (Magnetoencephalography) test. Information obtained by one,
The information processing apparatus according to claim 1. The estimating means uses at least one of the subject subjective information and the subject indirect objective information and the direct objective information, and uses an inference model constituted by the at least one of the user subjective information and the user indirect objective information. Estimating the user emotion information from the
The information processing device according to claim 1. A means for determining a reference user from among a plurality of asymmetric users by referring to at least one of user subjective information and user indirect objective information of an asymmetric user different from a target user whose emotion is to be estimated,
The estimating means is at least one of the subject subjective information and the subject indirect objective information, at least one of the user subjective information and the user indirect objective information, and at least one of the user subjective information and the user indirect objective information of the reference user. Estimating the user emotion information of the target user with reference to one and the user emotion information of the reference user,
The information processing device according to claim 1.   A program for causing a computer to function as each unit according to claim 1.