JP2021079041A - Empathy degree evaluation system, empathy degree evaluation method and program - Google Patents

Abstract

To evaluate an empathy degree of a subject to a visual stimulation.SOLUTION: A brain wave extraction unit 6 uses information at a plurality of eye retention start points in eye movement of a subject when viewing a visual stimulation which is presented related to a presented key word, and extracts a plurality of retention time brain waves corresponding to the plurality of eye retention start points one-to-one from the brain waves of the subject. A region dividing unit 7 divides the visual stimulation into a plurality of divided regions. A region identification unit 8 identifies a corresponding divided region out of the plurality of divided regions for the plurality of retention time brain waves on the basis of viewpoint position information of the subject on the visual stimulation and the plurality of retention time brain waves. An eye-retention-related potential calculation unit 9 calculates an eye-retention-time-related potential on the basis of a corresponding one group of retention time brain waves out of the plurality of retention time brain waves, for each divided region. An evaluation unit 10 evaluates an empathy degree to whole visual stimulation of the subject related to the key word, on the basis of an amplitude value of the plurality of eye-retention-related potentials corresponding to the plurality of divided regions one-to-one.SELECTED DRAWING: Figure 1

Description

The present disclosure relates generally to empathy evaluation systems, empathy evaluation methods and programs, and more specifically to empathy evaluation systems, empathy evaluation methods and programs that evaluate the subject's empathy to visual stimuli.

Patent Document 1 proposes an analysis support device that supports analysis of measurement results by a physiological index and a viewpoint tracking device obtained from physiological measurement for a subject viewing a content.

The analysis support device described in Patent Document 1 includes an emotional state calculating means, a storage means, an associating means, and a display means.

The storage means are the physiological index of the subject obtained from the physiological measurement, the viewpoint coordinate data of the subject measured by the viewpoint tracking device, the scene image or snapshot image related to the content captured by the viewpoint tracking device, and the physiological measurement. The time difference between the measurement start time of the physiological index and the measurement start time by the viewpoint tracking device is stored. The emotional state calculation means calculates an index of the emotional state of the subject based on the physiological index. The associating means associates the subject's viewpoint coordinates with the scene image or snapshot image related to the content based on the subject's viewpoint coordinate data measured by the viewpoint tracking device. The display means synchronizes the time series based on the time difference, and displays the scene image or snapshot image to which the viewpoint coordinates of the subject are associated and the index of the emotional state on the same screen.

In Patent Document 1, as examples of an index of emotional state, "interest / concentration" (degree of interest / interest and concentration when browsing an object) and "stress / relaxed state" (stress / when browsing an object / Relaxed state), "pleasant / unpleasant" (pleasant / unpleasant state when browsing the object), "awakening degree" (awakening degree when browsing the object), "basic emotion" (emotions and sorrows when browsing the object, etc.) Basic emotional state), "autonomic nerve activity", etc. are listed.

Japanese Unexamined Patent Publication No. 2015-188649

In the analysis support device disclosed in Patent Document 1, there is a time difference between the time when the subject sees the content and the time when the physiological reaction appears. Therefore, simply synchronizing the time series based on the time difference is the place where the subject is looking. And the emotional state may be out of alignment. Therefore, even if the analysis support device disclosed in Patent Document 1 is used, it is difficult to evaluate the sympathy of the subject with respect to the visual stimulus.

An object of the present disclosure is to provide an empathy evaluation system, an empathy evaluation method and a program capable of evaluating the empathy of a subject with respect to a visual stimulus.

The sympathy evaluation system according to one aspect of the present disclosure includes an electroencephalogram extraction unit, a region division unit, a region identification unit, an eye retention-related potential calculation unit, and an evaluation unit. The electroencephalogram extraction unit uses information at a plurality of eyeball retention start points in the eye movement of the subject when viewing the presented visual stimulus in relation to the presented keyword, and when viewing the visual stimulus. From the brain waves of the subject, a plurality of staying brain waves corresponding to one-to-one at the start time of the plurality of eyeball stays are extracted. The region division portion divides the visual stimulus into a plurality of division regions. Based on the viewpoint position information of the subject in the visual stimulus calculated from the eye movement and the plurality of staying electroencephalograms, the region specifying portion of the plurality of divided regions of the plurality of staying electroencephalograms. Identify the corresponding split area. The eyeball retention-related potential calculation unit calculates the eyeball retention-related potential based on the corresponding group of stationary electroencephalograms among the plurality of stationary electroencephalograms for each of the plurality of divided regions. The evaluation unit evaluates the sympathy of the subject with respect to the entire visual stimulus with respect to the keyword based on the amplitude values of the plurality of eye retention-related potentials corresponding to the plurality of divided regions on a one-to-one basis.

The sympathy evaluation method according to one aspect of the present disclosure includes a first step, a second step, a third step, a fourth step, a fifth step, a sixth step, a seventh step, and an eighth step. With steps. In the first step, a visual stimulus is further presented to the subject to whom the keyword is presented. In the second step, the brain waves and eye movements of the subject while watching the visual stimulus are measured. In the third step, a plurality of eyeball retention start time points are detected from the eyeball movement of the subject. In the fourth step, a plurality of staying electroencephalograms corresponding to one-to-one at the start time of the plurality of eyeball staying are extracted from the subject's brain waves. In the fifth step, a plurality of viewpoint position information corresponding to one-to-one at the start time of the plurality of eyeball stops is acquired. In the sixth step, the corresponding divided regions among the plurality of divided regions are specified for the plurality of stationary electroencephalograms based on the plurality of viewpoint position information and the plurality of stationary electroencephalograms. In the seventh step, the eyeball resting related potential is calculated based on the corresponding group of staying electroencephalograms among the plurality of staying electroencephalograms for each of the plurality of divided regions. In the eighth step, the sympathy of the subject with respect to the entire visual stimulus regarding the keyword is evaluated based on the amplitude values of the plurality of eye retention-related potentials corresponding to the plurality of divided regions on a one-to-one basis.

The program according to one aspect of the present disclosure is a program for causing a computer system to execute the sympathy evaluation method.

The empathy evaluation system, the empathy evaluation method, and the program according to the above aspect of the present disclosure can evaluate the empathy of a subject with respect to a visual stimulus.

FIG. 1 is a configuration diagram of an empathy evaluation system according to an embodiment. FIG. 2 is an image showing an example of a visual stimulus presented to a subject with respect to the same empathy evaluation system. FIG. 3 is an explanatory diagram of the relationship between the subject's eye movements, electroencephalograms, electroencephalograms at rest, and divided regions with respect to the same empathy evaluation system. FIG. 4 is an explanatory diagram of the trajectory of the subject's viewpoint in the visual stimulus presented to the subject with respect to the same empathy evaluation system. FIG. 5 is a waveform diagram showing an example of the waveform of the eyeball retention-related potential with respect to the same sympathy evaluation system. FIG. 6 is a flowchart illustrating the operation of the same sympathy evaluation system. FIG. 7 is an image showing another example of the visual stimulus presented to the subject with respect to the same empathy evaluation system.

(Embodiment)
Hereinafter, the sympathy evaluation system 1 according to the embodiment will be described with reference to FIGS. 1 to 7.

(1) Outline The sympathy evaluation system 1 according to the embodiment is a system for evaluating the sympathy of a subject with respect to a visual stimulus presented to the subject in relation to a keyword conveyed to the subject in advance. The keyword is conveyed to the subject, for example, as a question. Here, in the empathy evaluation system 1, for example, the subject's brain waves measured when the subject is looking at the visual stimulus 100 (for example, an image of the interior of a house as shown in FIG. 2) and the subject's visual stimulus. The sympathy for the entire visual stimulus 100 is evaluated by using the eye movement of the subject measured while looking at 100. The visual stimulus 100 does not have a gaze point for the subject to gaze at. In other words, the subject is informed that he / she may freely move his / her gaze while looking at the visual stimulus 100 before being presented with the visual stimulus 100. The keyword is a theme for determining the empathy of the subject, and is conveyed to the subject as a question, for example. When presenting an image of the interior of a house as shown in FIG. 2 as a visual stimulus 100, the keyword questions are, for example, "Do you like", "Do you dislike", "Do you want to live", and "Gorgeous". Do you feel it? " The sympathy is an index of the emotion of the subject, and is the degree of sensation that the subject who is transmitted the keyword sympathizes with the visual stimulus 100 when he / she sees the visual stimulus 100. The empathy is high if the subject feels like the visual stimulus 100 when he / she sees the visual stimulus 100 in the case of the question "Do you like" the keyword question, and is low if he / she does not like it.

(2) Overall configuration of the sympathy evaluation system As shown in FIG. 1, the sympathy evaluation system 1 includes an electroencephalogram extraction unit 6, a region division unit 7, a region identification unit 8, and an eyeball retention-related potential calculation unit 9. , And an evaluation unit 10. The electroencephalogram extraction unit 6 obtains information on a plurality of eyeball retention start points in the eye movement (see FIG. 3) of the subject when viewing the visual stimulus 100 (see FIG. 2) presented in relation to the presented keyword. It is used to extract a plurality of staying electroencephalograms (see FIG. 3) having a one-to-one correspondence with a plurality of eyeball staying start time points from the subject's brain waves. The region division unit 7 divides the visual stimulus 100 into a plurality of (for example, four) division regions A1 to A4 (see FIG. 4). The region identification unit 8 corresponds to the plurality of divided regions A1 to A4 for the plurality of stationary electroencephalograms based on the viewpoint position information of the subject in the visual stimulus 100 calculated from the eye movement and the plurality of stationary electroencephalograms. Identify the division area. The eye retention-related potential calculation unit 9 calculates the eye fixation-related potentials (Eye Fixation Rerated Potentials) for each of the plurality of divided regions A1 to A4 based on the corresponding group of stationary electroencephalograms among the plurality of stationary electroencephalograms. .. The evaluation unit 10 evaluates the sympathy of the subject with respect to the entire visual stimulus 100 regarding the keyword based on the amplitude values of the plurality of eye retention-related potentials corresponding to the plurality of divided regions A1 to A4 on a one-to-one basis.

Further, the empathy evaluation system 1 further includes a visual stimulus presentation unit 2, an electroencephalogram measurement unit 3, an eye movement measurement unit 4, and an eye retention detection unit 5. In the following, for convenience of explanation, first, the visual stimulus presentation unit 2, the electroencephalogram measurement unit 3, the eye movement measurement unit 4, and the eyeball retention detection unit 5 will be described.

The visual stimulus presentation unit 2 includes an image output device that presents an image as a visual stimulus to the subject. The image output device is, for example, a PC (Personal Computer) monitor, a TV (Television) monitor, a projector, a tablet, a smartphone, a head mount display, and a VR (Virtual Reality) headset.

The visual stimulus 100 is, for example, an image generated by capturing an image of the interior of a house with a camera, as shown in FIG. The house may be a detached house or an apartment house. The image is a still image. The visual stimulus 100 is not limited to a still image, but may be a video. The image here is a moving image.

The image is not limited to the captured image itself, and may be, for example, an image obtained by processing the captured image or a CG (Computer Graphics) image created based on the captured image. For example, the image based on the captured image may be an image obtained by performing brightness correction on the image captured by the camera. Further, the image based on the captured image may be an image obtained by superimposing a CG (Computer Graphics) image on the captured image.

The keyword presented to the subject before the visual stimulus 100 is presented by, for example, at least one of the voice of the respondent responding to the subject, the automatic voice of the robot, the paper medium, and the visual stimulus presenting unit 2.

The electroencephalogram measuring unit 3 measures the electroencephalogram of the subject. The electroencephalograph unit 3 is, for example, an electroencephalograph. The electroencephalogram measuring unit 3 detects an electroencephalogram signal by, for example, measuring potential changes in a plurality of electrodes mounted on the head of the subject.

The plurality of electrodes worn on the subject's head include a lead-out electrode, a reference electrode, and a ground electrode. For the plurality of electrodes, for example, based on the international 10-20 method, for example, the lead-out electrode is arranged on Pz (median parietal region), the reference electrode is arranged on A1 (right earlobe), and the ground electrode is arranged on the forehead. The lead-out electrode may be arranged at Cz (cranial apex) or Oz (occipital region) around Pz instead of Pz (median parietal region). The electroencephalograph may be a head-mounted electroencephalograph.

The eye movement measurement unit 4 measures the eye movement of the subject. The eye movement measuring unit 4 also measures the line of sight of the subject. The eye movement measuring unit 4 includes, for example, an eye tracker (stationary eye tracker, eyeglass-type eye tracker, etc.), a system combining an electro-oculography measuring instrument and a stationary eye tracker, and an electro-oculography measuring instrument and a glasses-type eye tracker. It is a combined system, etc.

The eyeball retention detection unit 5 detects the time when the eyeball retention starts (see FIG. 3) from the eyeball movement measured by the eyeball movement measurement unit 4. Here, the eyeball retention detection unit 5 detects a plurality of eyeball retention start time points in the eye movement of the subject when viewing the presented visual stimulus in relation to the presented keyword. FIG. 3 schematically shows an example of eye movements measured by the eye movement measurement unit 4. The unit of the vertical axis relating to the eye movement in FIG. 3 is, for example, "degree" when the eye movement measuring unit 4 is an eye tracker, and is, for example, "μV" when the eye movement measuring unit 4 includes an electro-oculography measuring device. .. In addition to eye movements, FIG. 3 also shows electroencephalograms and a plurality of electroencephalograms at rest. The start time of eye retention corresponds to the end of eye movement called a saccade, as illustrated in FIG. A saccade is an eye movement when a subject's line of sight moves rapidly. In the example of FIG. 3, eight eyeball retention start time points are detected. In the eyeball retention detection unit 5, the eyeball retention start time is, for example, an automatic detection model at the eyeball retention start time point constructed in advance by using machine learning from the eyeball movement (data) measured by the eyeball movement measurement unit 4. Can be detected using. Further, in the eyeball retention detection unit 5, for example, the eyeball movement (see FIG. 3) measured by the eyeball movement measurement unit 4 is displayed as an image on the display, and the user of the co-sensitivity evaluation system 1 visually displays the eyeball movement at the start of eye movement. By designating the position to be determined with a mouse or the like, the time point at which the eyeball stays started may be detected from the coordinates of the designated position.

The eye movement detection unit 5 not only detects the start time of eye retention from the eye movement measured by the eye movement measurement unit 4, but also calculates the viewpoint coordinates on the visual stimulus 100 from the line of sight formed by the eye movement measurement unit 4. To do. The viewpoint coordinates are the coordinates of the viewpoint at which the line of sight of the subject and the visual stimulus 100 (image) intersect. FIG. 4 schematically shows the locus of the viewpoint on the visual stimulus 100 with a chain line.

The region division unit 7 divides the visual stimulus 100 (see FIG. 2) presented by the visual stimulus presentation unit 2 into a plurality of division regions A1 to A4 (see FIG. 4). FIG. 4 is an example in which the region dividing portion 7 divides the visual stimulus 100 into four equal parts. In FIG. 4, the upper left divided area is the divided area A1, the upper right divided area is the divided area A2, the lower left divided area is the divided area A3, and the lower right divided area is the divided area A4. The region division portion 7 is not limited to the case where the entire visual stimulus 100 is divided into a plurality of division regions A1 to A4, and a part of the entire visual stimulus 100 may be divided into a plurality of division regions. May be separated rather than next to each other. The region division unit 7 may, for example, provide a plurality of visual stimuli 100 according to the user's designation of the co-sensitivity evaluation system 1 for the visual stimulus 100 presented on the visual stimulus presentation unit 2 or a PC monitor different from the visual stimulus presentation unit 2. It may be divided into the division areas of. For example, the user of the empathy evaluation system 1 divides into a plurality of division areas A1 to A4 by designating a range on the visual stimulus 100 with an input device (mouse or the like). The method of dividing the visual stimulus 100 into a plurality of divided regions in the region division portion 7 is not particularly limited. For example, the region division unit 7 may automatically divide the visual stimulus 100 into a plurality of division regions by using a region division algorithm created in advance. Further, the region dividing unit 7 identifies the shape and position of each of the plurality of objects included in the visual stimulus 100 by using an object recognition algorithm created in advance, and a plurality of divided regions corresponding to the plurality of objects on a one-to-one basis. It may be divided into. In the co-sensitivity evaluation system 1, when the visual stimulus presentation unit 2 directly presents the actual visual stimulus to the subject, the captured image generated by capturing the visual stimulus seen by the subject with an imaging device such as a camera is used. A visual stimulus is divided into a plurality of divided regions by the region dividing unit 7.

The electroencephalogram extraction unit 6 extracts a plurality of stationary electroencephalograms from the electroencephalograms measured by the electroencephalogram measurement unit 3. The electroencephalogram at the time of staying is obtained by extracting the electroencephalogram generated from the start point of the eyeball retention detected by the eyeball retention detection unit 5 as shown in FIG. The electroencephalogram extraction unit 6 acquires the electroencephalogram measured by the electroencephalogram measurement unit 3 as an electroencephalogram signal from the electroencephalogram measurement unit 3. The period for extracting the electroencephalogram at the time of staying is, for example, a period from -100 ms to 600 ms starting from the start point of eyeball staying. The time width of the period during which the electroencephalogram at rest is extracted is 700 ms, but is not limited to this.

The region identification unit 8 has a plurality of divided regions A1 to A4 for a plurality of stationary electroencephalograms based on the subject's viewpoint position information (viewpoint coordinates) in the visual stimulus 100 calculated from the eye movement and the plurality of stationary electroencephalograms. Specify the corresponding division area. In short, the region identification unit 8 identifies where in the visual stimulus 100 each of the plurality of stationary electroencephalograms extracted by the electroencephalogram extraction unit 6 is looking. Further, the area specifying unit 8 has a plurality of viewpoint coordinates by comparing the coordinates of the plurality of divided areas A1 to A4 divided by the area dividing unit 7 with the viewpoint coordinates calculated by the eyeball retention detection unit 5. It is specified which of the divided areas A1 to A4 belongs to. The region specifying unit 8 specifies which of the divided regions A1 to A4 each of the plurality of stationary electroencephalograms belongs to. In the example of FIG. 3, in order to make it easy to understand the divided region corresponding to a plurality of stationary electroencephalograms on a one-to-one basis, the sign of the divided region corresponding to each stationary electroencephalogram (A1) is below the waveform of each stationary electroencephalogram. Any one of ~ A4) is described. Regarding multiple (8) EEGs at rest, the first EEG at rest in the time series corresponds to the divided region A1, the second EEG corresponds to the divided region A2, and the third EEG at rest corresponds to the divided region A2. Corresponding to the divided region A2, the 4th stationary EEG corresponds to the divided region A4, the 5th stationary EEG corresponds to the divided region A1, the 6th stationary EEG corresponds to the divided region A3, 7 The eighth stationary electroencephalogram corresponds to the divided region A4, and the eighth stationary electroencephalogram corresponds to the divided region A2. In other words, in the example of FIG. 3, the divided region A1 corresponds to the first electroencephalogram at rest and the fifth electroencephalogram at rest, and the divided region A2 corresponds to the second electroencephalogram at rest and the third electroencephalogram at rest. Corresponds to the 8th stagnant electroencephalogram, the 6th stagnant electroencephalogram corresponds to the divided region A3, and the 4th stagnant electroencephalogram and the 7th stagnant brain wave correspond to the divided region A4. Corresponds to time brain waves.

The eyeball retention-related potential calculation unit 9 calculates the eyeball retention-related potential for each of the plurality of divided regions A1 to A4 based on the corresponding group of stationary electroencephalograms among the plurality of stationary electroencephalograms. More specifically, the eyeball retention-related potential calculation unit 9 is a group of stationary electroencephalograms identified as belonging to the divided region A1 in the region identification unit 8 (in FIG. 3, the first electroencephalogram at rest and the fifth electroencephalogram at rest). The electroencephalogram) is used to calculate the eye retention-related potential associated with the divided region A1. Further, the eyeball retention-related potential calculation unit 9 is a group of stationary electroencephalograms identified as belonging to the divided region A2 in the region identification unit 8 (in FIG. 3, the second electroencephalogram at rest and the third electroencephalogram at rest and 8). Using the second electroencephalogram at rest), the eye retention-related potential associated with the divided region A2 is calculated. Further, the eyeball retention-related potential calculation unit 9 uses a group of stationary electroencephalograms (the sixth electroencephalogram at rest in FIG. 3) identified as belonging to the divided region A3 in the region specifying unit 8 to the divided region A3. Calculate the associated eye retention-related potential. Further, the eyeball retention-related potential calculation unit 9 is a group of stationary electroencephalograms identified as belonging to the divided region A4 in the region identification unit 8 (in FIG. 4, the fourth electroencephalogram at rest, the seventh electroencephalogram at rest, and 8). Using the second electroencephalogram at rest), the eye retention-related potential associated with the divided region A4 is calculated. Here, the eyeball retention-related potential calculation unit 9 calculates the eyeball retention-related potential by performing the addition averaging of a group of brain waves at rest in each of the plurality of divided regions A to A4. In the eyeball retention-related potential calculation unit 9, the starting points of a group of stationary brain waves can be aligned on one time axis. The eye-holding-related potential calculation unit 9 determines the starting point of addition of a group of staying electroencephalograms. FIG. 5 shows an example of the additive average stationary electroencephalogram calculated by performing the additive average of a group of stationary electroencephalograms corresponding to the divided region A1. FIG. 5 shows the electroencephalogram at the time of additional average stopping in the section from −300 ms (before 300 ms) to 600 ms (after 600 ms) starting from the start point of eyeball retention. The predetermined time (for example, 600 ms), which is the time width for extracting the electroencephalogram at the time of staying in the above-mentioned electroencephalogram extraction unit 6, is defined as a range that always includes the lambda reaction (lambda wave) in the waveform of the eyeball retention potential. In FIG. 5, the amplitude value of the lambda reaction in the waveform of the eye retention-related potential is defined as H1.

The evaluation unit 10 evaluates the sympathy of the subject with respect to the entire visual stimulus 100 regarding the above keyword based on the amplitude values of the plurality of eyeball retention-related potentials corresponding to the plurality of divided regions A1 to A4 on a one-to-one basis. Here, the evaluation unit 10 has an eyeball retention-related potential amplitude value H1 in the division region A1, an eyeball-related potential amplitude value in the division region A2, and an eyeball-related potential in the division region A3 obtained by the eyeball retention-related potential calculation unit 9. The co-sensitivity to the entire visual stimulus 100 is calculated from the amplitude value of the eyeball and the eyeball-related potential of the divided region A4. The amplitude value of the eye retention-related potential is not limited to the amplitude value of the lambda reaction, and may be P300 of the event-related potential of the electroencephalogram.

As a method of calculating the sympathy, the evaluation unit 10 compares the amplitude value with the reference value for each of the plurality of divided regions A1 to A4, and when the amplitude value exceeds the reference value, the sympathy for the divided region is obtained. Is high, and if the amplitude value is lower than the reference value, it is judged that the sympathy for the divided region is low, and the sympathy for the entire visual stimulus 100 is calculated from the sympathy of each of the plurality of divided regions A1 to A4. .. Here, the evaluation unit 10 weights, for example, the amplitude values of a plurality of eyeball retention-related potentials corresponding to one-to-one in the plurality of divided regions A1 to A4, and sympathizes with the entire visual stimulus 100 of the subject regarding the above keywords. Calculate the degree. When weighting the amplitude value, for example, weighting is performed using logistic regression. The weighting may be weighting using a statistical regression model other than logistic regression.

When weighting the amplitude values of a plurality of eye retention-related potentials using logistic regression and calculating the sympathy for the entire visual stimulus 100 of the subject with respect to the above keyword, the evaluation unit 10 is, for example, the following (1). The value obtained by multiplying the probability p obtained by the formula by 100 is obtained as the sympathy of the percentage display.

In the above equation (1), x, x ₁ , x ₂ , ..., X _k are the amplitude values of the eyeball resting related potentials for each of the divided regions A1, A2, ..., Ak. The divided region Ak is the k-th divided region when the visual stimulus 100 is divided into k pieces, and is the divided region A4 when the visual stimulus 100 is divided into four as described above. a, b ₁ , b ₂ , ..., B _k are coefficients. Here, b ₁ , b ₂ , ..., B _k are coefficients to be multiplied by the amplitude value of the eyeball resting related potential for each of the divided regions A1, A2, ..., Ak.

The empathy evaluation system 1 includes a computer system. The main configuration of a computer system is a processor and memory as hardware. By executing the program recorded in the memory of the computer system by the processor, the eyeball retention detection unit 5, the brain wave extraction unit 6, the region division unit 7, the region identification unit 8, the eyeball retention-related potential calculation unit 9 and the evaluation in the present disclosure. The function as the unit 10 is realized. The program may be pre-recorded in the memory of the computer system, may be provided through a telecommunications line, and may be recorded on a non-temporary recording medium such as a memory card, optical disk, hard disk drive, etc. that can be read by the computer system. May be provided. A processor in a computer system is composed of one or more electronic circuits including a semiconductor integrated circuit (IC) or a large scale integrated circuit (LSI). The integrated circuit such as IC or LSI referred to here has a different name depending on the degree of integration, and includes an integrated circuit called a system LSI, VLSI (Very Large Scale Integration), or ULSI (Ultra Large Scale Integration). Further, an FPGA (Field-Programmable Gate Array) programmed after the LSI is manufactured, or a logical device capable of reconfiguring the junction relationship inside the LSI or reconfiguring the circuit partition inside the LSI should also be adopted as a processor. Can be done. A plurality of electronic circuits may be integrated on one chip, or may be distributed on a plurality of chips. The plurality of chips may be integrated in one device, or may be distributed in a plurality of devices. The computer system referred to here includes a microprocessor having one or more processors and one or more memories. Therefore, the microprocessor is also composed of one or a plurality of electronic circuits including a semiconductor integrated circuit or a large-scale integrated circuit.

Further, it is not an essential configuration for the sympathy evaluation system 1 that a plurality of functions in the sympathy evaluation system 1 are integrated in one housing, and the components of the sympathy evaluation system 1 are a plurality of housings. It may be provided in a distributed manner. Further, at least a part of the functions of the empathy evaluation system 1 may be realized by a cloud (cloud computing) or the like.

(3) Operation of the sympathy evaluation system Hereinafter, an operation example of the sympathy evaluation system 1 will be described with reference to the flowchart shown in FIG.

First, after the keyword is presented to the subject, the empathy evaluation system 1 presents the visual stimulus 100 in the visual stimulus presentation unit 2 (step S1).

The electroencephalogram measuring unit 3 measures the subject's brain wave when the visual stimulus 100 is presented to the subject (step S2), and the eye movement measuring unit measures the subject's eye movement when the visual stimulus 100 is presented to the subject. Measure according to 4 (step S3). Further, in the empathy evaluation system 1, the region division unit 7 divides the visual stimulus 100 presented by the visual stimulus presentation unit 2 into a plurality of division regions A1 to A4 (step S6).

In the sympathy evaluation system 1, after completing steps S2 and S3, the eyeball retention detection unit 5 detects the eyeball retention start time point and the viewpoint coordinates from the eyeball movement measured in step S3 (step S4).

In the empathy evaluation system 1, after step S4, the electroencephalogram extraction unit 6 extracts the electroencephalogram at rest from the electroencephalogram measured in step S2 (step S5).

In the co-sensitivity evaluation system 1, after step S6, the region identification unit 8 compares the viewpoint coordinates detected in step S4 with the coordinates of the divided regions A1 to A4 divided in step S6, and the subject divides which division. It is specified whether the areas A1 to A4 have been viewed (step S7). As a result, the resting electroencephalogram extracted in step S5 can be associated with the divided region seen by the subject in the visual stimulus 100.

In the co-sensitivity evaluation system 1, after the steps S5 and S7 are completed, the eyeball retention-related potential calculation unit 9 performs the addition averaging of a group of resting brain waves specified for each of the plurality of division regions A1 to A4. , The eyeball retention-related potential is calculated (step S8).

In the sympathy evaluation system 1, after step S8, the evaluation unit 10 calculates the sympathy for the visual stimulus 100 based on the eye retention-related potential obtained in step 8 (step S9).

(4) Empathy evaluation method The empathy evaluation method according to the embodiment is the first step, the second step, the third step, the fourth step, the fifth step, the sixth step, and the seventh step. And the eighth step.

In the first step, the visual stimulus 100 is further presented to the subject to whom the keyword is presented. The first step corresponds to step S1 described in the operation of the sympathy evaluation system 1.

In the second step, the brain waves and eye movements of the subject while viewing the visual stimulus 100 are measured. The second step corresponds to steps S2 and S3 described in the operation of the sympathy evaluation system 1.

In the third step, a plurality of eyeball retention start time points are detected from the eyeball movement of the subject. The third step corresponds to step S4 described in the operation of the sympathy evaluation system 1.

In the fourth step, a plurality of staying electroencephalograms corresponding to one-to-one at the start time of the plurality of eyeball staying are extracted from the subject's brain waves. The fourth step corresponds to step S5 described in the operation of the sympathy evaluation system 1.

In the fifth step, a plurality of viewpoint position information corresponding to one-to-one at the start time of the plurality of eyeball stops is acquired. The fifth step corresponds to step S4 described in the operation of the sympathy evaluation system 1.

In the sixth step, the corresponding divided regions among the plurality of divided regions A1 to A4 are specified for the plurality of stationary electroencephalograms based on the plurality of viewpoint position information and the plurality of stationary electroencephalograms. The sixth step corresponds to step S7 described in the operation of the sympathy evaluation system 1.

In the seventh step, the eyeball resting related potential is calculated for each of the plurality of divided regions A1 to A4 based on the corresponding group of staying electroencephalograms among the plurality of staying electroencephalograms. The seventh step corresponds to step S8 described in the operation of the sympathy evaluation system 1.

In the eighth step, the sympathy of the subject with respect to the entire visual stimulus 100 regarding the above keyword is evaluated based on the amplitude values of the plurality of eye retention-related potentials corresponding to the plurality of divided regions A1 to A4 on a one-to-one basis. The eighth step corresponds to step S9 described in the operation of the sympathy evaluation system 1.

(5) Effect In the empathy evaluation system 1 according to the embodiment, the evaluation unit 10 is a subject related to the above keyword based on the amplitude values of a plurality of eyeball retention-related potentials corresponding to a plurality of divided regions A1 to A4 on a one-to-one basis. Evaluate the sympathy for the entire visual stimulus 100. Therefore, the sympathy evaluation system 1 according to the embodiment can evaluate the sympathy of the subject with respect to the visual stimulus 100.

Further, in the sympathy evaluation system 1 according to the embodiment, it is possible to temporally synchronize the viewpoint coordinates of the subject when the subject is looking at the visual stimulus 100 and the eyeball retention-related potential generated by the eyeball retention of the subject. .. As a result, in the empathy evaluation system 1 according to the embodiment, the part (viewpoint coordinates) that the subject sees in the visual stimulus 100, the change in the brain activity that occurs in the subject's brain when the subject sees the visual stimulus 100, and so on. It is possible to evaluate the sympathy associated with. As a result, the sympathy evaluation system 1 according to the embodiment can evaluate the sympathy for the divided region including the portion that the subject was looking at in the visual stimulus 100.

The above embodiment is only one of various embodiments of the present disclosure, and various modifications can be made according to the design and the like as long as the object of the present disclosure can be achieved.

For example, in the empathy evaluation system 1, the visual stimulus presentation unit 2, the electroencephalogram measurement unit 3, the eye movement measurement unit 4, and the eye retention detection unit 5 are not essential components. Further, the sympathy evaluation system 1 may acquire the electroencephalogram (data) measured by the electroencephalogram measuring unit 3 from an external storage device. Further, in the sympathy evaluation system 1, the eye movement (data) measured by the eye movement measurement unit 4 may be acquired from an external storage device. Further, in the empathy evaluation system 1, the electroencephalogram at the time of staying detected by the eyeball staying detection unit 5 may be acquired from an external storage device.

Further, the image as the visual stimulus 100 presented by the visual stimulus presenting unit 2 is not limited to the image obtained by capturing the image from one place in the house as shown in FIG. 2 with a camera or the like, and is shown in FIG. 7, for example. As described above, it may be an image in which a plurality of images obtained by separately capturing a plurality of places in the house with a camera are arranged. In the example of FIG. 7, the visual stimulus 100 is divided into four divided regions A1 to A4 in the region division portion 7. In the example of FIG. 7, the divided area A1 corresponds to an image obtained by imaging the kitchen with a camera, the divided area A2 corresponds to an image obtained by capturing an image of the bathroom with a camera, and the divided area A3 corresponds to a toilet. Corresponds to the image obtained by capturing the image with the camera, and the divided area A4 corresponds to the image obtained by capturing the storage space with the camera.

Further, the visual stimulus presentation unit 2 is not limited to the image output device, and may directly present the actual visual stimulus to the subject. In this case, it is sufficient that the region dividing unit 7 can acquire the image obtained by capturing the actual visual stimulus with an imaging device such as a camera as the visual stimulus 100.

The evaluation unit 10 may calculate the sympathy based on the amplitude value of the eye retention-related potential when the subject is presented with a visual stimulus that serves as a baseline created in advance for evaluating the sympathy. Further, the evaluation unit 10 calculates the co-sensitivity based not only on the amplitude value of P300 of the event-related potential of the brain wave, but also on the amplitude value of P300 of the event-related potential of the brain wave and the latency of P300 at the event-related potential of the brain wave. You may.

Further, the evaluation unit 10 evaluates the sympathy of the subject with respect to the entire visual stimulus 100 regarding the above keyword based on the amplitude values of the plurality of eyeball retention-related potentials corresponding to the plurality of divided regions A1 to A4 on a one-to-one basis. , The sympathy may be evaluated by referring to a table in which the relationship between the amplitude value of the eye retention-related potential and the sympathy is stored in advance.

Further, the sympathy evaluation system 1 may further include an output unit that outputs the result of the evaluation unit 10. The output unit is, for example, a device capable of outputting at least one of an image and an audio. The image is output using a display device such as a liquid crystal display device or an OLED (Organic Light Emitting Diode) display. The sound is output using the speaker. The output unit presents the evaluation result of the evaluation unit 10 by at least one of an image and an sound.

The visual stimulus assumed by the co-sensitivity evaluation system 1 is not limited to the image of the house, and may be, for example, the appearance of a facility other than the house or an image inside the facility, the appearance of a moving body, or an image inside the moving body. “Non-residential facilities” are, for example, hospitals, office buildings, factories, commercial complexes, museums, amusement parks, theme parks, airports, railway stations, dome stadiums, and hotels. The moving body is, for example, a ship, a railroad vehicle, or an aircraft.

(5) Summary The following aspects are disclosed in the present specification.

The sympathy evaluation system (1) according to the first aspect includes an electroencephalogram extraction unit (6), a region division unit (7), a region identification unit (8), an eye retention-related potential calculation unit (9), and an eyeball retention-related potential calculation unit (9). It includes an evaluation unit (10). The electroencephalogram extraction unit (6) uses information at the start of multiple eye retention in the subject's eye movement when viewing the presented visual stimulus (100) in relation to the presented keyword, and uses the visual stimulus (100). ) Is extracted from the brain waves of the subject, which corresponds one-to-one with the start points of multiple eyeballs. The region division unit (7) divides the visual stimulus into a plurality of division regions (A1, A2, A3, A4). The region identification unit (8) has a plurality of divided regions (A1, A2, A3) for a plurality of stationary electroencephalograms based on the subject's viewpoint position information in the visual stimulus calculated from the eye movement and the plurality of stationary electroencephalograms. , A4), specify the corresponding division area. The eye-retention-related potential calculation unit (9) calculates the eye-retention-related potential for each of the plurality of division regions (A1, A2, A3, A4) based on the corresponding group of EEG-related potentials among the plurality of EEG. calculate. The evaluation unit (10) evaluates the sympathy of the subject with respect to the entire visual stimulus regarding the keyword based on the amplitude values of the plurality of eye retention-related potentials corresponding to the plurality of divided regions on a one-to-one basis.

In the sympathy evaluation system (1) according to the first aspect, it is possible to evaluate the sympathy of the subject with respect to the visual stimulus (100).

In the sympathy evaluation system (1) according to the second aspect, in the first aspect, the evaluation unit (10) uses keywords from the amplitude values of the plurality of eyeball retention-related potentials corresponding to the plurality of divided regions on a one-to-one basis. Calculate the empathy for the entire subject's visual stimulus (100) with respect to.

In the sympathy evaluation system (1) according to the second aspect, the sympathy can be quantified.

In the sympathy evaluation system (1) according to the third aspect, in the second aspect, the evaluation unit (10) with respect to the amplitude values of the plurality of eyeball retention-related potentials corresponding to the plurality of divided regions on a one-to-one basis. Weight and calculate the empathy for the subject's overall visual stimulus for the keyword.

In the sympathy evaluation system (1) according to the third aspect, it is possible to evaluate the sympathy for the entire visual stimulus with higher accuracy. It is also possible to evaluate whether the plurality of divided regions contribute to the empathy for the entire visual stimulus (100).

In the sympathy evaluation system (1) according to the fourth aspect, in the third aspect, the evaluation unit (10) has a plurality of eyeball retention-related potentials having a one-to-one correspondence with the plurality of divided regions (A1 to A4). The amplitude value is weighted using logistic regression and the sympathy for the entire visual stimulus (100) is calculated.

In the sympathy evaluation system (1) according to the fourth aspect, it is possible to more objectively evaluate the sympathy for the entire visual stimulus (100).

The empathy evaluation system (1) according to the fifth aspect further includes a visual stimulus presenting unit (2) in any one of the first to fourth aspects. The visual stimulus presentation unit (2) presents a visual stimulus to the subject.

In the sympathy evaluation system (1) according to the fifth aspect, it is not necessary to separately prepare the visual stimulus presenting unit (2) when presenting the visual stimulus (100) to the subject, and the usability is improved.

The sympathy evaluation system according to the sixth aspect further includes an electroencephalogram measuring unit (3) in any one of the first to fifth aspects. The electroencephalogram measuring unit (3) measures the electroencephalogram of the subject when the subject is watching the visual stimulus.

In the sympathy evaluation system (1) according to the sixth aspect, it is not necessary to separately prepare an electroencephalogram measuring unit (3), which improves usability.

The empathy evaluation system (1) according to the seventh aspect further includes an eye movement measurement unit (4) in any one of the first to sixth aspects. The eye movement measuring unit (4) measures the eye movement of the subject when the subject is looking at the visual stimulus (100).

In the sympathy evaluation system (1) according to the seventh aspect, it is not necessary to separately prepare the eye movement measurement unit (4), and the usability is improved.

The sympathy evaluation method according to the eighth aspect is the first step, the second step, the third step, the fourth step, the fifth step, the sixth step, the seventh step, and the eighth step. And. In the first step, a visual stimulus (100) is further presented to the subject to whom the keyword is presented. In the second step, the brain waves and eye movements of the subject while watching the visual stimulus (100) are measured. In the third step, a plurality of eyeball retention start time points are detected from the eyeball movement of the subject. In the fourth step, a plurality of staying electroencephalograms corresponding to one-to-one at the start time of the plurality of eyeball staying are extracted from the subject's brain waves. In the fifth step, a plurality of viewpoint position information corresponding to one-to-one at the start time of the plurality of eyeball stops is acquired. In the sixth step, the corresponding divided regions among the plurality of divided regions (A1 to A4) are specified for the plurality of stationary electroencephalograms based on the plurality of viewpoint position information and the plurality of stationary electroencephalograms. In the seventh step, the eyeball resting related potential is calculated for each of the plurality of divided regions (A1 to A4) based on the corresponding group of staying electroencephalograms among the plurality of staying electroencephalograms. In the eighth step, the sympathy for the entire visual stimulus (100) of the subject regarding the above keyword is evaluated based on the amplitude values of the plurality of eye retention-related potentials corresponding to one-to-one in the plurality of divided regions (A1 to A4). ..

In the sympathy evaluation method according to the eighth aspect, it is possible to evaluate the sympathy of the subject with respect to the visual stimulus (100).

The program according to the ninth aspect is a program for causing a computer system to execute the sympathy evaluation method of the eighth aspect.

In the program according to the ninth aspect, it is possible to evaluate the subject's empathy with respect to the visual stimulus (100).

1 Co-sensitivity evaluation system 2 Visual stimulus presentation unit 3 Brain wave measurement unit 4 Eye movement measurement unit 5 Eyeball retention detection unit 6 Brain wave extraction unit 7 Region division unit 8 Region identification unit 9 Eyeball retention-related potential calculation unit 10 Evaluation unit 100 Visual stimulation A1 ~ A4 division area H1 amplitude value

Claims (9)

Using the information at the start of multiple eye retention in the subject's eye movements when viewing the presented visual stimulus in relation to the presented keyword, the subject's brain waves when viewing the visual stimulus are described as described above. An electroencephalogram extraction unit that extracts a plurality of electroencephalograms at the time of staying, which have a one-to-one correspondence at the start of multiple eyeballs.
A region division portion that divides the visual stimulus into a plurality of division regions,
Based on the viewpoint position information of the subject in the visual stimulus calculated from the eye movement and the plurality of stationary electroencephalograms, the corresponding divided regions among the plurality of divided regions are specified for the plurality of stationary electroencephalograms. Area identification part to do,
For each of the plurality of divided regions, an eyeball retention-related potential calculation unit that calculates an eyeball retention-related potential based on a corresponding group of stationary electroencephalograms among the plurality of stationary electroencephalograms.
An evaluation unit for evaluating the sympathy of the subject with respect to the entire visual stimulus with respect to the keyword based on the amplitude values of the plurality of eye retention-related potentials corresponding to the plurality of divided regions on a one-to-one basis is provided.
Empathy evaluation system. The evaluation unit calculates the sympathy for the entire visual stimulus of the subject regarding the keyword from the amplitude values of the plurality of eye retention-related potentials corresponding to the plurality of divided regions on a one-to-one basis.
The empathy evaluation system according to claim 1. The evaluation unit weights the amplitude values of the plurality of eye retention-related potentials corresponding to the plurality of divided regions on a one-to-one basis, and calculates the sympathy of the subject for the entire visual stimulus with respect to the keyword.
The empathy evaluation system according to claim 2. The evaluation unit weights the amplitude values of the plurality of eye retention-related potentials corresponding to the plurality of divided regions on a one-to-one basis by using logistic regression, and calculates the sympathy for the entire visual stimulus. ,
The empathy evaluation system according to claim 3. A visual stimulus presenting unit that presents the visual stimulus to the subject is further provided.
The empathy evaluation system according to any one of claims 1 to 4. Further provided is an electroencephalogram measuring unit that measures the electroencephalogram of the subject when the subject is viewing the visual stimulus.
The empathy evaluation system according to any one of claims 1 to 5. Further comprising an eye movement measuring unit for measuring the eye movement of the subject when the subject is viewing the visual stimulus.
The empathy evaluation system according to any one of claims 1 to 6. The first step of presenting further visual stimuli to the subject to whom the keyword was presented, and
The second step of measuring each of the subject's brain waves and eye movements while viewing the visual stimulus, and
A third step of detecting a plurality of eyeball retention start time points from the eyeball movement of the subject, and
A fourth step of extracting a plurality of staying electroencephalograms corresponding to one-to-one at the start time of the plurality of eyeball staying from the brain waves of the subject.
The fifth step of acquiring a plurality of viewpoint position information corresponding to one-to-one at the start time of the plurality of eyeball stops, and
Based on the plurality of viewpoint position information and the plurality of stationary electroencephalograms, the sixth step of identifying the corresponding divided regions among the plurality of divided regions of the plurality of stationary electroencephalograms.
The seventh step of calculating the eyeball resting related potential based on the corresponding group of staying electroencephalograms among the plurality of staying electroencephalograms for each of the plurality of divided regions.
It comprises an eighth step of evaluating the subject's sympathy for the entire visual stimulus with respect to the keyword based on the amplitude values of the plurality of eye retention-related potentials corresponding to the plurality of divided regions on a one-to-one basis.
Empathy evaluation method. To cause a computer system to execute the empathy evaluation method according to claim 8.
program.

Images