WO2022025025A1

WO2022025025A1 - Emotion analysis system and emotion analysis device

Info

Publication number: WO2022025025A1
Application number: PCT/JP2021/027638
Authority: WO
Inventors: 渉三神谷
Original assignee: 株式会社I’mbesideyou
Priority date: 2020-07-31
Filing date: 2021-07-27
Publication date: 2022-02-03
Also published as: WO2022024355A1; JPWO2022025025A1

Abstract

The present invention comprises: a moving image acquisition unit 11 which acquires a moving image obtained for a subject; a vital reaction analysis unit 12 which analyzes, on the basis of the moving image acquired by the moving image acquisition unit 11, a change in vital reaction generated due to a change in emotion of the subject; and an emotion evaluation unit 13 which evaluates the degree of emotion adjusted according to how easily the same emotion occurs in the subject on the basis of the change in vital reaction for the subject analyzed by the vital reaction analysis unit 12, wherein, the change in vital reaction for the subject is analyzed on the basis of the moving image obtained for the subject, an emotion reaction absolute value, which is based on an evaluation criterion leveled between a plurality of subjects according to how easily the same emotion occurs in the subject, is calculated on the basis of the analyzed change in vital reaction, and thus the degree of emotion is enabled to be evaluated in a true sense for the subject, and the degrees of emotion between different subjects can be objectively contrasted.

Description

Emotion analysis system and emotion analysis device

The present invention relates to an emotion analysis system and an emotion analysis device, and more particularly to a system and an apparatus for analyzing emotions of a user participating in an online session.

Conventionally, there is known a technique of comparing a normal (expressionless) facial expression of a subject with a current facial expression to determine the degree of emotion of the subject (for example, Patent Documents 1 to 4). reference). Further, a technique for recognizing emotions in consideration of individuality from voice and images is also known (see, for example, Patent Document 5).

In Patent Document 4, a reference parameter generated based on the operator's calm voice is stored in advance, and the emotion parameter is compared with the emotion parameter generated based on the operator's voice acquired by the voice acquisition unit. It is disclosed to specify the degree of emotion. Further, in Patent Document 4, in view of the fact that there are individual differences in the appearance of the difference between the reference parameter and the emotion parameter, the average value of the emotion parameter is obtained for each operator, and the difference between the reference parameter and the average value is obtained. It is also disclosed that the emotional degree is relatively specified for each operator based on the ratio of the difference between the reference parameter and the emotional parameter.

Japanese Unexamined Patent Publication No. 2011-154665 Japanese Unexamined Patent Publication No. 2012-8949 Japanese Unexamined Patent Publication No. 2013-300 Japanese Unexamined Patent Publication No. 2015-141428 Japanese Unexamined Patent Publication No. 2001-83984

However, in the techniques described in Patent Documents 1 to 3, the degree of emotion is only evaluated by how much the current facial expression changes from the normal facial expression. Therefore, there is a problem that this evaluation information cannot be used as information for objectively comparing the degree of emotion between different subjects.

For example, it is assumed that the subject A is determined to have a current emotion X level of 3 based on the difference from his / her normal facial expression, and the subject B is also determined to have a current emotion X level of 3. .. However, the meaning of emotion X (the degree of emotion in the true sense) is different between the level 3 of the subject A who easily causes emotion X and the level 3 of the subject B who does not easily generate emotion X. Since the techniques described in Patent Documents 1 to 3 cannot evaluate the degree of emotion in the true sense of the word, it is not possible to objectively compare the degree of emotion between different subjects.

On the other hand, in the technique described in Patent Document 4, in view of individual differences in the magnitude of emotions, the average value of emotion parameters is obtained for each operator, and the average value is used for each operator. The degree of emotion is relatively specified. However, even in the technique described in Patent Document 4, the degree of emotion is not specified in consideration of the ease of occurrence of emotion for each operator, and the above-mentioned problem cannot be solved.

It is an object of the present invention to be able to objectively compare the degree of emotion between different subjects by making it possible to evaluate the degree of true emotions regarding the subject.

In order to solve the above-mentioned problems, in the emotion analysis system of the present invention, changes in the biological reaction of the subject are analyzed based on the moving images obtained for the subject, and based on the analyzed changes in the biological reaction, the changes in the biological reaction are analyzed. According to the evaluation criteria leveled among a plurality of subjects, the degree of emotions adjusted according to the likelihood of the same emotions being generated by the subjects is evaluated.

According to the present invention configured as described above, the degree of emotion is evaluated in consideration of the susceptibility to generate different emotions for each subject, and therefore the degree of true emotions regarding the subject is evaluated. This makes it possible to objectively compare the degree of emotion between different subjects.

It is a figure which shows the whole structure example of the emotion analysis system by this embodiment. It is a block diagram which shows the functional composition example of the emotion analysis apparatus by this embodiment.

Hereinafter, an embodiment of the present invention will be described with reference to the drawings. FIG. 1 is a diagram showing an overall configuration example of an emotion analysis system according to the present embodiment. As shown in FIG. 1, the emotion analysis system of the present embodiment is referred to as an emotion analysis device 100, a plurality of user terminals 200 _-1 , _200-2 , ... (Hereinafter, unless otherwise specified, it is simply referred to as a user terminal 200. ) And a session management server 300. The emotion analysis device 100, the user terminal 200, and the session management server 300 are connected via a communication network 500 such as the Internet or a mobile phone network.

The emotion analysis system of the present embodiment is caused by a change in emotions of the subject based on a moving image obtained about the subject (participant of the online session), for example, in an environment where an online session is performed by a plurality of participants. It is a system that analyzes the changes in the biological reaction that occur and evaluates the degree of emotion of the subject according to the evaluation criteria leveled among the plurality of subjects based on the analyzed changes in the biological reaction.

The online session is, for example, an online conference, an online class, an online chat, or the like, in which a plurality of user terminals 200 installed at a plurality of locations are connected to the emotion analysis device 100 and the session management server 300 via a communication network 500. A moving image can be exchanged between a plurality of user terminals 200 through an emotion analysis device 100 and a session management server 300. An application program (hereinafter referred to as a session application) necessary for exchanging moving images in an online session is installed in a plurality of user terminals 200.

The moving image handled in the online session includes a face image (actually, an image of a body part other than the face and a background image) and a voice of a user (participant in the online session) who uses the user terminal 200. .. The user's face image and voice are acquired by a camera and a microphone provided in the user terminal 200 or connected to the user terminal 200, and transmitted to the session management server 300. Then, the face image and voice of each user transmitted to the session management server 300 are acquired by the emotion analysis device 100, and are transmitted from the emotion analysis device 100 to the session application of each user terminal 200. The moving image transmitted from the user terminal 200 may be acquired by the emotion analysis device 100 and transferred from the emotion analysis device 100 to the session management server 300. Alternatively, the moving image may be transmitted from the user terminal 200 to both the emotion analysis device 100 and the session management server 300.

In addition, moving images include images such as materials shared and viewed by multiple users. The material image to be viewed by the user is transmitted from any user terminal 200 to the session management server 300. Then, the material image transmitted to the session management server 300 is acquired by the emotion analysis device 100, and is transmitted from the emotion analysis device 100 to the session application of each user terminal 200.

By the above operation, the face image or the document image of the plurality of users is displayed on the display in each of the plurality of user terminals 200, and the voices of the plurality of users are output from the speaker. Here, by the function of the session application installed in the user terminal 200, the face image and the material image can be switched to display only one of them on the display screen, or the display area can be divided into the face image and the material image. Can be displayed at the same time. Further, it is possible to display the image of one of a plurality of users on the full screen, or to display the image of a part or all of the users on a small screen.

It is also possible to switch the camera on / off and the microphone on / off by the function of the session application installed in the user terminal 200. For example, when the camera is turned off in the user terminal _200-1 , the face image taken by the camera of the user terminal _200-1 is transmitted to the session management server 300 and the emotion analysis device 100, but each of them is transmitted from the emotion analysis device 100. It is not transmitted to the user terminal 200. Similarly, when the microphone is turned off in the user terminal _200-1 , the sound collected by the microphone of the user terminal _200-1 is transmitted to the session management server 300 and the emotion analysis device 100, but is transmitted from the emotion analysis device 100. It is not transmitted to each user terminal 200.

FIG. 2 is a block diagram showing a functional configuration example of the emotion analysis device 100 according to the present embodiment. As shown in FIG. 2, the emotion analysis device 100 of the present embodiment includes a moving image acquisition unit 11, a biological reaction analysis unit 12, and an emotion evaluation unit 13 as functional configurations. Further, the emotion analysis device 100 of the present embodiment includes a moving image storage unit 101 as a storage medium.

Each of the above functional blocks 11 to 13 can be configured by any of hardware, DSP (Digital Signal Processor), and software. For example, when configured by software, each of the above functional blocks 11 to 13 is actually configured to include a computer CPU, RAM, ROM, etc., and is a program stored in a recording medium such as RAM, ROM, a hard disk, or a semiconductor memory. Is realized by the operation of.

The moving image acquisition unit 11 acquires moving images (face image, voice, material image) transmitted from each user terminal 200 during the online session from the session management server 300. The moving image acquisition unit 11 stores the moving image acquired from each user terminal 200 in the moving image storage unit 101 in association with information that can identify each user (for example, a user ID).

It does not matter whether the face image acquired from the session management server 300 is set to be displayed on the screen of each user terminal 200 (whether the camera is set to on or off). No. That is, the moving image acquisition unit 11 acquires the face image from the session management server 300, including the face image displayed on the display of each user terminal 200 and the face image not being displayed. Further, it does not matter whether the sound acquired from the session management server 300 is set to be output from the speaker of each user terminal 200 (whether the microphone is set to on or off). No. That is, the moving image acquisition unit 11 acquires audio from the session management server 300, including audio being output from the speaker of each user terminal 200 and audio being non-output.

The biological reaction analysis unit 12 outputs a moving image (whether or not it is a face image displayed on the screen of the user terminal 200, whether or not it is a face image displayed on the screen of the user terminal 200, from the speaker of the user terminal 200) acquired by the moving image acquisition unit 11 and stored in the moving image storage unit 101. Based on (whether or not it is the voice inside), we analyze the changes in biological reactions caused by changes in emotions for each of the multiple participants. In the present embodiment, the biological reaction analysis unit 12 separates the moving image acquired by the moving image acquisition unit 11 into a set of facial images (a collection of frame images) and voice, and analyzes changes in the biological reaction from each.

For example, the biological reaction analysis unit 12 analyzes the user's face image using the frame image separated from the moving image acquired by the moving image acquisition unit 11, and thereby at least one of the facial expression, the line of sight, the pulse, and the movement of the face. Analyze changes in biological reactions related to one. In addition, the biological reaction analysis unit 12 analyzes changes in the biological reaction regarding at least one of the user's speech content and voice quality by analyzing the voice separated from the moving image acquired by the moving image acquisition unit 11.

When a person's emotions change, it appears as changes in biological reactions such as facial expressions, eyes, pulse, facial movements, speech content, and voice quality. In this embodiment, changes in the user's emotions are analyzed by analyzing changes in the user's biological reaction. In the present embodiment, the biological reaction analysis unit 12 calculates a biological reaction index value that reflects the content of the change in the biological reaction by quantifying the change in the biological reaction according to a predetermined standard.

Analysis of changes in facial expressions is performed, for example, as follows. That is, the biological reaction analysis unit 12 identifies a facial region from the frame image for each frame image, and analyzes which facial expression element the facial expression corresponds to according to an image analysis model machine-learned in advance. do. Then, based on the analysis result, whether or not a facial expression change occurs between consecutive frame images, and if a facial expression change occurs, whether it is a positive facial expression change or a negative facial expression change, and how large it is. It analyzes whether the facial expression change is occurring, and calculates the facial expression change index value according to the analysis result.

Facial expression elements are, for example, neutral / calm / happy / surprised / sad / angry / fearful / disgust. be. Of these, joy and surprise are positive facial expression elements, and sadness, anger, fear, and disgust are negative facial expression elements.

The biological reaction analysis unit 12 calculates a total score of 100 for each of the plurality of facial expression elements for the facial expression in each frame image. For example, neutrality = 10, calmness = 10, joy = 30, surprise = 20, sadness = 10, anger = 10, fear = 5, disgust = 5, and so on. The corresponding score is calculated for each facial expression element. Then, for example, the facial expression element having the maximum score is determined as the facial expression in the frame image. In the following, the facial expression score (the maximum score calculated for a plurality of facial expression elements) determined for each frame image is referred to as a “facial expression score”.

The biological reaction analysis unit 12 determines between consecutive frame images depending on whether at least one of the facial expression element determined for each frame image and the facial expression score calculated for each frame image has changed from the previous frame. Determine if the facial expression has changed. Here, even if the biological reaction analysis unit 12 determines that the facial expression change has occurred when the score change amount from the previous frame is equal to or more than a predetermined threshold value when there is no change in the facial expression element of the maximum score. good. The magnitude of the facial expression change can be determined by the amount of change from the previous frame of the facial expression score.

In addition, the biological reaction analysis unit 12 causes a positive facial expression change when the facial expression score of the positive facial expression increases from the previous frame and when the negative facial expression of the previous frame changes to the positive facial expression of the current frame. It is determined that it is. On the other hand, the biological reaction analysis unit 12 causes a negative facial expression change when the facial expression score of the negative facial expression increases from the previous frame and when the positive facial expression of the previous frame changes to the negative facial expression of the current frame. It is determined that it is.

Further, the biological reaction analysis unit 12 uses the direction of facial expression change (positive → positive, positive → negative, negative → positive, negative → negative) and the magnitude of facial expression change as explanatory variables, and the facial expression change index value as the objective variable. The facial expression change index value is calculated using a predetermined function. In this function, for example, when the facial expression is reversed (positive → negative, negative → positive), the absolute value of the facial expression change index value is larger than when it is not reversed, and the greater the degree of facial expression change, the larger the facial expression change. A function that increases the absolute value of the index value, and when the facial expression changes in the positive direction (positive → positive, negative → positive), it becomes a positive value, and when the facial expression changes in the negative direction (positive → negative). , Negative → Negative) can be a function that has a negative value.

Here, an example of analyzing a facial expression change between consecutive frame images has been described, but the facial expression change may be analyzed every predetermined time interval (for example, every 500 milliseconds). This also applies to the analysis of the change in the line of sight, the analysis of the change in the pulse, and the analysis of the change in the movement of the face described below.

Analysis of changes in the line of sight is performed, for example, as follows. That is, the biological reaction analysis unit 12 identifies the eye region from the frame image for each frame image and analyzes the direction (line of sight) of both eyes. Then, the biological reaction analysis unit 12 calculates the line-of-sight change index value according to the analysis result of the line-of-sight change. For example, the biological reaction analysis unit 12 calculates the angle of the line of sight from the front for each frame image, and calculates the moving average or the moving variance between a plurality of frames of the angle as the line-of-sight change index value.

The biological reaction analysis unit 12 may analyze where the user is looking. The change in the line of sight is also related to the degree of concentration of the user. For example, it analyzes whether the speaker's face being displayed, the shared material being displayed, or the outside of the screen is being viewed. In addition, it may be possible to analyze whether the movement of the line of sight is large or small, and whether the movement is frequent or infrequent. Then, the biological reaction analysis unit 12 calculates the line-of-sight change index value according to the analysis result of the line-of-sight change.

For example, the biological reaction analysis unit 12 uses the place of viewing (speaker's face, shared material, outside the screen), the magnitude of eye movement, and the frequency of eye movement as explanatory variables, and the eye change index value. The line-of-sight change index value is calculated using a predetermined function with. This function is, for example, a function in which the absolute value of the line-of-sight change index value changes depending on the place of viewing, and the absolute value of the line-of-sight change index value increases as the movement of the line of sight increases and the frequency of the movement of the line of sight increases. It is possible.

Analysis of pulse changes is performed, for example, as follows. That is, for each frame image, the face area is specified from the frame image. Then, using a trained image analysis model that captures the numerical value of the face color information (G in RGB), the change in the G color on the face surface is analyzed. By arranging the results along the time axis, a waveform showing the change in color information is formed, and the pulse is specified from this waveform. When a person is nervous, the pulse becomes faster, and when he / she feels calm, the pulse becomes slower. The biological reaction analysis unit 12 calculates a pulse change index value according to the analysis result of the pulse change. For example, the biological reaction analysis unit 12 calculates the moving average or the moving variance of the pulse values specified for each frame as the pulse change index value.

Analysis of changes in facial movement is performed, for example, as follows. That is, the biological reaction analysis unit 12 identifies a face region from the frame image for each frame image and analyzes the orientation of the face. Then, the biological reaction analysis unit 12 calculates a face orientation change index value according to the analysis result of the face orientation change. For example, the biological reaction analysis unit 12 calculates the difference in the orientation of the face from the true state for each frame image by roll pitch yaw, and determines the moving average or the moving variance of the difference between a plurality of frames as the face orientation change index value. Calculated as.

The biological reaction analysis unit 12 may analyze where the user is looking. For example, it analyzes whether the speaker's face being displayed, the shared material being displayed, or the outside of the screen is being viewed. In addition, it may be possible to analyze whether the movement of the face is large or small, and whether the movement is frequent or infrequent. The movement of the face and the movement of the line of sight may be combined and analyzed. For example, it may be possible to analyze whether the speaker's face being displayed is viewed straight, whether the speaker is viewed with an upper eye or a lower eye, or whether the speaker is viewed from an angle. The biological reaction analysis unit 12 calculates a face orientation change index value according to the analysis result of the face orientation change.

For example, the biological reaction analysis unit 12 determines the viewing location (speaker's face, shared materials, outside the screen), the direction in which the location is viewed, the magnitude of facial movement, and the frequency of facial movement. Is used as an explanatory variable, and a predetermined function with the face orientation change index value as the objective variable is used to calculate the face orientation change index value. This function, for example, changes the absolute value of the face orientation change index value depending on the place of viewing and the direction in which the person is looking. It is possible to make the function so that the absolute value of the value becomes large.

The content of the statement is analyzed as follows, for example. That is, the biological reaction analysis unit 12 converts the voice into a character string by performing a known voice recognition process on the voice for a specified time (for example, a time of about 30 to 150 seconds), and morphologically analyzes the character string. By doing so, words unnecessary for expressing conversation such as particles and acronyms are removed. Then, the remaining words are vectorized by the TF-IDF (Term Frequency-Inverse Document Frequency) method, etc., and based on the characteristics of the vector, it is analyzed whether a positive emotional change is occurring or a negative emotional change is occurring. , Calculate the remark content index value according to the analysis result. For example, what kind of remark content is used by using a database or the like that stores information relating the vector feature amount and the remark content type based on the vector feature calculated according to the remark content. To estimate. Then, it is possible to calculate the statement content index value by using a predetermined function using the estimation result as an explanatory variable and the statement content index value as the objective variable.

As another example, the following may be performed. That is, the biological reaction analysis unit 12 matches the words extracted from the content of remarks within the specified time with a dictionary (definition of whether each word is positive or negative), and the number of appearances of positive words and negative words. Count the number of appearances. Then, the biological reaction analysis unit 12 calculates the remark content index value by using a predetermined function using each count value as an explanatory variable and the remark content index value as the objective variable.

Voice quality analysis is performed as follows, for example. That is, the biological reaction analysis unit 12 identifies the acoustic characteristics of the voice by performing a known voice analysis process on the voice for a specified time (for example, a time of about 30 to 150 seconds). Then, the voice quality change index value is calculated based on the value representing the acoustic feature. For example, the biological reaction analysis unit 12 calculates MFCC (mel frequency cepstrum coefficient) as an acoustic feature of speech, and calculates a moving average or a moving variance for each predetermined time interval of the MFCC as a voice quality change index value. MFCC is an example and is not limited to this.

The biological reaction analysis unit 12 determines whether a positive voice quality change is occurring, a negative voice quality change is occurring, and how loud the voice quality change is occurring, based on the acoustic characteristics of the voice. It may be analyzed and the voice quality change index value according to the analysis result may be calculated. For example, similar to the analysis of facial expressions, it analyzes which emotional element of neutrality / calmness / joy / surprise / sadness / anger / fear / disgust corresponds to the voice according to a voice analysis model machine-learned in advance. do. Then, based on the analysis result, whether or not the emotional change occurs at a predetermined time interval, and if the emotional change occurs, whether it is a positive emotional change or a negative emotional change, and how large it is. It analyzes whether or not the emotional change is occurring, and calculates the voice quality change index value according to the analysis result.

The biological reaction analysis unit 12 uses at least one of the facial expression change index value, the line-of-sight change index value, the pulse change index value, the face orientation change index value, the speech content index value, and the voice quality change index value calculated as described above. The biological reaction index value is calculated. For example, the biological reaction index value is calculated by weighting the facial expression change index value, the line-of-sight change index value, the pulse change index value, the face orientation change index value, the speech content index value, and the voice quality change index value.

The emotion evaluation unit 13 evaluates the degree of emotion of the subject according to the evaluation criteria leveled among the plurality of subjects based on the change in the biological reaction analyzed for the subject by the biological reaction analysis unit 12. For example, the emotion evaluation unit 13 has an emotional response based on an evaluation standard leveled among a plurality of subjects based on the change in the biological reaction (biological reaction index value) analyzed for the subject by the biological reaction analysis unit 12. Calculate the absolute value.

The emotional response absolute value calculated by the emotional evaluation unit 13 is, for example, a value obtained by adjusting the biological reaction index value calculated by the biological reaction analysis unit 12 according to the likelihood of the same emotion occurring by the subject. For example, the emotion evaluation unit 13 calculates the absolute emotional response value by multiplying the biological reaction index value calculated by the biological reaction analysis unit 12 by a weight value according to the frequency of causing the same emotion.

As another example, the emotion evaluation unit 13 has a facial expression change index value, a line-of-sight change index value, a pulse change index value, a face orientation change index value, a speech content index value, and a voice quality change index value calculated by the biological reaction analysis unit 12. Emotions by multiplying each of the above (hereinafter, may be abbreviated as each index value) or at least one index value in each index value by a weight value according to the frequency of causing the same emotion. The absolute reaction value may be calculated. In the following, the index value (any of the biological reaction index value, each index value, and at least one index value) used for calculating the absolute emotional response value is referred to as a calculation target index value.

For example, even if the calculated target index value calculated for the target person A and the calculated target index value calculated for the target person B are the same value, the susceptibility to generate the same emotion (frequency of generating the same emotion). ) Is different between the subject A and the subject B, the absolute emotional response value calculated by the emotion evaluation unit 13 is different between the subject A and the subject B. As an example, the emotion evaluation unit 13 calculates the absolute emotional response value according to a function such that the weight value becomes smaller as the same emotion is more likely to occur, and the weight value becomes larger as the same emotion is less likely to occur.

The frequency of causing the same emotion counts, for example, the number of times that approximately the same value occurs based on a plurality of calculated index values calculated for each predetermined time interval (for example, every 500 milliseconds) during an online session. It can be obtained by doing. The term "almost the same value" as used herein means not only a value having exactly the same value but also a value having a predetermined width that allows a predetermined difference to be regarded as the same value. It should be noted that not only the calculated target index value calculated in one online session but also the calculated target index value calculated in a plurality of online sessions may be used to determine the frequency of causing the same emotion.

In the case of this example, the emotion evaluation unit 13 has, for example, information representing each emotion (division of a calculation target index value having a predetermined width) and a weight value set according to the frequency of generating the same emotion for each emotion. The table information associated with and is stored in advance. Then, the emotion evaluation unit 13 extracts the weight value corresponding to the analysis result by collating the analysis result by the biological reaction analysis unit 12 with the table information, and multiplies the weight value by the calculation target index value. Calculate the absolute emotional response value. In addition, instead of the table information, a function designed to calculate the weight value based on the information representing each emotion may be used.

As described above, in the present embodiment, the degree of emotions considering the susceptibility to different emotions for each subject is evaluated by using the emotional response absolute value calculated based on the frequency of causing the same emotions. Therefore, it is possible to evaluate the degree of emotion in the true sense of the subject, and it is possible to objectively compare the degree of emotion between different subjects.

The processing of the biological reaction analysis unit 12 and the emotion evaluation unit 13 described above may be performed in real time when the moving image acquisition unit 11 acquires the moving images of a plurality of subjects, or the moving image storage unit 101. It may be performed after the fact by using the moving image stored in.

In the above embodiment, an example of evaluating the degree of emotion adjusted according to the likelihood of the same emotion being generated by the subject has been described, but the present invention is not limited to this. For example, the emotion evaluation unit 13 is a degree of emotion based on the magnitude of the difference in the current biological reaction to the biological reaction in normal times, and the emotion is adjusted according to the likelihood of the same emotion being generated by the subject. The degree may be evaluated. The biological reaction in normal times can be, for example, a major biological reaction analyzed based on the past biological reaction distribution of the same subject.

For example, the emotion evaluation unit 13 uses the calculated index value calculated by the biological reaction analysis unit 12 to determine the magnitude of the difference in the current biological reaction to the biological reaction in normal times and the susceptibility to the same emotion by the subject. The absolute value of emotional response is calculated by adjusting accordingly. The absolute emotional response value calculated in this way is a value representing the degree of emotion based on the magnitude of the difference in the current biological response to the biological response in normal times, and the subject is likely to generate the same emotion or occurs. It is a value adjusted according to the degree of difficulty.

Further, the method of determining the frequency of causing the same emotion described in the above embodiment is an example, and the present invention is not limited to this. For example, the same emotion based on the emotional element (neutral, calm, joy, surprise, sadness, anger, fear, disgust) analyzed by the biological reaction analysis unit 12 at predetermined time intervals during the online session. By counting the number of times the element occurs, the frequency with which the same emotion is generated may be determined. Alternatively, joy and surprise are defined as "pleasant" emotions, while sadness, anger, fear and disgust are defined as "unpleasant" emotions, and every predetermined time interval during an online session. Based on the emotional elements analyzed by the biological reaction analysis unit 12, the frequency of generating "pleasant" emotions and the frequency of generating "unpleasant" emotions may be determined.

When the frequency is obtained by counting the number of times the same emotion element occurs based on the emotion element analyzed by the biological reaction analysis unit 12, the emotion evaluation unit 13 is, for example, information representing each emotion (each emotion element). And the table information corresponding to the weight value set according to the frequency of causing the same emotion for each emotion is stored in advance. Then, the emotion evaluation unit 13 extracts the weight value corresponding to the analysis result by collating the analysis result by the biological reaction analysis unit 12 with the table information, and multiplies the weight value by the score corresponding to the emotion element. By doing so, the calculation target index value is calculated, and further, the emotional response absolute value is calculated from the calculation target index value. In addition, instead of the table information, a function designed to calculate the weight value based on the information representing each emotion may be used.

Similarly, when determining the frequency of causing pleasant / unpleasant emotions based on the emotional elements analyzed by the biological reaction analysis unit 12, the emotional evaluation unit 13 may obtain, for example, information representing each emotion (comfort / discomfort classification). ) And the weight value set according to the frequency of causing the same emotion for each emotion are stored in advance. Then, the emotion evaluation unit 13 extracts the weight value corresponding to the analysis result by collating the analysis result by the biological reaction analysis unit 12 with the table information, and the emotion element whose weight value is defined as pleasant or unpleasant. The calculation target index value is calculated by multiplying the score corresponding to, and the emotional response absolute value is calculated from the calculation target index value. In addition, instead of the table information, a function designed to calculate the weight value based on the information representing each emotion may be used.

Further, in the above embodiment, an example in which the frequency of generating the same emotion is used as a measure for expressing the susceptibility to the same emotion has been described, but the present invention is not limited to this. For example, the nature or personality of the subject may be used in place of or in addition to the frequency with which the same emotions occur.

Further, in the above embodiment, an example of analyzing emotions with a participant as a target of analysis in an environment where an online session is performed through a session management server 300 between a plurality of user terminals 200 used by a plurality of participants. As explained, the target audience for emotion analysis is not limited to participants in online sessions. That is, any person who is in an environment where a moving image can be acquired can analyze emotions as a target person of the present embodiment.

Other than that, all of the above embodiments are merely examples of the embodiment of the present invention, and the technical scope of the present invention should not be construed in a limited manner by this. That is, the present invention can be implemented in various forms without departing from its gist or its main features.

11 Moving image acquisition unit 12 Biological reaction analysis unit 13 Emotion evaluation unit 100 Emotion analysis device

Claims

It is an emotion analysis system that analyzes emotions with the above participants as the analysis target in an environment where an online session is held by multiple participants.
A moving image acquisition unit that acquires a moving image transmitted from the user terminal of the target person during the online session, and a moving image acquisition unit.
Based on the moving image acquired by the moving image acquisition unit, the biological reaction analysis unit that analyzes the change in the biological reaction caused by the emotional change of the subject, and the biological reaction analysis unit.
An emotion evaluation unit that evaluates the degree of emotion adjusted according to the likelihood of the same emotion being generated by the subject based on the change in the biological reaction analyzed for the subject by the biological reaction analysis unit. An emotion analysis system characterized by being prepared.
The emotion evaluation unit is the degree of emotion based on the magnitude of the difference in the current biological reaction to the biological reaction in normal times, and the degree of emotion adjusted according to the likelihood of the same emotion being generated by the subject. The emotion analysis system according to claim 1, wherein the emotional analysis system is characterized in that.
The biological reaction analysis unit calculates the biological reaction index value by quantifying the change in the biological reaction according to a predetermined standard.
The emotion evaluation unit is characterized in that it calculates an absolute emotional response value, which is a value obtained by adjusting the biological reaction index value calculated by the biological reaction analysis unit according to the likelihood of the same emotion occurring by the subject. The emotion analysis system according to claim 1.
The biological reaction analysis unit calculates the biological reaction index value by quantifying the change in the biological reaction according to a predetermined standard.
The emotion evaluation unit uses the biological reaction index value calculated by the biological reaction analysis unit as the magnitude of the difference in the current biological reaction with respect to the biological reaction in normal times, and the tendency of the subject to cause the same emotion. The emotion analysis system according to claim 2, wherein the absolute value of the emotional response is calculated by adjusting according to the situation.
The emotion evaluation unit calculates the absolute emotional response value by multiplying the biological reaction index value calculated by the biological reaction analysis unit by a weight value according to the frequency at which the subject causes the same emotion. The emotion analysis system according to claim 3, wherein the emotion analysis system is characterized in that.
The biological reaction analysis unit analyzes at least one of the facial image and the voice in the moving image acquired by the moving image acquisition unit, and relates to at least one of facial expression, line of sight, pulse, facial movement, speech content, and voice quality. By quantifying the change in the biological reaction according to a predetermined standard, at least one of the facial expression change index value, the line-of-sight change index value, the pulse change index value, the face orientation change index value, the speech content index value, and the voice quality change index value can be obtained. Calculate and
The emotion evaluation unit includes the facial expression change index value, the line-of-sight change index value, the pulse change index value, the face orientation change index value, the remark content index value, and the voice quality change index value calculated by the biological reaction analysis unit. The emotion analysis system according to claim 3, wherein the emotion response absolute value is calculated by multiplying at least one of the values by a weight value according to the frequency at which the subject causes the same emotion. ..
Based on the moving image obtained for the subject, the biological reaction analysis unit that analyzes the change in the biological reaction caused by the change in emotions of the subject, and the biological reaction analysis unit.
An emotion evaluation unit that evaluates the degree of emotion adjusted according to the likelihood of the same emotion being generated by the subject based on the change in the biological reaction analyzed for the subject by the biological reaction analysis unit. An emotion analysis device characterized by being equipped.
The emotion evaluation unit is the degree of emotion based on the magnitude of the difference in the current biological reaction to the biological reaction in normal times, and the degree of emotion adjusted according to the likelihood of the same emotion being generated by the subject. The emotion analysis apparatus according to claim 7, wherein the emotional analysis device is characterized in that.
The biological reaction analysis unit calculates the biological reaction index value by quantifying the change in the biological reaction according to a predetermined standard.
The emotion evaluation unit is characterized in that it calculates an absolute emotional response value, which is a value obtained by adjusting the biological reaction index value calculated by the biological reaction analysis unit according to the likelihood of the same emotion occurring by the subject. The emotion analysis device according to claim 7.
The biological reaction analysis unit calculates the biological reaction index value by quantifying the change in the biological reaction according to a predetermined standard.
The emotion evaluation unit uses the biological reaction index value calculated by the biological reaction analysis unit as the magnitude of the difference in the current biological reaction with respect to the biological reaction in normal times, and the tendency of the subject to cause the same emotion. The emotion analysis device according to claim 8, wherein the absolute value of the emotional response is calculated by adjusting according to the above.