JP7197957B2 - Reaction analysis system and reaction analysis device - Google Patents

Description

The present invention relates to a reaction analysis system and reaction analysis device, and more particularly to a system and device for analyzing biological reactions of users participating in an online session.

2. Description of the Related Art Conventionally, there is known a system in which an image of a participant captured by a camera is displayed on a screen in an online conference or the like in which a plurality of participants participate, so that the participant's state can be grasped. In addition, there is also a known system in which, when shared materials are displayed on the main screen in an online conference, etc., the facial images of the participants are displayed on a small screen so that the participant's state can be grasped (for example, , see Patent Document 1).

On the other hand, in view of the fact that users tend to dislike displaying their own faces on the other party's screen as they are in videophones, we analyzed changes in facial expressions from captured images during conversations, and used the analysis results as the default Japanese Patent Application Laid-Open No. 2002-301000 discloses that the facial expression image is reflected in the facial expression image and transmitted to the other party. In Patent Document 2, in addition to an embodiment in which the above-described processing is always performed, the above-described processing is performed when a button operation is performed during a call, and a default facial expression image is displayed on the other party's side when no button operation is performed. A transmitting embodiment is disclosed.

Techniques for analyzing the emotions of a person based on a face image of the person captured by a camera are also known (see Patent Documents 3 and 4, for example). Patent Literature 3 discloses analyzing the degree of expression of emotions from facial expression information obtained from the faces of a plurality of subjects. Further, Patent Document 4 discloses that in a video conference system, the line-of-sight position is detected from the face image included in the video of the speaker, the speaker's emotion is estimated, and the emotion is presented to the speaker who is speaking. ing.

Japanese Patent Application Laid-Open No. 2020-48149 JP 2007-213364 A JP 2019-58625 A JP-A-2005-269207

Many of the recent web conference systems have a camera on/off function, and can switch whether to display the photographed image of the participant on the screen. When the camera is turned off, the participant's photographed image is not displayed on the screen, so other participants cannot grasp the state of the participant whose camera is turned off. On the other hand, if the technique described in Patent Document 2 is used, by transmitting to the other party a facial expression image generated by reflecting changes in the facial expression analyzed from the captured image, the other party can It is possible to grasp the change in the facial expression of

However, in the system described in Patent Document 2, a default facial expression image (initial setting image) is transmitted to the other party, or a facial expression image (edited image) generated by reflecting changes in the speaker's facial expression is sent to the other party. However, the photographed image of the speaker is not transmitted to the other party. Therefore, there is a problem that it is impossible to grasp the state of the other party from the photographed image.

The present invention has been made to solve such problems, and in an environment where an online session is held by a plurality of people, according to the switching of the display / non-display of the captured image by the participants, it is suitable for each. An object of the present invention is to make it possible to grasp the state of a participant in a manner.

In order to solve the above problems, in the reaction analysis system of the present invention, in an environment where an online session is held by a plurality of participants, based on the moving images obtained from the user terminals of the participants during the online session, At least for a non-display participant whose face image is not displayed on the screen of the user terminal during the online session, the biological reaction is analyzed and the analysis result is presented.

According to the present invention configured as described above, in an environment where an online session is held by a plurality of people, at least the face images of the participants whose face images are set to be displayed on the screen are presented, and the face images are displayed. For participants who are set not to display on the screen, at least the analysis results of changes in biological reactions are presented, so depending on the display / non-display of the captured image by the participant, the mode suitable for each It is possible to grasp the state of the participants.

1 is a block diagram showing an example of the overall configuration of a reaction analysis system according to this embodiment; FIG. 1 is a block diagram showing a functional configuration example of a reaction analysis device according to this embodiment; FIG. FIG. 10 is a diagram showing an example of information indicating changes in emotions displayed on the user terminal;

An embodiment of the present invention will be described below with reference to the drawings. FIG. 1 is a diagram showing an example of the overall configuration of a reaction analysis system according to this embodiment. As shown in FIG. 1, the reaction analysis system of this embodiment includes a reaction analysis device 100, a plurality of user terminals 200 _-1 , 200 _-2 , . ) and a session management server 300 . These reaction analysis device 100, user terminal 200 and session management server 300 are connected via a communication network 500 such as the Internet or a mobile phone network.

In an environment where an online session is held by a plurality of participants, the reaction analysis system of the present embodiment can be used for a participant whose face image is not displayed on the screen of the user terminal 200 during the online session (hereinafter referred to as a hidden participant). ), the biological reaction is analyzed based on the moving image obtained from the user terminal 200, and the analysis result is presented. It should be noted that the emotions of the participants may be analyzed through the analysis of biological reactions.

An online session is, for example, an online conference, an online class, an online chat, or the like. A moving image can be exchanged between a plurality of user terminals 200 through the reaction analysis device 100 and the session management server 300 . An application program (hereinafter referred to as a session application) necessary for exchanging moving images in an online session is installed in the plurality of user terminals 200 .

The moving images handled in the online session include the face image of the user using the user terminal 200 (actually, body parts other than the face and background images are also included) and voice. A user's face image and voice are acquired by a camera and a microphone provided in the user terminal 200 or connected to the user terminal 200 and transmitted to the session management server 300 . The face image and voice of each user transmitted to the session management server 300 are acquired by the reaction analysis device 100 and transmitted from the reaction analysis device 100 to the session application of each user terminal 200 . The moving image transmitted from the user terminal 200 may be acquired by the reaction analysis device 100 and transferred from the reaction analysis device 100 to the session management server 300 . Alternatively, the moving image may be transmitted from user terminal 200 to both reaction analysis device 100 and session management server 300 .

Moving images also include images such as materials that are shared and viewed by a plurality of users. A material image viewed by the user is transmitted from any user terminal 200 to the session management server 300 . The material image transmitted to the session management server 300 is acquired by the reaction analysis device 100 and transmitted from the reaction analysis device 100 to the session application of each user terminal 200 .

By the above operation, in each of the plurality of user terminals 200, the face images or document images of the plurality of users are displayed on the display, and the voices of the plurality of users are output from the speakers. Here, by the function of the session application installed in the user terminal 200, the face image and the document image are switched on the screen of the display to display only one of them, or the face image and the document image are displayed by dividing the display area. can be displayed at the same time. In addition, it is possible to display an image of one of a plurality of users on the full screen, or to display images of some or all of the users by dividing them into small screens.

It is also possible to switch on/off the camera and switch on/off the microphone by using the function of the session application installed in the user terminal 200 . For example, when the camera of the user terminal _200-1 is turned off, the face image captured by the camera of the user terminal _200-1 is transmitted to the session management server 300 and the reaction analysis device 100. It is not transmitted to the user terminal 200. Similarly, when the microphone of the user terminal 200 _-1 is turned off, the sound collected by the microphone of the user terminal 200 _-1 is transmitted to the session management server 300 and the reaction analysis device 100. It is not transmitted to each user terminal 200.

In this embodiment, a plurality of user terminals 200 _-1 , 200 _-2 , . terminals used by participants. Hosts of online sessions are, for example, instructors of online classes, chairpersons and facilitators of online meetings, coaches of sessions for coaching purposes, and the like. The online session host may be one of the participants participating in the online session or another person not participating in the online session. In the following description, the organizer and participants are referred to as "users" when not distinguished from each other.

In this embodiment, the reaction analysis device 100 acquires moving images transmitted from a plurality of user terminals 200, and analyzes the biological reactions of participants in the online session based on the moving images. When the host of the online session is also a participant, it is not essential to analyze the host's biological reaction.

FIG. 2 is a block diagram showing a functional configuration example of the reaction analysis device 100 according to this embodiment. As shown in FIG. 2, the reaction analysis device 100 of this embodiment includes a moving image acquisition unit 11, a biological reaction analysis unit 12, and a reaction information presentation unit 13 as functional configurations. The reaction analysis device 100 of this embodiment also includes a moving image storage unit 101 as a storage medium.

Each of the functional blocks 10 to 13 can be configured by hardware, DSP (Digital Signal Processor), or software. For example, when configured by software, each of the functional blocks 10 to 13 is actually configured with a CPU, RAM, ROM, etc. of a computer, and a program stored in a recording medium such as RAM, ROM, hard disk, or semiconductor memory. is realized by the operation of

The moving image acquisition unit 11 acquires, from the session management server 300, moving images (face images, voices, and document images) transmitted from each user terminal 200 during an online session. The moving image acquisition unit 11 stores the moving images acquired from each user terminal 200 via the session management server 300 in the moving image storage unit 101 in association with information (for example, user ID) that can identify each user.

It does not matter whether the face image acquired from the session management server 300 is set to be displayed on the screen of each user terminal 200 (whether the camera is turned on or off). No. That is, the moving image acquisition unit 11 acquires face images from the session management server 300 including face images being displayed and face images not being displayed on the display of each user terminal 200 . Also, it does not matter whether the voice acquired from the session management server 300 is set to be output from the speaker of each user terminal 200 (whether the microphone is set to ON or OFF). No. That is, the moving image acquisition unit 11 acquires sounds from the session management server 300 including sounds being output and sounds not being output from the speaker of each user terminal 200 .

Based on the moving image acquired by the moving image acquisition unit 11 and stored in the moving image storage unit 101, the biological reaction analysis unit 12 performs a non-display participation process in which the face image is not displayed on the screen of the user terminal 200 during the online session. Analyze changes in biological reactions caused by changes in emotions of participants (for example, participants whose cameras are turned off). In this embodiment, the biological reaction analysis unit 12 separates the moving image acquired by the moving image acquisition unit 11 into a set of face images (collection of frame images) and voice, and analyzes changes in the biological reaction from each.

For example, the biological reaction analysis unit 12 analyzes the user's facial image using a frame image separated from the moving image acquired by the moving image acquisition unit 11 to obtain at least one of facial expression, gaze, pulse, and facial movement. Analyze changes in biological reactions related to Further, the biological reaction analysis unit 12 analyzes the voice separated from the moving image acquired by the moving image acquisition unit 11 to analyze changes in the biological reaction related to at least one of the user's utterance content and voice quality.

When people's emotions change, it appears as changes in biological reactions such as facial expressions, eyes, pulse, facial movements, content of remarks, and voice quality. In this embodiment, changes in biological reactions caused by changes in user's emotions are analyzed. Also, the change in the user's emotion may be analyzed through the analysis of the change in biological reaction.

Analysis of changes in facial expressions is performed, for example, as follows. That is, for each frame image, the biological reaction analysis unit 12 identifies the facial region from the frame image, and analyzes which facial expression element corresponds to the facial expression according to the image analysis model machine-learned in advance. do. Then, based on the analysis results, it is determined whether facial expression changes occur between consecutive frame images, and if facial expression changes occur, whether the facial expression changes are positive or negative, and to what extent. Analyze whether there is a change in facial expression.

Facial expression elements include, for example, neutral/calm/happy/surprised/sad/angry/fearful/disgust. be. Of these, joy and surprise are positive facial expression elements, and sadness, anger, fear, and disgust are negative facial expression elements.

The biological reaction analysis unit 12 calculates a total score of 100 for each of the facial expression elements in each frame image. For example, neutral = 10, calmness = 10, joy = 30, surprise = 20, sadness = 10, anger = 10, fear = 5, disgust = 5, etc. A corresponding score is calculated for each facial expression element. Then, for example, the facial expression element with the highest score is determined as the facial expression in that frame image. Hereinafter, the facial expression score determined for each frame image (maximum score among scores calculated for a plurality of facial expression elements) is referred to as "expression score".

The biological reaction analysis unit 12 determines whether or not at least one of the facial expression element determined for each frame image and the facial expression score calculated for each frame image has changed from the previous frame. Determine whether or not facial expression change occurs. Here, if there is no change in the facial expression element with the maximum score, the biological reaction analysis unit 12 may determine that the facial expression has changed if the amount of change in the score from the previous frame is equal to or greater than a predetermined threshold. good. The magnitude of change in facial expression can be determined by the amount of change in the facial expression score from the previous frame.

In addition, the biological reaction analysis unit 12 determines that a positive facial expression change occurs when the facial expression score of a positive facial expression increases from the previous frame, or when a negative facial expression in the previous frame changes to a positive facial expression in the current frame. determined to be On the other hand, the biological reaction analysis unit 12 determines that a negative facial expression change occurs when the facial expression score of the negative facial expression increases from the previous frame, or when the positive facial expression in the previous frame changes to the negative facial expression in the current frame. determined to be

Here, an example of analyzing facial expression changes between consecutive frame images has been described, but facial expression changes may be analyzed for each predetermined time interval (for example, every 500 milliseconds). This also applies to the analysis of changes in line of sight, the analysis of changes in pulse, and the analysis of changes in facial movements, which will be described below.

The analysis of changes in line of sight is performed, for example, as follows. That is, for each frame image, the biological reaction analysis unit 12 identifies an eye region in the frame image and analyzes the direction of both eyes (line of sight). For example, it analyzes whether the person is looking at the face of the speaker being displayed, whether the shared material being displayed is being looked at, where in the shared material being displayed is being looked at, and whether the user is looking outside the screen. Also, it may be analyzed whether the eye movement is large or small, or whether the movement is frequent or infrequent. A change in line of sight is also related to the user's degree of concentration.

Analysis of changes in pulse rate is performed, for example, as follows. That is, for each frame image, the face area is specified in the frame image. Then, using a trained image analysis model that captures numerical values of face color information (G of RGB), changes in the G color of the face surface are analyzed. By arranging the results along the time axis, a waveform representing changes in color information is formed, and the pulse is identified from this waveform. When a person is tense, the pulse speeds up, and when the person is calm, the pulse slows down.

Analysis of changes in facial movement is performed, for example, as follows. That is, the biological reaction analysis unit 12 identifies the facial region in each frame image and analyzes the direction of the face, thereby analyzing where the user is looking. For example, it analyzes whether the person is looking at the face of the speaker being displayed, whether the shared material being displayed is being looked at, where in the shared material being displayed is being looked at, and whether the user is looking outside the screen. Further, it may be analyzed whether the movement of the face is large or small, or whether the movement is frequent or infrequent. The movement of the face and the movement of the line of sight may be analyzed together. For example, it may be analyzed whether the face of the speaker being displayed is viewed straight, whether the face is viewed with upward or downward gaze, or whether the face is viewed obliquely.

Analysis of the content of the statement is performed, for example, as follows. That is, the biological reaction analysis unit 12 converts the voice into a character string by performing known voice recognition processing on the voice for a specified time (for example, about 30 to 150 seconds), and morphologically analyzes the character string. By doing so, words such as particles and articles that are unnecessary for expressing conversation are removed. Then, the remaining words are vectorized by the TF-IDF (Term Frequency-Inverse Document Frequency) method or the like, and based on the characteristics of the vector, whether a positive emotional change is occurring or a negative emotional change is occurring, and which Analyze whether emotional changes of a certain magnitude occur. For example, based on the feature of the vector calculated according to the content of the statement, using a database that stores information that associates the feature amount of the vector with the type of the content of the statement, what type of statement is to estimate

As another example, the following may be done. That is, the biological reaction analysis unit 12 compares the words extracted from the content of the utterances within the designated time with a dictionary (which defines whether each word is positive or negative), and determines the number of occurrences of positive words and the number of negative words. By counting the number of appearances, it is analyzed whether a positive emotional change has occurred, whether a negative emotional change has occurred, and to what extent the emotional change has occurred.

Analysis of voice quality is performed, for example, as follows. That is, the biological reaction analysis unit 12 performs a known sound analysis process on the sound for a specified time (for example, about 30 to 150 seconds) to determine the acoustic characteristics of the sound (for example, MFCC (Mel frequency cepstrum coefficient)). ). Then, based on the values representing the acoustic features, it is analyzed whether a positive change in voice quality has occurred, whether a negative change in voice quality has occurred, and to what extent the change in voice quality has occurred. For example, similar to the analysis of facial expressions, analyze which emotional elements of neutral/calm/joy/surprise/sadness/anger/fear/disgust the voice corresponds to according to the voice analysis model that has been trained in advance by machine learning. do. Then, based on the analysis results, it is determined whether an emotional change occurs in each predetermined time interval, whether the emotional change is positive or negative, and the magnitude of the emotional change. Analyze whether there is a change in the emotional state of the person. Note that MFCC is an example of acoustic features of speech, and the present invention is not limited to this.

In addition, in the present embodiment, the biological reaction analysis unit 12 determines whether a non-display participant who is not displayed on the screen is displayed on the screen during the online session, based on the sound in the moving image acquired by the moving image acquisition unit 11. Analyze which part of the displayed shared material was spoken when it was displayed. For example, the biological reaction analysis unit 12 analyzes which page of the shared material having a plurality of pages is displayed when the non-displaying participant speaks.

The reaction information presenting unit 13 presents information indicating changes in biological reactions analyzed by the biological reaction analyzing unit 12 for hidden participants who are not displayed on the screen of the user terminal 200 . For example, the reaction information presenting unit 13 presents the host of the online session with information indicating changes in biological reactions. As a result, on the screen of the user terminal 200 used by the organizer, information indicating changes in the analyzed biological reactions of the non-display participants is displayed.

Note that the reaction information presenting unit 13 may present information indicating changes in emotions instead of or in addition to information indicating changes in biological reactions. For example, an emotional element (any of neutral, calmness, joy, surprise, sadness, anger, fear, and disgust) analyzed by the biological reaction analysis unit 12 may be presented. Alternatively, two of joy and surprise are defined as "pleasant" emotions, while four of sadness, anger, fear and disgust are defined as "unpleasant" emotions, and the emotions analyzed by the biological reaction analysis unit 12. Based on the element, information indicating an emotion of either "pleasant" or "unpleasant" may be presented.

For example, the reaction information presenting unit 13 causes the host's user terminal 200 to display a dashboard screen in which a list of identification information (names, nicknames, IDs, etc.) of the hidden participants is displayed. When one of the hidden participants is designated, it is possible to display information indicating changes in biological reaction or information indicating changes in emotions regarding the designated hidden participant. FIG. 3 is a diagram showing an example of information indicating changes in emotion displayed on the user terminal 200 in this way.

FIG. 3(a) shows scores for each of multiple emotional elements analyzed based on facial expression or voice quality in chronological order at frame image intervals or predetermined time intervals (e.g., 500 millisecond intervals). It is a line graph. FIG. 3(b) shows an average value of multiple scores obtained in time series as shown in FIG. 3(a) (average score over online sessions) for each of multiple emotional elements.

Here, as an example of information indicating a change in biological reaction or information indicating a change in emotion, an example of displaying the score of each emotional element is shown, but the present invention is not limited to this. For example, a score that represents the degree of concentration for an online session is calculated based on changes in gaze or facial movement, and the score is displayed in chronological order in a line graph, or an average score is displayed throughout the online session. can be In addition, based on changes in the pulse, a score that expresses the degree of calmness (calm) among emotional factors is calculated, and the score is displayed in chronological order in a line graph, and the average score throughout the online session is displayed. You may make it Alternatively, calculate a score representing the degree of positivity or negativity based on facial expressions, speech content, or sound quality, and display the score in chronological order in a line graph, or display the average score throughout the online session. You may do so.

In addition, the reaction information presentation unit 13 displays the face image of a participant whose face image is set to be displayed on the screen (for example, a participant whose camera is turned on; hereinafter referred to as a display participant). is presented on the user terminal 200 of the organizer, while information indicating changes in biological reactions of non-display participants whose face images are set not to be displayed on the screen is presented on the user terminal 200 of the organizer. can be In the user terminal 200 of the organizer, for example, the screen is divided into small screens for the number of participants, and a face image is displayed on the small screen for the displayed participants, and a biological reaction is displayed on the small screen for the non-displayed participants. It is possible to display information indicating the change. The small screens for the number of participants may all be displayed on the display at once, or only the small screens of some of the participants may be displayed on the display and scrolled. Alternatively, the organizer may be allowed to designate the displayed participants and/or the non-displayed participants for whom the information indicating the change in biological reaction or the information indicating the change in emotion is to be displayed.

As described in detail above, according to the present embodiment, on the screen of the user terminal 200 used by the organizer of the online session, the face images of the displayed participants are displayed, and the biological reactions of the non-displayed participants change. Or the analysis result of the change of emotion is displayed. For this reason, in an environment where an online session is held with a plurality of participants, the organizer can grasp the state of the display participant whose face image is displayed on the screen from the face image, As for the non-display participants whose images are not displayed on the screen, it is possible to comprehend the situation from information indicating changes in biological reactions or changes in emotions. This allows the organizer to grasp the state of the participants in a manner suitable for each of the participants, according to the display/non-display switching of the face images of the participants.

The processing of the biological reaction analysis unit 12 and the reaction information presentation unit 13 described above may be performed in real time when the moving image acquisition unit 11 acquires moving images of a plurality of participants, or the moving image storage unit The moving image stored in the memory 101 may be used for post-processing.

In the above embodiment, the participant whose face image is set not to be displayed on the screen is the participant whose camera of the user terminal 200 is set to OFF, but the present invention is not limited to this. For example, when only the shared document is displayed on the screen, even if the participant has the camera turned on, the face image is set not to be displayed on the screen while the shared document is being displayed. Become a display participant.

Further, in the above-described embodiment, an example in which changes in biological reactions are analyzed only for non-display participants has been described, but the present invention is not limited to this. For example, based on the moving image acquired by the moving image acquisition unit 11, the biological reaction analysis unit 12 also analyzes the biological reaction of the displayed participant whose face image is displayed on the screen of the user terminal 200 during the online session. You may make it analyze a change.

In this case, the reaction information presentation unit 13 may present information indicating changes in biological reactions (or changes in emotions) analyzed by the biological reaction analysis unit 2 for the hidden participants and the displayed participants. As for the displayed participants, it is possible to grasp the situation from the face images displayed on the screen. It is possible to grasp the appearance of the displayed participants from both the image and the analysis result.

When the biological reaction analysis unit 12 analyzes changes in biological reactions of all participants in the online session (display participants and non-display participants), changes in biological reactions (or changes in emotions) analyzed for the displayed participants ) is to be presented to the organizer, the organizer or the displayed participants may designate from the user terminal 200. FIG. Similarly, the organizer or the hidden participant can specify from the user terminal 200 whether or not to present to the organizer information indicating changes in biological reactions (or changes in emotions) analyzed for the hidden participants. can be

Further, in the above embodiment, the reaction information presentation unit 13 can display the results of the participants (only the non-display participants, or the Based on the analysis result of what part of the shared material was being viewed by all members, the same applies below), a heat map can be projected and displayed on the shared material on the screen. good. The heat map referred to here is, for example, a map in which the color becomes darker as the number of places viewed by a plurality of participants increases.

Alternatively, the reaction information presentation unit 13 displays a heat display on the shared material on the screen, based on the analysis result of the voice, which part (page) of the shared material is being displayed and which part (page) of the shared material is being displayed. The map may be projected and displayed. The heat map referred to here is, for example, a map in which, among shared materials having a plurality of pages, the color of a page displayed when a plurality of participants uttered more voices becomes darker.

As yet another example, the reaction information presenting unit 13 provides a first analysis result regarding at least one of line-of-sight and facial movement indicating where the participant was looking in the shared material being displayed, and which part of the shared material was viewed. A heat map may be projected and displayed based on the second analysis result regarding the voice, whether the participant spoke while being displayed. The heat map referred to here is, for example, a map in which the color of a specific area within a page reflecting the first analysis result and the color of the entire page reflecting the second analysis result are overlapped.

It should be noted that each of the above-described embodiments merely shows an example of implementation of the present invention, and the technical scope of the present invention should not be construed to be limited. Thus, the invention may be embodied in various forms without departing from its spirit or essential characteristics.

11 moving image acquisition unit 12 biological reaction analysis unit 13 reaction information presentation unit

Claims (10)

A reaction analysis system that analyzes the biological reactions of the participants in an environment where an online session is held by a plurality of participants,
a moving image acquisition unit that acquires a moving image transmitted from the user terminal of the participant during the online session;
Based on the moving image acquired by the moving image acquiring unit, analyze changes in biological reactions of non-display participants who are participants whose face images are not displayed on the screen of the user terminal at least during the online session. a biological reaction analysis unit to
a reaction information presentation unit that presents information indicating changes in the biological reaction analyzed at least for the non-display participant by the biological reaction analysis unit;
Based on the audio in the video image acquired by the video image acquisition unit, the biological reaction analysis unit determines which part of the shared material displayed on the screen during the online session is being displayed. Analyze whether the participant spoke,
The reaction information presentation unit projects a heat map onto the shared material based on the analysis result of which part of the shared material is being displayed during the online session and the participant speaks out. A reaction analysis system characterized by displaying. Based on the video image acquired by the video image acquisition unit, the biological reaction analysis unit also performs display participants whose face images are displayed on the screen of the user terminal during the online session. Analyzing changes in the above biological reactions,
2. The reaction according to claim 1, wherein the reaction information presenting unit presents information indicating changes in the biological reaction analyzed for the non-display participant and the displayed participant by the biological reaction analysis unit. analysis system. The biological reaction analysis unit analyzes a change in biological reaction related to at least one of eye movement and facial movement based on the facial image in the moving image acquired by the moving image acquisition unit, thereby Analyze where you are looking at the shared document displayed on the above screen,
The reaction information presentation unit is characterized by projecting and displaying a heat map on the shared material based on an analysis result of where the participants are looking in the shared material displayed on the screen. The reaction analysis system according to claim 1 or 2. 4. The reaction analysis system according to any one of claims 1 to 3, wherein the reaction information presenting unit presents information indicating changes in the biological reaction to a host of the online session. 5. The reaction analysis system according to claim 4, wherein the organizer or the participant can specify whether or not to present the information indicating the change in the biological reaction to the organizer. The biological reaction analysis unit analyzes changes in the participant's emotions through analysis of changes in the biological reaction,
6. The reaction information presenting unit according to any one of claims 1 to 5, wherein the reaction information presenting unit presents the information indicating the change in emotion instead of or in addition to the information indicating the change in biological reaction. Reaction analysis system. A non-display participant who is a participant whose face image is not displayed on the screen of the user terminal at least during the online session based on the moving image obtained for the participant during the online session conducted by a plurality of participants About, a biological reaction analysis unit that analyzes changes in biological reactions,
a reaction information presentation unit that presents information indicating changes in the biological reaction analyzed at least for the non-display participant by the biological reaction analysis unit;
The biological reaction analysis unit determines which part of the shared material displayed on the screen during the online session was displayed and when the participant spoke, based on the voice in the moving image. parse and
The reaction information presentation unit creates a heat map based on the analysis result of which part of the shared material displayed on the screen during the online session is displayed and when the participant speaks out. A reaction analysis device characterized by projecting and displaying on shared materials. Based on the moving image, the biological reaction analysis unit also analyzes the change in the biological reaction of the display participant whose face image is displayed on the screen of the user terminal during the online session. ,
8. The reaction according to claim 7, wherein the reaction information presentation unit presents information indicating changes in the biological reaction analyzed for the non-display participant and the display participant by the biological reaction analysis unit. analysis equipment. The biological reaction analysis unit analyzes changes in biological reactions related to at least one of eye movement and facial movement based on the face image in the moving image, thereby enabling the participant to share the image displayed on the screen. Analyze where you are looking at the material,
The reaction information presentation unit is characterized by projecting and displaying a heat map on the shared material based on an analysis result of where the participants are looking in the shared material displayed on the screen. The reaction analysis device according to claim 7 or 8. The biological reaction analysis unit analyzes changes in the participant's emotions through analysis of changes in the biological reaction,
10. The reaction information presenting unit according to any one of claims 7 to 9 , wherein the information indicating the change in emotion is presented instead of or in addition to the information indicating the change in biological reaction. Reaction analyzer.

Images