WO2022113248A1 - ビデオミーティング評価端末及びビデオミーティング評価方法 - Google Patents

ビデオミーティング評価端末及びビデオミーティング評価方法 Download PDF

Info

Publication number
WO2022113248A1
WO2022113248A1 PCT/JP2020/044126 JP2020044126W WO2022113248A1 WO 2022113248 A1 WO2022113248 A1 WO 2022113248A1 JP 2020044126 W JP2020044126 W JP 2020044126W WO 2022113248 A1 WO2022113248 A1 WO 2022113248A1
Authority
WO
WIPO (PCT)
Prior art keywords
evaluation
video meeting
video
terminal
moving image
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Ceased
Application number
PCT/JP2020/044126
Other languages
English (en)
French (fr)
Japanese (ja)
Inventor
渉三 神谷
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Imbesideyou Inc
Original Assignee
Imbesideyou Inc
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Imbesideyou Inc filed Critical Imbesideyou Inc
Priority to JP2022515716A priority Critical patent/JP7445331B2/ja
Priority to PCT/JP2020/044126 priority patent/WO2022113248A1/ja
Publication of WO2022113248A1 publication Critical patent/WO2022113248A1/ja
Anticipated expiration legal-status Critical
Ceased legal-status Critical Current

Links

Images

Classifications

    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N5/00—Details of television systems
    • H04N5/76—Television signal recording
    • H04N5/765—Interface circuits between an apparatus for recording and another apparatus

Definitions

  • This disclosure relates to a video meeting evaluation terminal, a video meeting evaluation system, a video meeting evaluation program, and a video meeting evaluation method.
  • Patent Document 1 a system for teaching some knowledge online or giving explanations is known (see, for example, Patent Document 1).
  • the method of measuring the effect by the above-mentioned questionnaire tends to be subjective, and it is insufficient as a method of objectively measuring the effect of the content of the video meeting.
  • an object of the present invention is to objectively evaluate the content of a video meeting in particular.
  • a storage unit that stores moving images obtained from video meetings executed between user terminals, An evaluation target identification unit that identifies the evaluation target of the stored moving image, It is provided with an evaluation unit for calculating at least an evaluation value related to a face image or sound included in the specified moving image.
  • a video meeting evaluation terminal is obtained.
  • the contents of the embodiments of the present disclosure will be listed and described.
  • the present disclosure comprises the following configurations.
  • a storage unit that stores moving images obtained from video meetings executed between user terminals, An evaluation target identification unit that identifies the evaluation target from the stored moving image, It is provided with an evaluation unit for calculating at least an evaluation value related to a face image or sound included in the specified moving image.
  • Video meeting evaluation terminal [Item 2] The video meeting evaluation terminal described in item 1, which is the video meeting evaluation terminal.
  • the evaluation target specifying unit specifies whether or not the moving image is an evaluation target for each file.
  • Video meeting evaluation terminal [Item 3] The video meeting evaluation terminal described in item 1, which is the video meeting evaluation terminal.
  • the evaluation target specifying unit specifies whether or not it is an evaluation target in units of a frame group consisting of a predetermined number of the frames in at least one moving image.
  • [Item 4] The video meeting evaluation terminal described in item 3, which is the video meeting evaluation terminal.
  • the evaluation target specifying unit specifies at least one of a time including a predetermined voice or a silent time as the frame group.
  • [Item 5] The video meeting evaluation terminal described in item 3, which is the video meeting evaluation terminal.
  • the evaluation target specifying unit specifies at least one of a time when a predetermined object is displayed or a time when a predetermined object is not displayed as the frame group.
  • [Item 6] The video meeting evaluation terminal according to any one of items 1 to 5.
  • the video meeting evaluation terminal provides graph information in chronological order of the evaluation values.
  • [Item 7] The video meeting evaluation terminal according to any one of items 1 to 6.
  • the video meeting evaluation terminal calculates a plurality of evaluation values obtained by evaluating the face image from a plurality of different viewpoints.
  • [Item 8] The video meeting evaluation terminal according to any one of items 1 to 7.
  • the video meeting evaluation terminal calculates the evaluation value together with the sound included in the moving image.
  • [Item 9] The video meeting evaluation terminal according to any one of items 1 to 8.
  • the video meeting evaluation terminal calculates the evaluation value together with an object other than the face image contained in the moving image.
  • [Item 10] The video meeting evaluation terminal according to any one of items 1 to 9.
  • An identifier generation means for generating an identifier associated with the moving image, and an identifier generation means. Further having a sharing means for sharing the identifier and the evaluation value in association with each other.
  • Video meeting evaluation terminal [Item 11] A storage unit that stores moving images obtained from an executed video meeting, Evaluation target identification unit that identifies the evaluation target of the stored moving image, A video meeting evaluation program that makes a video meeting evaluation terminal function as an evaluation unit that calculates evaluation values related to at least a face image or sound included in the specified moving image.
  • a storage step that stores video images obtained from video meetings performed between user terminals, Evaluation target identification step to specify the evaluation target of the stored moving image, Including an evaluation step of calculating an evaluation value for at least a facial image or sound contained in the identified moving image.
  • Video meeting evaluation method [Item 13] A video meeting evaluation system including at least a first user terminal and a second user terminal. At least the first user terminal A storage unit that stores moving images obtained from a video meeting executed with the second user terminal, and a storage unit. An evaluation target identification unit that identifies the evaluation target of the stored moving image, It is provided with an evaluation unit for calculating at least an evaluation value related to a face image or sound included in the specified moving image.
  • Video meeting rating system
  • a video meeting evaluation system including at least a first user terminal, a second user terminal, and a server device. At least the server device A storage unit that stores moving images obtained from the video meeting, An evaluation target identification unit that identifies the evaluation target of the stored moving image, It is provided with an evaluation unit for calculating at least an evaluation value related to a face image or sound included in the specified moving image. Video meeting rating system.
  • the emotion analysis system of the present embodiment is unique in that, in an environment where a video meeting (hereinafter referred to as an online session including one-way and two-way) is held by a plurality of people, the analysis target person among the plurality of people is different from the others. It is a system that analyzes emotions (feelings that occur in response to one's own or others' words and actions, such as comfort / discomfort or the degree thereof) and the content of speech.
  • An online session is, for example, an online conference, an online class, an online chat, etc., in which terminals installed in multiple locations are connected to a server via a communication network such as the Internet, and moving images are transmitted between the terminals through the server. It is designed to be able to communicate.
  • the moving images handled in the online session include the face image and voice of the user who uses the terminal.
  • the moving image also includes an image such as a material shared and viewed by a plurality of users. It is possible to switch between the face image and the material image on the screen of each terminal to display only one of them, or to divide the display area and display the face image and the material image at the same time. Further, it is possible to display the image of one of a plurality of people on the full screen, or to display the image of a part or all of the users on a small screen.
  • the leader, facilitator, or administrator of the online session designates any user as the analysis target.
  • Organizers of online sessions include, for example, instructors of online classes, chairs and facilitators of online conferences, and coaches of sessions for coaching purposes.
  • the organizer of an online session is usually one of a plurality of users who participate in the online session, but may be another person who does not participate in the online session.
  • all the participants may be the analysis target without designating the analysis target person.
  • the leader, facilitator, or administrator of the online session (hereinafter collectively referred to as the organizer) to specify any user as the analysis target.
  • Organizers of online sessions include, for example, instructors of online classes, chairs and facilitators of online conferences, and coaches of sessions for coaching purposes.
  • the organizer of an online session is usually one of a plurality of users who participate in the online session, but may be another person who does not participate in the online session.
  • the video meeting evaluation system has at least an input unit such as a camera unit and a microphone unit, a display unit such as a display unit, and an output unit such as a speaker, and is a first user.
  • a user terminal 10 and 20 used by each of the second users, a video meeting service terminal 30 that provides bidirectional video meetings to the user terminals 10 and 20, and an evaluation terminal 40 that evaluates the video meeting.
  • the video meeting according to the present embodiment may be configured so that the user terminals 10 and 20 communicate with each other in a so-called end-to-end manner.
  • FIG. 2 is a diagram showing a hardware configuration example of a computer that realizes each of the terminals 10 to 40 according to the present embodiment.
  • the computer includes at least a control unit 11, a memory 12, a storage 13, a communication unit 14, an input / output unit 15, and the like. These are electrically connected to each other through the bus 16.
  • the control unit 11 is an arithmetic unit that controls the operation of each terminal as a whole, controls the transmission and reception of data between each element, and performs information processing necessary for application execution and authentication processing.
  • the control unit 11 is a processor such as a CPU, and executes each information processing by executing a program or the like stored in the storage 13 and expanded in the memory 12.
  • the memory 12 includes a main storage configured by a volatile storage device such as a DRAM and an auxiliary storage configured by a non-volatile storage device such as a flash memory or an HDD.
  • the memory 12 is used as a work area or the like of the control unit 11, and also stores a BIOS executed when the information sharing support device 10 is started, various setting information, and the like.
  • the storage 13 stores various programs such as application programs.
  • a database storing data used for each process may be built in the storage 13.
  • the video meeting service terminal 30 may record a moving image in an online session and store it in the storage 13.
  • the evaluation terminal 40 may acquire a moving image and store it in the storage 13 managed by the evaluation terminal 40 together with the analysis result (evaluation result).
  • the communication unit 14 connects the information sharing support device 10 to the network.
  • the communication unit 14 directly or network access points with an external device by, for example, a wired LAN, a wireless LAN, Wi-Fi (registered trademark), infrared communication, Bluetooth (registered trademark), short-range or non-contact communication, or the like. Communicate via.
  • the input / output unit 15 is, for example, an information input device such as a keyboard, a mouse, and a touch panel, and an output device such as a display.
  • the bus 16 is commonly connected to each of the above elements and transmits, for example, an address signal, a data signal, and various control signals.
  • the evaluation terminal acquires a moving image from the video meeting service terminal, identifies at least the facial image contained in the moving image for each predetermined frame unit, and calculates the evaluation value for the facial image. (Details will be described later).
  • the video meeting service provided by the video meeting service terminal (hereinafter, may be simply referred to as “the service”) is bidirectionally imaged and audio to the user terminals 10 and 20. Communication is possible.
  • This service displays a moving image acquired by the camera unit of the other user terminal on the display of the user terminal, and can output the sound acquired by the microphone unit of the other user terminal from the speaker.
  • this service is configured to be able to record (record) moving images and audio (collectively referred to as “moving images, etc.”) by either or both user terminals.
  • the recorded information Vs1 and Vs2 (hereinafter referred to as “recorded information”) are temporarily cached in the user terminal that started recording, and are either on the video meeting service terminal side or only locally on either user terminal, or. It will be recorded in both. The user can view the recorded information by himself / herself, share it with others, etc. within the scope of using this service.
  • the evaluation terminal 40 acquires the recorded information and performs analysis and evaluation as described later.
  • the acquisition route differs depending on whether the recorded information is stored on the video meeting service terminal side or the user terminal side (see the dotted arrow in the figure).
  • a download request may be made directly, or an access may be made via a predetermined URL.
  • the evaluation terminal 40 evaluates the moving image acquired as described above by the following analysis.
  • FIG. 4 is a block diagram showing a configuration example used in a series of evaluation processes according to the present embodiment.
  • the video meeting evaluation system of the present embodiment has, as a functional configuration, a video meeting that controls a video meeting via a communication control unit 201 that controls communication with another user terminal and a communication control unit 201.
  • It is provided with an evaluation unit 209 for calculating an evaluation value. Details of the evaluation unit 209 will be described later.
  • the evaluation target specifying unit 207 specifies whether or not the moving image is an evaluation target in file units.
  • the figure shows an example in which a moving image is stored in the video meeting service terminal 30.
  • the moving image is stored in the database with an ID that identifies each moving image, information about the content, and flag information as to whether or not to evaluate.
  • the moving image to which the 1 flag is associated is the evaluation target, and the moving image to which the 0 flag is associated is not the evaluation target.
  • the moving image with the ID "Urawa20201019001” is a moving image related to an online lesson, and the facial expressions and emotions of the students are evaluated for the lecture by the instructor.
  • the moving image with ID "Omiya20201019003” is a moving image related to personal interviews and is excluded from the targets to be evaluated from the viewpoint of personal information and privacy.
  • the flag may be automatically assigned based on the attribute information of the moving image (content, time zone, associated person, etc.), or may be manually assigned.
  • the evaluation target specifying unit 207 has a predetermined number of frame groups t1 to t5 in at least one moving image (the concept of width can be exemplified by, but is not limited to, time, number of frames, etc.). It may be possible to specify whether or not it is an evaluation target for each frame group consisting of. As shown, the frame groups t2 and t4 are evaluated. That is, the evaluation unit 209 evaluates only the frames belonging to the frame groups t2 and t4.
  • the evaluation target specifying unit 207 automatically identifies at least one of the sounded time (or silent time) including a predetermined voice as the frame group to be evaluated. May be good.
  • the evaluation target specifying unit 207 automatically specifies at least one of the times when the predetermined object is displayed (or the time when the predetermined object is not displayed) as the frame group to be evaluated. You may do it. In this case, for example, when the number of people that can be identified in the frame of the moving image is a certain number or more (or a certain amount or less), the frame group may be evaluated (not).
  • the evaluation unit 209 includes a moving image acquisition unit 11, a biological reaction analysis unit 12, a peculiarity determination unit 13, a related event identification unit 14, a clustering unit 15, and an analysis result notification unit 16.
  • Each of the above functional blocks 11 to 16 can be configured by any of hardware, DSP (Digital Signal Processor), and software provided in the evaluation terminal 40, for example.
  • DSP Digital Signal Processor
  • each of the above functional blocks 11 to 16 is actually configured to include a CPU, RAM, ROM, etc. of a computer, and is a program stored in a recording medium such as RAM, ROM, a hard disk, or a semiconductor memory. Is realized by the operation of.
  • the moving image acquisition unit 11 acquires a moving image obtained by shooting a plurality of people (multiple users) with a camera provided in each terminal during an online session. It does not matter whether the moving image acquired from each terminal is set to be displayed on the screen of each terminal. That is, the moving image acquisition unit 11 acquires the moving image from each terminal, including the moving image being displayed on each terminal and the moving image being hidden.
  • the biological reaction analysis unit 12 analyzes changes in the biological reaction of each of a plurality of people based on the moving image (whether or not it is displayed on the screen) acquired by the moving image acquisition unit 11.
  • the biological reaction analysis unit 12 separates the moving image acquired by the moving image acquisition unit 11 into a set of images (a collection of frame images) and a voice, and analyzes changes in the biological reaction from each.
  • the biological reaction analysis unit 12 analyzes the user's face image using the frame image separated from the moving image acquired by the moving image acquisition unit 11, and thereby at least one of the facial expression, the line of sight, the pulse, and the movement of the face. Analyze changes in biological reactions related to one. In addition, the biological reaction analysis unit 12 analyzes changes in the biological reaction regarding at least one of the user's speech content and voice quality by analyzing the voice separated from the moving image acquired by the moving image acquisition unit 11.
  • the biological reaction analysis unit 12 calculates a biological reaction index value that reflects the content of the change in the biological reaction by quantifying the change in the biological reaction according to a predetermined standard.
  • Analysis of changes in facial expressions is performed, for example, as follows. That is, for each frame image, a facial area is specified from the frame image, and the specified facial expressions are classified into a plurality of types according to an image analysis model trained in advance by machine learning. Then, based on the classification result, it is analyzed whether a positive facial expression change occurs between consecutive frame images, a negative facial expression change occurs, and how large the facial expression change occurs. The facial expression change index value according to the analysis result is output.
  • Analysis of changes in the line of sight is performed, for example, as follows. That is, for each frame image, the area of the eyes is specified from the frame image, and the orientation of both eyes is analyzed to analyze where the user is looking. For example, it analyzes whether the speaker's face being displayed, the shared material being displayed, or the outside of the screen is being viewed. In addition, it may be possible to analyze whether the movement of the line of sight is large or small, and whether the movement is frequent or infrequent. The change in the line of sight is also related to the degree of concentration of the user.
  • the biological reaction analysis unit 12 outputs the line-of-sight change index value according to the analysis result of the line-of-sight change.
  • Analysis of pulse changes is performed, for example, as follows. That is, for each frame image, the face area is specified from the frame image. Then, using a trained image analysis model that captures the numerical value of the face color information (G in RGB), the change in the G color on the face surface is analyzed. By arranging the results along the time axis, a waveform showing the change in color information is formed, and the pulse is specified from this waveform. When a person is nervous, the pulse becomes faster, and when he / she feels calm, the pulse becomes slower.
  • the biological reaction analysis unit 12 outputs a pulse change index value according to the analysis result of the pulse change.
  • Analysis of changes in facial movement is performed, for example, as follows. That is, for each frame image, the area of the face is specified from the frame image, and the orientation of the face is analyzed to analyze where the user is looking. For example, it analyzes whether the speaker's face being displayed, the shared material being displayed, or the outside of the screen is being viewed. In addition, it may be possible to analyze whether the movement of the face is large or small, and whether the movement is frequent or infrequent. The movement of the face and the movement of the line of sight may be combined and analyzed. For example, it may be possible to analyze whether the speaker's face being displayed is viewed straight, whether the speaker is viewed with an upper eye or a lower eye, or whether the speaker is viewed from an angle.
  • the biological reaction analysis unit 12 outputs a face orientation change index value according to the analysis result of the face orientation change.
  • the content of the statement is analyzed as follows, for example. That is, the biological reaction analysis unit 12 converts the voice into a character string by performing a known voice recognition process on the voice for a specified time (for example, a time of about 30 to 150 seconds), and performs morphological analysis of the character string. By doing so, words that are unnecessary for expressing conversation such as auxiliary words and acronyms are removed. Then, the remaining words are vectorized, and whether a positive emotional change is occurring, a negative emotional change is occurring, and how large the emotional change is occurring is analyzed, and the analysis result is used. Outputs the statement content index value.
  • Voice quality analysis is performed as follows, for example. That is, the biological reaction analysis unit 12 identifies the acoustic characteristics of the voice by performing a known voice analysis process on the voice for a specified time (for example, a time of about 30 to 150 seconds). Then, based on the acoustic characteristics, it is analyzed whether a positive voice quality change is occurring, a negative voice quality change is occurring, and how loud the voice quality change is occurring, and according to the analysis result. Outputs the voice quality change index value.
  • the biological reaction analysis unit 12 uses at least one of the facial expression change index value, the line-of-sight change index value, the pulse change index value, the face orientation change index value, the speech content index value, and the voice quality change index value calculated as described above.
  • the biological reaction index value is calculated.
  • the biological reaction index value is calculated by weighting the facial expression change index value, the line-of-sight change index value, the pulse change index value, the face orientation change index value, the speech content index value, and the voice quality change index value.
  • the peculiarity determination unit 13 determines whether or not the change in the biological reaction analyzed for the analysis target person is specific to the change in the biological reaction analyzed for a person other than the analysis target person. In the present embodiment, the peculiarity determination unit 13 compares the changes in the biological reaction analyzed for the analysis target person with those of others based on the biological reaction index values calculated for each of the plurality of users by the biological reaction analysis unit 12. To determine whether it is specific or not.
  • the peculiarity determination unit 13 calculates the variance of the biological reaction index value calculated for each of a plurality of persons by the biological reaction analysis unit 12, and compares the variance with the biological reaction index value calculated for the analysis target person. It is determined whether or not the change in the biological reaction analyzed for the person to be analyzed is specific compared to the other person.
  • the following three patterns can be considered as cases where the changes in the biological reaction analyzed for the person to be analyzed are more specific than those of others.
  • the first is the case where a relatively large change in the biological reaction has occurred in the person to be analyzed, although a particularly large change in the biological reaction has not occurred in the other person.
  • the second is the case where a particularly large change in the biological reaction has not occurred in the subject to be analyzed, but a relatively large change in the biological reaction has occurred in the other person.
  • the third is the case where a relatively large change in the biological reaction occurs in both the analysis target person and the other person, but the content of the change is different between the analysis target person and the other person.
  • the related event identification unit 14 identifies an event occurring with respect to at least one of the analysis subject, another person, and the environment when a change in the biological reaction determined to be specific by the peculiarity determination unit 13 occurs. .. For example, the related event identification unit 14 identifies the behavior of the analysis target person himself / herself from the moving image when a specific change in the biological reaction occurs for the analysis target person. In addition, the related event identification unit 14 identifies the behavior of another person from the moving image when a specific change in the biological reaction occurs for the analysis target person. In addition, the related event identification unit 14 identifies the environment when a specific change in the biological reaction occurs for the analysis target person from the moving image. The environment is, for example, the shared material displayed on the screen, the one reflected in the background of the person to be analyzed, and the like.
  • the clustering unit 15 includes changes in biological reactions determined to be specific by the peculiarity determination unit 13 (for example, one or a combination of eyes, pulse, facial movement, speech content, and voice quality) and the peculiarity.
  • the degree of correlation with the event (event specified by the related event identification unit 14) that occurs when a change in the biological reaction occurs is analyzed, and it is determined that the correlation is above a certain level.
  • the clustering unit 15 clusters the analysis target person or the event in any of a plurality of pre-segmented classifications according to the content of the event, the degree of negativeness, the magnitude of the correlation, and the like.
  • the clustering unit 15 clusters the analysis target person or the event in any of a plurality of pre-segmented classifications according to the content of the event, the degree of positiveness, the magnitude of the correlation, and the like.
  • the analysis result notification unit 16 determines at least one of the changes in the biological reaction determined to be specific by the peculiarity determination unit 13, the event specified by the related event identification unit 14, and the classification clustered by the clustering unit 15. , Notify the designated person of the analysis target (analysis target person or the organizer of the online session).
  • the analysis result notification unit 16 analyzes the analysis target as an event that occurs when a specific change in biological reaction occurs in the analysis target person (any of the above-mentioned three patterns; the same applies hereinafter). Notify the person to be analyzed of the person's own words and actions. As a result, the person to be analyzed can grasp that he / she has different emotions from others when he / she makes a certain word or action. At this time, the change of the specific biological reaction specified for the analysis target person may also be notified to the analysis target person. Further, the change in the biological reaction of the other person to be compared may be further notified to the analysis target person.
  • the analysis target person's own words and actions at that time are notified to the analysis target person. This makes it possible to discover words and behaviors that are well received by others and words and behaviors that are not well received by others, contrary to one's consciousness.
  • the analysis result notification unit 16 is the organizer of the online session, in which the event occurring when the analysis target person undergoes a specific change in the biological reaction different from the others, together with the specific change in the biological reaction. Notify to. This allows the organizer of the online session to know what kind of phenomenon influences what kind of emotional change as a phenomenon peculiar to the designated analysis target person. Then, it becomes possible to take appropriate measures for the analysis target person according to the grasped contents.
  • the analysis result notification unit 16 notifies the organizer of the online session of the event occurring when the analysis target person has a specific change in biological reaction different from that of others or the clustering result of the analysis target person. do.
  • the organizer of the online session can grasp the behavior tendency peculiar to the analysis target person and predict the behavior or state that may occur in the future, depending on which classification the specified analysis target person is clustered into. be able to. Then, it becomes possible to take appropriate measures for the analysis target person.
  • the biological reaction index value is calculated by quantifying the change in the biological reaction according to a predetermined standard, and the analysis target person is analyzed based on the biological reaction index value calculated for each of the plurality of persons.
  • An example of determining whether or not a change in a biological reaction has been made is specific compared to another person has been described, but the present invention is not limited to this example. For example, it may be as follows.
  • the biological reaction analysis unit 12 analyzes the movement of the line of sight for each of a plurality of people and generates a heat map showing the direction of the line of sight.
  • the change in the biological reaction analyzed for the analysis target person is measured by the comparison between the heat map generated for the analysis target person by the biological reaction analysis unit 12 and the heat map generated for the other person. It is determined whether or not it is specific by comparing with the change in the biological reaction analyzed for.
  • the video meeting evaluation system of the present embodiment includes a moving image acquisition unit 11, a biological reaction analysis unit 12, and a reaction information presentation unit 13a as functional configurations.
  • the reaction information presentation unit 13a presents information indicating changes in the biological reaction analyzed by the biological reaction analysis unit 12a, including participants who are not displayed on the screen.
  • the reaction information presentation unit 13a presents information indicating changes in the biological reaction to the leader, facilitator, or manager of the online session (hereinafter collectively referred to as the organizer).
  • Organizers of online sessions include, for example, instructors of online classes, chairs and facilitators of online conferences, and coaches of sessions for coaching purposes.
  • the organizer of an online session is usually one of a plurality of users who participate in the online session, but may be another person who does not participate in the online session.
  • the organizer of the online session can also grasp the state of the participants who are not displayed on the screen in the environment where the online session is held by multiple people.
  • the device described in the present specification may be realized as a single device, or may be realized by a plurality of devices (for example, a cloud server) which are partially or wholly connected by a network.
  • the control unit 11 and the storage 13 of the information sharing support device 10 may be realized by different servers connected to each other by a network.
  • the series of processes by the apparatus described in the present specification may be realized by using any of software, hardware, and a combination of software and hardware. It is possible to create a computer program for realizing each function of the information sharing support device 10 according to the present embodiment and implement it on a PC or the like. It is also possible to provide a computer-readable recording medium in which such a computer program is stored.
  • the recording medium is, for example, a magnetic disk, an optical disk, a magneto-optical disk, a flash memory, or the like. Further, the above computer program may be distributed, for example, via a network without using a recording medium.
  • ⁇ Structure 1> A video meeting evaluation system that analyzes specific emotions of the person to be analyzed among the multiple people in an environment where online sessions are held by multiple people.
  • ⁇ Structure 2> A moving image acquisition unit that acquires moving images obtained by shooting the above multiple people during the online session, and a moving image acquisition unit. Based on the moving image acquired by the moving image acquisition unit, the biological reaction analysis unit that analyzes the change in the biological reaction for each of the plurality of persons, and the biological reaction analysis unit. It is provided with a peculiarity determination unit for determining whether or not the change in the biological reaction analyzed for the analysis target person is specific to the change in the biological reaction analyzed for others other than the analysis target person.
  • a video meeting rating system that features that.
  • the biological reaction analysis unit analyzes changes in the biological reaction related to at least one of facial expression, line of sight, pulse, and facial movement by analyzing the facial image in the moving image acquired by the moving image acquisition unit.
  • ⁇ Structure 4> The biological reaction analysis unit is characterized in that it analyzes changes in the biological reaction regarding at least one of the content of speech and voice quality by analyzing the voice in the moving image acquired by the moving image acquisition unit. Or the video meeting rating system described in 3.
  • the biological reaction analysis unit calculates the biological reaction index value by quantifying the change in the biological reaction according to a predetermined standard.
  • the change in the biological reaction analyzed for the analysis target person is different from the analysis target person based on the biological reaction index value calculated for each of the plurality of persons by the biological reaction analysis unit.
  • the video meeting evaluation system according to any one of configurations 2 to 4, wherein it is determined whether or not it is specific with respect to the change in the biological reaction analyzed for another person.
  • the peculiarity determination unit calculates the dispersion of the biological reaction index value calculated for each of the plurality of persons by the biological reaction analysis unit, and the biological reaction index value calculated for the analysis target person and the dispersion.
  • Configuration 5 Described in Configuration 5, characterized in that it is determined by comparison whether or not the change in the biological reaction analyzed for the subject to be analyzed is specific to the change in the biological reaction analyzed for the other person.
  • Video meeting rating system. ⁇ Structure 7>
  • the biological reaction analysis unit analyzes the movement of the line of sight for each of the plurality of people and generates a heat map showing the direction of the line of sight.
  • the peculiarity determination unit is a change in the biological reaction analyzed for the analysis target person by comparing the heat map generated for the analysis target person by the biological reaction analysis unit with the heat map generated for the other person.
  • the video meeting evaluation system according to the configuration 3 is characterized in that it determines whether or not it is specific with respect to the change in the biological reaction analyzed for the other person.
  • ⁇ Structure 8> A related event identification unit that identifies an event occurring with respect to at least one of the analysis target person, the other person, and the environment when a change in a biological reaction determined to be specific by the specificity determination unit occurs.
  • ⁇ Structure 9> The degree of correlation between the change in the biological reaction determined to be specific by the specific biological reaction unit and the event occurring when the specific biological reaction change occurs is analyzed, and the correlation is at a certain level.
  • ⁇ Structure 10> Analysis result notification to notify the analysis target person or the organizer of the online session of at least one of the change in the biological reaction determined to be specific by the specificity determination unit and the event specified by the related event identification unit.
  • ⁇ Structure 11> At least one of the changes in the biological reaction determined to be specific by the peculiarity determination unit, the event specified by the related event identification unit, and the classification clustered by the clustering unit can be analyzed by the person to be analyzed or the above.
  • ⁇ Structure 12> In an environment where an online session is held by multiple participants, the above is based on the moving image obtained by shooting the above participants regardless of whether or not the participants are displayed on the screen during the online session.
  • a reaction analysis system that analyzes the reactions of participants and presents the analysis results.
  • ⁇ Structure 13> A moving image acquisition unit that acquires a moving image obtained by photographing the participants during the online session, and a moving image acquisition unit. Based on the moving image acquired by the moving image acquisition unit, the biological reaction analysis unit that analyzes the change in the biological reaction of the participants, and the biological reaction analysis unit.
  • the item 12 comprises a reaction information presenting unit that presents information indicating changes in the biological reaction analyzed by the biological reaction analysis unit, including participants not displayed on the screen.
  • ⁇ Structure 14> The biological reaction analysis unit analyzes changes in the biological reaction related to at least one of facial expression, line of sight, pulse, and facial movement by analyzing the facial image in the moving image acquired by the moving image acquisition unit.
  • Item 13 The reaction analysis system according to Item 13.
  • ⁇ Structure 15> Item 13 characterized in that the biological reaction analysis unit analyzes changes in the biological reaction regarding at least one of the content of speech and voice quality by analyzing the voice in the moving image acquired by the moving image acquisition unit.
  • the reaction analysis system according to item 14 ⁇ Structure 16> The reaction analysis system according to item 13, wherein the biological reaction analysis unit analyzes where the participants who are not displayed on the screen are looking at the shared material displayed on the screen.
  • ⁇ Structure 17> The reaction analysis system according to item 13, wherein the biological reaction analysis unit analyzes at what timing during the online session a participant who is not displayed on the screen makes a voice.
  • ⁇ Structure 18> The reaction analysis system according to any one of items 13 to 17, wherein the reaction information presenting unit presents information indicating changes in the biological reaction to the organizer of the online session.

Landscapes

  • Engineering & Computer Science (AREA)
  • Multimedia (AREA)
  • Signal Processing (AREA)
  • Two-Way Televisions, Distribution Of Moving Picture Or The Like (AREA)
PCT/JP2020/044126 2020-11-26 2020-11-26 ビデオミーティング評価端末及びビデオミーティング評価方法 Ceased WO2022113248A1 (ja)

Priority Applications (2)

Application Number Priority Date Filing Date Title
JP2022515716A JP7445331B2 (ja) 2020-11-26 2020-11-26 ビデオミーティング評価端末及びビデオミーティング評価方法
PCT/JP2020/044126 WO2022113248A1 (ja) 2020-11-26 2020-11-26 ビデオミーティング評価端末及びビデオミーティング評価方法

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
PCT/JP2020/044126 WO2022113248A1 (ja) 2020-11-26 2020-11-26 ビデオミーティング評価端末及びビデオミーティング評価方法

Publications (1)

Publication Number Publication Date
WO2022113248A1 true WO2022113248A1 (ja) 2022-06-02

Family

ID=81755711

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/JP2020/044126 Ceased WO2022113248A1 (ja) 2020-11-26 2020-11-26 ビデオミーティング評価端末及びビデオミーティング評価方法

Country Status (2)

Country Link
JP (1) JP7445331B2 (https=)
WO (1) WO2022113248A1 (https=)

Citations (6)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
JP2008204193A (ja) * 2007-02-20 2008-09-04 Nippon Telegr & Teleph Corp <Ntt> コンテンツ検索・推薦方法、コンテンツ検索・推薦装置およびコンテンツ検索・推薦プログラム
JP2018068618A (ja) * 2016-10-28 2018-05-10 株式会社東芝 感情推定装置、感情推定方法、感情推定プログラム、および感情カウントシステム
JP2019061594A (ja) * 2017-09-28 2019-04-18 株式会社野村総合研究所 会議支援システムおよび会議支援プログラム
JP2019058625A (ja) * 2017-09-26 2019-04-18 株式会社エモスタ 感情読み取り装置及び感情解析方法
JP2020013428A (ja) * 2018-07-20 2020-01-23 本田技研工業株式会社 表情推定装置および表情推定方法
JP2020160598A (ja) * 2019-03-25 2020-10-01 トヨタ紡織株式会社 情報処理装置、制御システム、情報処理装置の制御方法

Family Cites Families (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
JP6600126B2 (ja) * 2014-03-25 2019-10-30 フクダ電子株式会社 生体信号処理装置およびその制御方法
CN110807394A (zh) * 2019-10-23 2020-02-18 上海能塔智能科技有限公司 情绪识别方法、试驾体验评估方法、装置、设备与介质

Patent Citations (6)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
JP2008204193A (ja) * 2007-02-20 2008-09-04 Nippon Telegr & Teleph Corp <Ntt> コンテンツ検索・推薦方法、コンテンツ検索・推薦装置およびコンテンツ検索・推薦プログラム
JP2018068618A (ja) * 2016-10-28 2018-05-10 株式会社東芝 感情推定装置、感情推定方法、感情推定プログラム、および感情カウントシステム
JP2019058625A (ja) * 2017-09-26 2019-04-18 株式会社エモスタ 感情読み取り装置及び感情解析方法
JP2019061594A (ja) * 2017-09-28 2019-04-18 株式会社野村総合研究所 会議支援システムおよび会議支援プログラム
JP2020013428A (ja) * 2018-07-20 2020-01-23 本田技研工業株式会社 表情推定装置および表情推定方法
JP2020160598A (ja) * 2019-03-25 2020-10-01 トヨタ紡織株式会社 情報処理装置、制御システム、情報処理装置の制御方法

Also Published As

Publication number Publication date
JPWO2022113248A1 (https=) 2022-06-02
JP7445331B2 (ja) 2024-03-07

Similar Documents

Publication Publication Date Title
WO2022024194A1 (ja) 感情解析システム
JP2025089509A (ja) ビデオミーティング評価システム及びビデオミーティング評価サーバ
JP7445331B2 (ja) ビデオミーティング評価端末及びビデオミーティング評価方法
WO2022145043A1 (ja) ビデオミーティング評価端末、ビデオミーティング評価システム及びビデオミーティング評価プログラム
JP7477909B2 (ja) ビデオミーティング評価端末、ビデオミーティング評価システム及びビデオミーティング評価プログラム
JP7629227B2 (ja) ビデオミーティング評価システム及びビデオミーティング評価サーバ
JP7629226B2 (ja) ビデオミーティング評価端末、ビデオミーティング評価システム及びビデオミーティング評価プログラム
JP7688923B2 (ja) ビデオミーティング評価システム及びビデオミーティング評価サーバ
JP7724550B2 (ja) ビデオミーティング評価システム及びビデオミーティング評価サーバ
JP7471683B2 (ja) 反応通知システム
JP7465012B2 (ja) ビデオミーティング評価端末、ビデオミーティング評価システム及びビデオミーティング評価プログラム
JP7465013B2 (ja) ビデオミーティング評価端末、ビデオミーティング評価システム及びビデオミーティング評価プログラム
WO2022064619A1 (ja) ビデオミーティング評価システム及びビデオミーティング評価サーバ
JP7388768B2 (ja) 動画像分析プログラム
JP7849877B2 (ja) ビデオセッション評価端末、ビデオセッション評価システム及びビデオセッション評価プログラム
JP7100938B1 (ja) 動画像分析プログラム
JP7705176B2 (ja) 動画像分析システム
JP7734987B2 (ja) 動画像分析システム
JP7734989B2 (ja) 動画像分析システム
JP7121436B1 (ja) 動画像分析プログラム
JP7691149B2 (ja) 動画像分析システム
JP7691150B2 (ja) 動画像分析システム
JP7734983B2 (ja) 動画像分析システム
JP7121433B1 (ja) 動画像分析プログラム
WO2022145041A1 (ja) ビデオミーティング評価端末、ビデオミーティング評価システム及びビデオミーティング評価プログラム

Legal Events

Date Code Title Description
ENP Entry into the national phase

Ref document number: 2022515716

Country of ref document: JP

Kind code of ref document: A

121 Ep: the epo has been informed by wipo that ep was designated in this application

Ref document number: 20963513

Country of ref document: EP

Kind code of ref document: A1

NENP Non-entry into the national phase

Ref country code: DE

122 Ep: pct application non-entry in european phase

Ref document number: 20963513

Country of ref document: EP

Kind code of ref document: A1