JP2024053975A - Conference support device and method - Google Patents

Abstract

[Problem] We propose a conference support device and method capable of presenting useful information to each participant in real time. [Solution] A conference support method executed by a conference support device that supports a conference, which judges the behavior and state of each participant based on the output of each sensor that acquires information on the behavior and state of each participant, determines information to be presented to the participants based on the determined behavior and state of the participant and each participant's previously set position, and presents the determined information to the participant via a terminal provided on the corresponding participant's side. [Selected Figure] Figure 13

Description

The present invention relates to a conference support device and method, and is suitable for use in a conference support system that supports online conferences and hybrid conferences, for example.

With the recent trend towards DX (Digital Transformation), there is a demand for visualization of meetings (including workshops). In addition, while traditionally most meetings were held face-to-face, in recent years online meetings have been increasingly held due to the global spread of infectious diseases. Furthermore, hybrid meetings that combine face-to-face and online meetings are also becoming widely used.

Regarding visualization of a meeting, the following Patent Document 1 discloses that an avatar corresponding to a first user displayed on the terminal of a second user on the other side is made to move in a movement pattern corresponding to the non-verbal behavior of the first user.

Patent document 2 also discloses that in an online video conference, sensors such as a camera and a microphone are used to sense each of the conference attendees, and based on information on the movements of each attendee obtained by sensing, the speaker of each statement in the voice data is identified, and the voice data of each attendee is output as a timeline in the chronological order of the statements.

International Publication No. 2020/203999 JP 2019-061594 A

As disclosed in Patent Documents 1 and 2, when participating in an online conference, it is difficult to see the expressions and behavior of other participants, so the speech (audio), images, and various sensor information of other participants are visualized.

However, there was an issue that the amount of information displayed was so large that it was difficult for the participant to understand the information they needed. There was also an issue that the information displayed only covered the entire meeting or the other party, and did not take into account the participant's own actions or status at the time, meaning that the information was often not particularly useful to the participant.

The present invention has been made in consideration of the above points, and aims to propose a conference support device and method that can present useful information to each participant in real time.

In order to solve this problem, the present invention provides a conference support device that supports a conference, and includes a behavior/state determination unit that determines the behavior/state of each participant in real time based on the output of each sensor that acquires information on the behavior/state of each participant, a presentation information determination unit that determines information to be presented to the participants based on the behavior/state of the participants determined by the behavior/state determination unit and the previously set position of each participant, and a presentation processing unit that presents the information determined by the presentation information determination unit to the participants via a terminal provided on the corresponding participant's side.

The present invention also provides a conference support method executed by a conference support device that supports a conference, which includes a first step of determining the behavior and state of each participant in real time based on the output of each sensor that acquires information on the behavior and state of each participant, a second step of determining information to be presented to the participant based on the determined behavior and state of the participant and the previously set position of each participant, and a third step of presenting the determined information to the participant via a terminal provided on the corresponding participant's side.

According to the conference support device and method of the present invention, information according to the participant's position and the participant's behavior and status at that time is presented to the participant in real time via the participant's terminal.

The present invention provides a meeting support device and method that can present useful information to each participant in real time.

1 is a block diagram showing an overall configuration of a conference support system according to an embodiment of the present invention; FIG. 2 is a block diagram showing a logical configuration of the conference support device. 13 is a diagram showing an example of the configuration of a participant attribute management table. 11 is a diagram showing an example of the configuration of a sensing information management table. 13 is a diagram showing an example of the configuration of a determination action/state management table. 13A is a diagram showing an example of the configuration of a display rule management table for a facilitator, and FIG. 13B is a diagram explaining information used when determining whether each display item satisfies the display requirements. 13A is a diagram showing an example of the configuration of a display rule management table for superiors, and FIG. 13B is a diagram explaining information used when determining whether each display item satisfies a display requirement. 13A is a diagram showing an example of the configuration of a display rule management table for subordinates, and FIG. 13B is a diagram explaining information used when determining whether each display item satisfies the display requirements. 13A is a diagram showing an example of the configuration of a display rule management table for guests, and FIG. 13B is a diagram explaining information used when determining whether each display item satisfies a display requirement. 13 is a diagram showing an example of the configuration of a display requirement management table; 11 is a diagram showing an example of the configuration of a display information management table; 13 is a flowchart showing a processing procedure of an action/state determination process. 13 is a flowchart showing a processing routine for display information determination processing.

One embodiment of the present invention will be described in detail below with reference to the drawings.

(1) Configuration of the conference support system according to the present embodiment In Fig. 1, the conference support system according to the present embodiment is generally designated by reference numeral 1. This conference support system 1 is configured with a face-to-face participant system 2 installed at the venue of the conference (including a workshop, etc.), an online participant system 3 installed at each participant who participates in the conference online, and a conference support device 4.

The face-to-face participant system 2 is composed of a sensor group 10A that acquires various information about the behavior and state of each participant participating in the face-to-face conference through sensing, and a plurality of display devices 14A provided corresponding to each of these participants. The sensor group 10A is composed of an imaging device 11A, a sound collection device 12A, and a vibration sensor 13A provided corresponding to each participant participating in the face-to-face conference.

The imaging device 11A is composed of a general-purpose camera device and is used to capture the facial expressions of the corresponding participants. However, instead of providing an imaging device 11A for each participant who will be attending the meeting face-to-face, it is also possible to use, for example, a 360° camera. Each imaging device 11A transmits the video signal obtained by capturing the image to the meeting support device 4.

The sound collection device 12A is composed of a general-purpose microphone and is used to collect the sound emitted by the corresponding participant. However, instead of providing a sound collection device 12A for each participant who participates in the meeting face-to-face, for example, a multi-channel microphone array that can identify the direction of the sound source may be used. Each sound collection device 12A transmits an audio signal corresponding to the collected sound to the meeting support device 4.

The vibration sensor 13A is composed of, for example, an acceleration sensor, and is installed on the seat of the chair on which each participant who participates in the meeting face-to-face sits. The vibration sensor 13A detects vibrations caused by the participant nodding when seated in the chair on which the vibration sensor 13A is installed, and outputs a vibration signal corresponding to the detected vibration to the meeting support device 4.

The display device 14A is composed of, for example, a small LCD monitor and is used to display necessary information. Each display device 14A is installed near the seating position of the corresponding participant so that the participant can visually view the displayed information.

The online participant system 3 is composed of a sensor group 10B that acquires various information about the actions and status of the corresponding participants participating in the online conference by sensing, and a display device 14B that displays the conference screen and various other information to the participants, and is connected to the conference support device 4 via the network 5. The sensor group 10B is composed of an imaging device 11B, a sound collection device 12B, and a vibration sensor 13B.

The configurations and installation states of the imaging device 11B, sound collection device 12B, vibration sensor 13B, and display device 14B are similar to those of the imaging device 11A, sound collection device 12A, vibration sensor 13A, and display device 14A of the face-to-face participant system 2, so a description thereof will be omitted here. The video signal output from the imaging device 11B, the audio signal output from the sound collection device 12B, and the vibration signal output from the vibration sensor 13B are each transmitted to the conference support device 4 via the network 5.

The conference support device 4 is a computer device that has the function of determining the current behavior and status of each participant in real time based on various information regarding the behavior and status of each participant collected by the sensor group 10A of the face-to-face participant system 2 and the sensor group 10B of each online participant system 3, and displaying information for each participant that corresponds to the attributes of that participant and the behavior and status of that participant at that time on the participant's display device 14A, 14B as appropriate.

This conference support device 4 is configured with information processing resources such as a CPU (Central Processing Unit) 20, a memory 21, a storage device 22, and an input device 23. The CPU 20 is a processor that controls the operation of the entire conference support device 4. The memory 21 is configured, for example, from a volatile semiconductor memory, and is used as a working memory for the CPU 20.

The storage device 22 is composed of a non-volatile large-capacity storage device such as a hard disk drive or SSD (Solid State Drive), and is used to store various programs and data that needs to be stored for a long period of time. When the conference support device 4 is started up or when needed, the necessary programs are loaded from the storage device 22 into the memory 21, and the CPU 20 executes the programs loaded into the memory 21, thereby performing various processes by the conference support device 4 as a whole, as described below.

In this embodiment, the memory 21 stores an image processing program 30, a voice processing program 31, a nod detection program 32, a behavior/state determination program 33, a display information determination program 34, and a display processing program 35, which are loaded from the storage device 22 and will be described later, and the storage device 22 stores an information management database 36 consisting of a number of management tables, as will be described later.

The input device 23 is composed of, for example, a mouse and a keyboard, and is used by the participant in charge of the conference (hereinafter referred to as the facilitator or facilitator) to input various data and various operations to the conference support device 4.

Figure 2 shows the logical configuration of the conference support device 4 according to this embodiment. As shown in this Figure 2, the conference support device 4 includes an image processing unit 40, an audio processing unit 41, a nod detection unit 42, an action/state determination unit 43, a display information determination unit 44, and a display processing unit 45 as functional units, and includes a participant attribute management table 50, a sensing information management table 51, a determined action/state management table 52, a plurality of display rule management tables 53, a display requirement management table 54, and a display information management table 55 stored as management tables in the information management database 36.

In the following, it is assumed that information regarding which imaging devices 11A, 11B, sound collection devices 12A, 12B, vibration sensors 13A, 13B, and display devices 14A, 14B are associated with each participant who will participate in the conference, is provided in advance to the image processing unit 40, the audio processing unit 41, the nodding detection unit 42, and the display processing unit 45. In the following, it is assumed that each participant is assigned a unique identifier (hereinafter referred to as a participant ID).

The image processing unit 40 is a functional unit that is realized by the CPU 20 (Fig. 1) executing the image processing program 30 of Fig. 1 that is loaded into the memory 21 (Fig. 1). The image processing unit 40 judges the facial expression of each participant at a predetermined judgment period (e.g., 1 second period) based on the video signals provided by each of the imaging devices 11A, 11B, and when it detects a smile or an expression that suggests the participant is about to speak (e.g., an expression such as silently opening and closing the mouth as if the participant is about to express an opinion), it notifies the behavior/state judgment unit 43 of that fact and the participant ID of that participant.

The image processing unit 40 also determines the facial direction of each participant attending the conference face-to-face based on the video signals provided by each imaging device 11A of the face-to-face participant system 2 at the above-mentioned determination period, and if it determines that any participant is looking at another participant, notifies the behavior/state determination unit 43 of the participant ID of that participant and the participant ID of the other participant that the participant was looking at.

The audio processing unit 41 is a functional unit that is realized by the CPU 20 executing the audio processing program 31 of FIG. 1 that is loaded into the memory 21. When any participant speaks, the audio processing unit 41 measures the volume of the speech at the above-mentioned determination period based on the audio signals provided by each of the sound collection devices 12A, 12B, and notifies the behavior/state determination unit 43 of the participant ID of the participant who made the speech and the measured volume of the speech.

The nodding detection unit 42 is a functional unit that is realized by the CPU 20 executing the nodding detection program 32 of FIG. 1 that is loaded into the memory 21. The nodding detection unit 42 analyzes the vibration signals provided by each of the vibration sensors 13A, 13B at the above-mentioned determination period, and when it detects that any of the participants is nodding, it notifies the behavior/state determination unit 43 of that fact and the participant ID of that participant.

The behavior/state determination unit 43 is a functional unit that is realized by the CPU 20 executing the behavior/state determination program 33 of FIG. 1 that is loaded into the memory 21. The behavior/state determination unit 43 sequentially stores and manages each piece of information notified from the image processing unit 40, the audio processing unit 41, and the nod detection unit 42 in the above-mentioned determination cycle in a sensing information management table 51 described below. The behavior/state determination unit 43 also determines the behavior/state of each participant at that time in the above-mentioned determination cycle based on each piece of information stored in the sensing information management table 51, and registers and manages the determination results in the determined behavior/state management table 52.

In this embodiment, the behavior/state determination unit 43 determines whether the participant is performing one of the following behaviors: "talking," "not talking and listening attentively," and "not talking and not interested/doing side work."

"Speaking" refers to a state in which the corresponding participant is speaking. "Not speaking and listening attentively" refers to a state in which the corresponding participant is not speaking (not speaking) and is listening attentively to what the participant who is currently speaking (hereinafter referred to as the speaker) is saying. "Not speaking and not interested/doing side work" refers to a state in which the corresponding participant is not speaking (not speaking) and is not interested in what the speaker is saying at the time or is doing work unrelated to the meeting.

The display information determination unit 44 is a functional unit that is realized by the CPU 20 executing the display information determination program 34 of FIG. 1 that is loaded into the memory 21. The display information determination unit 44 determines, for each participant, at the above-mentioned determination period, whether or not the current situation is one in which some information should be displayed on the participant's display device 14A, 14B, based on the information stored in the above-mentioned sensing information management table 51 and the determination action/state management table 52, and the information stored in the participant attribute management table 50, the display rule management tables 53, and the display requirement management table 54, which will be described later.

When the display information determination unit 44 determines that a certain information should be displayed for any of the participants, it refers to the display information management table 55 described below to determine the information to be displayed on the display device 14A, 14B of that participant at that time, and notifies the display processing unit 45 of the determined information together with the participant ID of that participant.

The display processing unit 45 is a functional unit that is realized by the CPU 20 executing the display processing program 35 of FIG. 1 that is loaded into the memory 21. Based on the information notified by the display information determination unit 44 and the participant ID, the display processing unit 45 transmits the information to the display device 14A, 14B of the participant to whom the participant ID has been assigned. As a result, the information is displayed on the display device 14A, 14B of the participant.

Meanwhile, the participant attribute management table 50 is a table used to manage the attributes of each participant that have been set in advance. Here, "participant attributes" refers to the position of the participant in the conference. In this embodiment, for example, if the conference is an internal conference, four attributes are defined as such "participant attributes (positions)": "facilitator" who acts as the moderator, "superior" who has a higher position among the participants, "subordinate" who has a lower position than the "superior" among the participants, and "guest" who is a guest-like presence such as a client.

As shown in FIG. 3, the participant attribute management table 50 is configured with a participant ID column 50A and an attribute column 50B. In the participant attribute management table 50, one row corresponds to one participant. The participant ID column 50A stores the participant ID of the corresponding participant, and the attribute column 50B stores the preset attribute (position) of that participant in the conference.

Therefore, in the example of Figure 3, the attribute (position) of the participant assigned the participant ID "0001" is "facilitator," and the attribute (position) of the participant assigned the participant ID "0002" is "supervisor."

The sensing information management table 51 is a table used to store information based on each notification provided by the image processing unit 40, the audio processing unit 41, and the nodding detection unit 42 at the determination period as described above, separately for each participant, and is generated for each participant.

As shown in FIG. 4, the sensing information management table 51 is configured with an elapsed time column 51A, a facial expression column 51B, a direction column 51C, a speech column 51D, a volume column 51E, a speech time column 51F, and a nod column 51G. In the sensing information management table 51, one record (row) corresponds to one analysis result notified from the image processing unit 40, the audio processing unit 41, and the nod detection unit 42 in the above-mentioned judgment cycle. In the following, the timing that arrives in such a judgment cycle and at which the image processing unit 40, the audio processing unit 41, and the nod detection unit 42 analyze the video signal, audio signal, or nod detection signal is referred to as the data analysis timing.

The elapsed time column 51A stores the time that has elapsed since the start of the conference at the corresponding data analysis timing. The facial expression column 51B stores a value corresponding to the facial expression of the corresponding participant detected by the image processing unit 40 during the period from the previous data analysis timing to the corresponding data analysis timing (hereinafter, this is referred to as the most recent data analysis period). For example, if the image processing unit 40 detects a smiling face as the facial expression of the corresponding participant during the most recent data analysis period, a value of "1" is stored in the facial expression column 51B; if the image processing unit 40 detects a facial expression that suggests the participant is about to speak, a value of "2" is stored; and otherwise a value of "0" is stored in the facial expression column 51B.

In addition, the orientation column 51C stores the participant ID of the other participant notified to the behavior/state determination unit 43 by the image processing unit 40 when the image processing unit 40 determines that the face of the corresponding participant who participated in the meeting face-to-face during the immediately preceding data analysis period was facing the other participant (i.e., the corresponding participant was looking at the other participant).

The speech column 51D stores a flag (hereinafter, referred to as a speech flag) indicating whether the voice processing unit 41 detected the corresponding participant's speech during the immediately preceding data analysis period. In the example of FIG. 4, if the voice processing unit 41 detected the corresponding participant's speech during the data analysis period, the speech flag is set to "1," and if not, the speech flag is set to "0."

The volume column 51E stores the volume of the speech (speech volume) measured by the audio processing unit 41 when the audio processing unit 41 detects the speech of the corresponding participant during the immediately preceding data analysis period, and the speech time column 51F stores the duration from when the speech started.

Furthermore, the nod column 51G stores a flag (hereinafter, referred to as a nod flag) indicating whether the nod detection unit 42 detected that the corresponding participant nodded during the immediately preceding data analysis period. In the example of FIG. 4, if the nod detection unit 42 detected that the corresponding participant nodded during the data analysis period, the nod flag is set to "1," and if not, the nod flag is set to "0."

Therefore, in the example of Figure 4, the participant corresponding to this Figure 4 does not smile or look as if he or she is about to speak, nor does he or she speak, for example, during the period from "00:00:01" to "00:00:02" after the start of the conference, but turns his or her face toward the participant with participant ID "1", and is detected to have nodded at least "1" time during this period.

The determined action/state management table 52 is a table used to manage the action or state that each participant was performing at that time, which was determined at a determination period by the action/state determination unit 43 based on the information stored in the sensing information management table 51, and as shown in FIG. 5, is configured with a participant ID column 52A and a determined action/state column 52B. In the determined action/state management table 52, one record (row) corresponds to one participant.

The participant ID column 52A stores the participant ID of the corresponding participant, and the determined action/status column 52B stores the action that the participant was performing at that time or the state of the participant at that time, as determined by the action/status determination unit 43.

In the case of this embodiment, as described above, the behavior/state determination unit 43 determines whether the participant is "talking," "not talking and listening attentively," or "not talking and not interested/doing side work," so in principle, the determined behavior/state column 52B of each record stores one of the following information: "talking," "not talking and listening attentively," "not talking and not interested/doing side work," or "-," which means that none of these apply.

However, if the participant is a "facilitator," and the behavior/state determined by the behavior/state determination unit 43 is "not speaking and listening attentively" or "not speaking and not interested/doing side work," the information "not speaking" is stored in the determined behavior/state column 52B (see the determined behavior column for participant ID "0001" in Figure 5).

Therefore, for a "facilitator," if the information "not speaking" is stored in the corresponding judgment behavior/status column 52B, this means that the behavior at that time was either "not speaking and listening attentively" or "not speaking and not interested/doing side work."

On the other hand, the display rule management table 53 is a table used to manage the display rules of predetermined information, such as what items of information are displayed on the display devices 14A and 14B of participants when what attributes (positions) of participants take what actions or are in what states and what requirements are met.

There are four types of display rule management tables 53, one for "facilitator," one for "supervisor," one for "subordinate," and one for "guest," each corresponding to a different attribute of a participant. Each of these four types of display rule management tables 53 is created in advance and stored in the information management database 37.

Figure 6 (A) shows the display rule management table 53A for the "facilitator." As shown in Figure 6 (A), the display rule management table 53A for the "facilitator" is composed of a first area 53AA associated with the action/state of the "facilitator" being speaking, and a second area 53AB associated with the action/state of the "facilitator" not speaking.

The first area 53AA is composed of one or more columns (hereinafter referred to as display item columns) 53AAA corresponding to each item (hereinafter referred to as display items) whose information should be displayed on the display device 14A, 14B of the "facilitator" depending on the situation when the "facilitator" is currently speaking. In this embodiment, when the "facilitator" is speaking, the only display item whose information is displayed on the display device 14A, 14B of the "facilitator" is the display item "Number of people who agree: 1", so the first area 53AA is composed only of the display item column 53AAA "Number of people who agree: 1". Note that "Number of people who agree: 1" is a display item whose content displays the number of participants who agree with what the "facilitator" is saying on the display device 14A, 14B of the "facilitator" while the "facilitator" is speaking.

Similarly, the second area 53AB is composed of one or more display item columns 53ABA, 53ABB, 53ABC, each corresponding to a display item that displays information on the display device 14A, 14B of the "facilitator" depending on the situation when the "facilitator" is not speaking. In the present embodiment, when the "facilitator" is not speaking, three display items, "Level of excitement," "Number of people who agree: 2," and "People who want to speak," are specified as display items that should display information corresponding to the "facilitator"'s display device 14A, 14B depending on the situation. Therefore, the second area 53AB is composed of three display item columns 53ABA to 53ABC, "Level of excitement," "Number of people who agree: 2," and "People who want to speak."

"Excitement level" is a display item that displays the degree of excitement of the meeting (excitement level) due to the speech of participants other than the "facilitator" on the "facilitator's" display devices 14A and 14B. Details of "excitement level" will be described later. "Number of participants who agree 2" is a display item that displays the number of other participants who agree with the speech of participants other than the "facilitator" on the "facilitator's" display devices 14A and 14B, and "Someone wants to speak" is a display item that displays the participant ID of a participant who looks like they want to speak (want to speak) on the "facilitator's" display devices 14A and 14B.

Furthermore, each of the display item columns 53AAA, 53ABA to 53ABC in the first and second areas 53AA, 53AB is divided into a display item column 53AC, a display priority column 53AD, and a display information column 53AE, respectively.

The display item column 53AC stores the names of the display items corresponding to the display item columns 53AAA, 53ABA to 53ABC, and the display priority column 53AD stores the display priority of the display items. The display priority is the priority when displaying the information of the corresponding display item, and is determined independently for each case in which the participant to whom the display rule management table 53 applies is speaking and not speaking.

When a participant to whom the display rule management table 53 applies is speaking or not speaking and a situation arises in which information on multiple display items should be displayed, only the information on the display item with the highest display priority is displayed on the participant's display device, and information on the other display items is not displayed. Furthermore, if there are multiple display items with the highest display priority, information on all of these display items is displayed on the participant's display devices 14A and 14B.

Furthermore, the display information number column 53AE stores an identification number (hereinafter referred to as the display information number) unique to the information to be displayed when the display requirements for displaying the information of the corresponding display item are met. When the display requirements for any display item are met, the display information number stored in the display information number column 53AE of the display item column 53AAA, 53ABA to 53ABC corresponding to that display item is used as a search key to search for the information to be displayed at that time in the display information management table 55 described below, and the detected information is displayed on the display device 14A, 14B of the participant to which the display rule management table 53 is applied.

Meanwhile, Figures 7(A), 8(A) and 9(A) show display rule management tables 53B, 53C and 53D for "Superior", "Subordinate" and "Guest", respectively. As shown in Figures 7(A) to 9(A), the basic configuration of each of the display rule management tables 53B to 53D for "Superior", "Subordinate" and "Guest" is the same as the display rule management table 53A for "Facilitator".

However, if the attribute of a participant is "superior," "subordinate," or "guest," three display items, "Number of consenters 1," "Support," and "Speech duration," are specified as display items that display information corresponding to the participant's display device 14A, 14B depending on the situation when the participant is speaking.

For this reason, in the display rule management tables 53B to 53D for "Superior," "Subordinate," or "Guest," the first areas 53BA, 53CA, 53DA are made up of display item columns 53BAA, 53CAA, 53DAA corresponding to the display item "Number of consenters: 3," display item columns 53BAB, 53CAB, 53DAB corresponding to the display item "Support," and display item columns 53BAC, 53CAC, 53DAC corresponding to the display item "Speech time."

Note that "Many Agreeees 1" is a display item that, when it is determined that there are many other participants who agree with the content of a speech made by a participant ("Superior", "Subordinate" or "Guest") to whom the display rule management table 53 (53B to 53D) applies, displays the number of such participants on the display devices 14A and 14B of the participant.

"Boost" is a display item whose content is to display a message encouraging a participant to speak on the display device 14A, 14B when the participant to whom the display rule management table 53 (53B to 53D) applies is speaking. "Speech duration" is a display item whose content is to display the duration of the speech on the display device 14A, 14B when the participant to whom the display rule management table 53 (53B to 53D) applies is speaking.

In addition, when the attribute of a participant is "superior," "subordinate," or "guest," the behavior/state of not speaking is subdivided into the behavior/states of "listening attentively" and "not interested/doing work from home." "Listening attentively" corresponds to the behavior/state of the participant "not speaking and listening attentively," and "not interested/doing work from home" corresponds to the behavior/state of the participant "not speaking and not interested/doing work from home."

When a participant to whom the display rule management table 53 (53B to 53D) is applied is in the behavior/state of "not speaking and listening attentively," two display items, "interjection" and "influence/effect," are specified as display items that display information corresponding to the participant's display device 14A, 14B depending on the situation.

In addition, if the target participant is in the "not speaking and not interested/doing side work" behavior, a display item called "non-listening time" is defined as a display item that displays information corresponding to the target participant's display device depending on the situation.

For this reason, in the display rule management table 53 (53B-53D) for "Superior", "Subordinate" or "Guest", the second area 53BB, 53CB, 53DB is further divided into areas 53BBX, 53CBX, 53DBX for "Listening attentively" and areas 53BBY, 53CBY, 53DBY for "Not interested/doing side work".

The "listening attentively" areas 53BBX, 53CBX, 53DBX are made up of display item columns 53BAB, 53CBA, 53DBA corresponding to the display item "interfering" and display item columns 53BBB, 53CBB, 53DBB corresponding to the display item "influence/effect." The "not interested/doing side work" areas 53BBY, 53CBY, 53DBY are made up of display item columns 53BBC, 53CBC, 53DBC corresponding to the display item "not interested/doing side work."

"Interjection" is a display item that displays a message on the display device 14A, 14B of a target participant encouraging the target participant to speak when other participants finish speaking, when the target participant is not speaking and looks as if he or she wants to speak. "Influence/Effect" is a display item that displays the number of participants who have spoken on a topic that the target participant started, and the duration of the topic to date, while the discussion on that topic is ongoing, on the display device 14A, 14B of the target participant. "Non-listening time" is a display item that displays the duration of the behavior/state (hereinafter referred to as "non-listening time") when the target participant is taking an action or being in a state that suggests that he or she is not interested in the conference or is performing work unrelated to the conference.

In addition, in the display rule management tables 53 (53B, 53C, 53D) for "Superior", "Subordinate" and "Guest", for display items marked as "Hidden" in the display information columns 53BE, 53CE, 53DE of the display item columns (53BAA-53BAC, 53BBA-53BBC), (53CAA-53CAC, 53CBA-53CBC), (53DAA-53DAC, 53DBA-53DBC), no determination is made as to whether the current situation is such that the information for that display item should be displayed on the display devices 14A, 14B of the target participant, and as a result, the information for these display items is not displayed on the display devices 14A, 14B of the target participant.

On the other hand, the display requirements management table 54 is a table used to manage the requirements (hereinafter referred to as display requirements) for displaying the information of each display item defined in each display rule management table 53 (53A-53D) for "facilitator," "supervisor," "subordinate," and "guest" on the target participant's display device 14A, 14B. The display requirements for each display item are determined in advance and are stored and managed in advance in the display requirements management table 54.

As shown in FIG. 10, the display requirements management table 54 is configured with a display item column 54A and a display requirements column 54B. In the display requirements management table 54, one record (row) corresponds to one display item.

The display item column 54A stores the name of the corresponding display item, and the display requirements column 54B stores the requirements (display requirements) for displaying the information of that display item on the display devices 14A and 14B of the target participants.

Therefore, in the example of Figure 10, for example, the display item "Number of Agreeing Participants: 1" is a display item whose content is to display the number of other participants who agree with what the target participant (here, only the "facilitator") is saying while that participant is speaking, so it is indicated that the display requirement is that the target participant is speaking.

The determination of whether the display requirements for each display item are met (whether the situation is such that the information for that display item can be displayed) is made using the information marked with an "O" in Figures 6(B), 7(B), 8(B), and 9(B).

6(B) to 9(B), the "Facial Expression," "Direction," "Speech," "Speech Time," and "Nod" columns 53AG, 53BG, 53CG, and 53DG in the "Self" columns 53AF, 53BF, 53CF, and 53DF correspond to the facial expression column 51B, direction column 51C, speech column 51D, volume column 51E, speech time column 51F, and nod column 51G in the sensing information management table 51 (FIG. 4) of the target participant (here, the "facilitator"). The "Facial Expression," "Direction," "Speech," "Volume," "Speech Time," and "Nodding" columns 53AI, 53BI, 53CI, and 53DI in the "Others" columns 53AH, 53BH, 53CH, and 53DH correspond to the facial expression column 51B, direction column 51C, speech column 51D, volume column 51E, speech time column 51F, and nodding column 51G in the sensing information management table 51 of participants other than the target participant, respectively.

Therefore, for example, when determining whether the display requirements for displaying information for a display item such as "Number of people who agree: 1" have been met, the system will determine that the target participant ("facilitator") is speaking by referring to the value of the speech flag stored in the speech column 51D of the sensing information management table 51 for the target participant, and will count the number of people who agree with the content of the target participant's speech using the information stored in the facial expression column 51B, direction column 51C, and nod column 51G of the sensing information management table 51 for each participant other than the target participant.

The display information management table 55 is a table used to manage the correspondence between the display information numbers stored in the display information number columns 53AE, 53BE, 53CE, and 53DE of each display rule management table 53 (53A to 53D) and the specific information that should actually be displayed on the corresponding participant's display device 14A or 14B, and is created in advance and stored in the information management database 37.

As shown in FIG. 11, the display information management table 55 is configured with a display information number column 55A and a display information column 55B. The display information column 55B stores the specific content of the information to be displayed on the corresponding participant's display device 14A, 14B, and the display information number column 55A stores a display information number that is assigned to the information and is unique to that information.

In FIG. 11, information in parentheses means that the specific numerical value of the information is to be displayed on the display device 14A, 14B of the corresponding participant, and information in quotation marks means that the message in the quotation marks is to be displayed on the display device 14A, 14B of the corresponding participant.

Therefore, in the example of Figure 11, the information assigned with a display information number of "1" is the "number of people who agree" (i.e., the number of participants who agree with what the speaker said), and the display information assigned with a display information number of "4" is the message "Like!".

(2) Specific Processing Contents of Each Functional Unit Next, specific processing contents of each functional unit of the above-mentioned conference support device 4 will be described. However, the processing contents of the image processing unit 40, the audio processing unit 41, the nod detection unit 42, and the display processing unit 45 are general-purpose processing contents using existing technology, so their description will be omitted. Here, in particular, the method of determining the behavior of each participant by the behavior/state determination unit 43 and the method of determining display information by the display information determination unit 44 will be described.

(2-1) Specific Processing Contents of the Behavior/State Determination Unit The behavior/state determination unit 43 uses the sensing information management table 51 (Figure 4) of each participant to determine the current behavior of each participant at each of the above-mentioned data analysis timings as follows.

The behavior/state determination unit 43 first refers to the speech column 51D of the latest record in the sensing information management table 51 of the participant to be processed (target participant). If the speech flag stored in the speech column 51D is "1", the behavior/state determination unit 43 determines that the current behavior of the target participant is speaking.

Furthermore, if the speech flag is "0", the behavior/state determination unit 43 determines that the target participant is currently not speaking. If it is determined that the target participant is not speaking, the behavior/state determination unit 43 refers to the speech column 51D of the latest record in the sensing information management table 51 for each participant other than the target participant, and determines whether each participant other than the target participant is speaking in the same manner as described above. If the behavior/state determination unit 43 determines that any participant other than the target participant is speaking, it determines whether the target participant is seriously listening to that participant (speaker).

In this embodiment, if the target participant nods at least once since the speaker started speaking, or if the target participant has been facing the speaker for a certain percentage (e.g., 90%, hereafter referred to as the first percentage) or more since the speaker started speaking (limited to face-to-face conference participants), the target participant is deemed to be listening to the speaker attentively.

The behavior/state determination unit 43 first refers to the sensing information management table 51 of the speaker to identify the time when the speaker started speaking (hereinafter, this is called the speech start time). This speech start time is the earliest time at which the value of the speech flag stored in the speech column 51D of the sensing information management table 51 started to be "1" continuously or almost continuously up to the current time.

The behavior/state determination unit 43 then refers to the sensing information management table 51 of the target participant to determine whether the nod flag stored in the nod column 51G of at least one of the records from the record of the speaker's speech start time to the latest record is set to "1", and whether the proportion of records in which the speaker's speaker ID is stored in the direction column 51C of the records from the record of the speaker's speech start time to the latest record is equal to or greater than a first proportion.

If a positive result is obtained in at least one of the former and latter determinations, the behavior/state determination unit 43 determines that the target participant's current behavior is "not speaking and listening attentively." Note that such determination of the target participant's behavior/state based on the direction of the face is limited to face-to-face conference participants. The same applies below.

In addition, when the behavior/state determination unit 43 determines that any participant other than the target participant is speaking, it determines whether the target participant is not interested in the content of the speaker or is performing work unrelated to the meeting (side work).

In this embodiment, if the target participant has not nodded even once since the speaker started speaking, or if the target participant has been facing the speaker for a certain percentage or less (e.g., 20%, hereafter referred to as the second percentage) since the speaker started speaking, it is determined that the target participant is not interested in what the speaker is saying or is engaged in work unrelated to the meeting (side work).

The behavior/state determination unit 43 then refers to the sensing information management table 51 of the target participant to determine whether the nod flags stored in the nod column 51G of each record from the record of the speaker's speech start time to the latest record are all set to "0" and whether the proportion of records in which the speaker's speaker ID is stored in the direction column 51C of each record from the record of the speaker's speech start time to the latest record is less than or equal to a second proportion.

If a positive result is obtained in at least one of the former and latter judgments, the behavior/state judgment unit 43 judges that the target participant's current behavior is "not speaking and not interested, or doing side work."

The behavior/state determination unit 43 executes the above behavior determination process for all participants to determine whether the behavior of each participant at that time corresponds to any of "talking," "not talking and listening attentively," and "not talking and not interested or doing side work," and stores the determination results in the above-mentioned determined behavior/state management table 52 (Figure 5).

(2-2) Specific Processing Contents of the Display Information Determination Unit The display information determination unit 44 acquires the attributes of the target participant from the participant attribute management table 50 (FIG. 3). The display information determination unit 44 also acquires the current behavior of the target participant determined by the behavior/state determination unit 43 ("talking", "not talking and listening seriously", "not talking and not interested/doing side work", or "-") from the determined behavior/state management table 52 (FIG. 5).

Then, based on the acquired attributes of the target participant and the current behavior and state of the target participant, the display information determination unit 44 refers to the display rule management table 53 according to the attributes of the target participant and determines whether there is a display item for which display information corresponding to the target participant's display device 14A, 14B should be displayed according to the situation.

If there is such a display item, the display information determination unit 44 obtains the display requirements of the display item from the display requirements management table 54 and determines whether the current situation satisfies the display requirements. If the display information determination unit 44 determines that the current situation satisfies the display requirements, it determines the information to be displayed on the display devices 14A and 14B of the target participant as follows, and outputs the determined information to the display processing unit 45.

(2-2-1) When the target participant is a "facilitator" and is "speaking" When the attribute of the target participant is "facilitator" and the current action of the target participant is "speaking," when referring to the display rule management table 53A for "facilitator" (Figure 6), the only corresponding display item is "Number of consenters: 1."

As described above, "Number of Agreeing Parties: 1" is a display item that displays the number of other participants who agree with what the "facilitator" is saying on the "facilitator's" display devices 14A, 14B while the "facilitator" is speaking. With reference to the display requirement management table 54 (Figure 10), it is stipulated that the display requirement for "Number of Agreeing Parties: 1" is that the target participant ("facilitator") is speaking. In this case, since the "facilitator" is speaking, the display information determination unit 44 determines that the current situation meets the display requirement for "Number of Agreeing Parties: 1".

Therefore, the display information determination unit 44 reads the display information number stored in the display information number column 53AE of the display information column 53AA (Fig. 6) for "Number of Agreeees 1" in the display information management table 53A for the "facilitator," and obtains the display information assigned with that display information number from the display information management table 55 (Fig. 11). In the example of Fig. 11, the display information is the "number of agreeees," that is, the number of participants who agree with what the "facilitator" said, so the display information determination unit 44 counts the number of other participants who agree with what the "facilitator" said.

In this embodiment, if the other participants have nodded at least once since the speaker started speaking, or if the other participants are facing the speaker and have a smile on their face, it is determined that the other participants agree with what the speaker has said.

The display information determination unit 44 then refers to the sensing information management table 51 (Figure 4) of each participant other than the "facilitator," and counts the total number of other participants who agree with the content of the "facilitator's" speech as the number of other participants who agree with the content of the "facilitator's" speech, including the other participants whose numerical value stored in the facial expression column 51B is "1" and whose participant ID of the "facilitator" is stored in the direction column 51C, and the other participants whose nod flag stored in the nod column 51G of at least one of the current records in the corresponding sensing information management table 51 is "1" from the time the "facilitator" started speaking.

The display information determination unit 44 then determines the count result as the display information to be displayed on the display devices 14A and 14B of the target participant, and outputs this count result together with the participant ID of the target participant to the display processing unit 45.

(2-2-2) When the target participant is a "facilitator" and "not speaking" When the attribute of the target participant is "facilitator" and the current behavior/status of the target participant is "not speaking"("not speaking and listening attentively" or "not speaking and not interested/doing side work"), referring to the display rule management table 53A for the "facilitator," in the example of Figure 6, three items are specified as display items when the "facilitator" is not speaking: "Level of excitement,""Number of people who agree: 2," and "People wanting to speak."

In addition, the display items "Level of excitement," "Number of people who agree 2," and "People who wish to speak" all have a display priority of "1," so for all display items that satisfy the display requirements, it is necessary to display display information corresponding to the display devices 14A and 14B of the target participants. Therefore, for each of these three display items, the display information determination unit 44 determines whether the current situation satisfies the display requirements for that display item, and determines display information for display items that satisfy the display requirements.

For example, the display item "Excitement level" is a display item that displays the excitement level of the conference based on the speech of participants other than the target participant (here, the "facilitator") as described above. With reference to the display requirement management table 54, it is stipulated that a participant other than the target participant is speaking as a display requirement for "Excitement level." The display information determination unit 44 then determines whether or not the current situation satisfies this display requirement.

Specifically, the display information determination unit 44 refers to the sensing information management table 51 of each participant other than the target participant, and checks whether there is a participant whose speech flag stored in the speech column 51D of the latest record is set to "1". If such a participant is present, the display information determination unit 44 determines that the current situation satisfies the display requirements for the display item "excitement level".

When the display information determination unit 44 determines that the current situation meets the display requirements for the display item "excitement level," it reads the display information number stored in the display information number column 53AE of the display information column 53ABA for "excitement level" in the display rule management table 53A for the "facilitator," and obtains the information assigned with that display information number from the display information management table 55. In FIG. 11, since "excitement level" is the display information, the display information determination unit 44 calculates the excitement level of the current meeting in the immediately preceding data analysis period.

In this embodiment, the degree of excitement during a data analysis period is calculated by normalizing and adding together the total speech volume of all participants during the immediately preceding data analysis period, the number of participants who had a smiling expression during that data analysis period, and the number of participants who nodded during that data analysis period.

The display information determination unit 44 then refers to the sensing information management table 51 of each participant other than the "facilitator," reads all the volumes stored in the volume column 51E of the latest records in these sensing information management tables 51, adds them up, and normalizes them. The display information determination unit 44 also normalizes the number of participants other than the "facilitator" who have a "1" stored in the facial expression column 51B of the latest record in the corresponding sensing information management table 51. Furthermore, the display information determination unit 44 normalizes the number of participants other than the "facilitator" who have a nod flag set to "1" in the nod column 51G. The display information determination unit 44 then calculates the sum of the values obtained by these normalizations as the excitement level for that data analysis period.

The display information determination unit 44 then determines the excitement level calculated in this manner as the display information to be displayed on the display devices 14A and 14B of the target participant (the "facilitator"), and outputs this excitement level together with the participant ID of the target participant to the display processing unit 45.

The display item "Number of Agreeing Parties 2" is a display item that displays the number of other participants who agree with the content of the speech of a participant other than the "facilitator," as described above. With reference to the display requirement management table 54, it is stipulated that a participant other than the "facilitator" is speaking as a display requirement for "Number of Agreeing Parties 2." The display information determination unit 44 then determines whether or not the current situation satisfies this display requirement.

Specifically, the display information determination unit 44 refers to the sensing information management table 51 of each participant other than the "facilitator" and checks whether there is a participant whose speech flag stored in the speech column 51D of the latest record is set to "1". If such a participant is present, the display information determination unit 44 determines that the current situation meets the display requirements for the display item "Number of consenters: 2".

When the display information determination unit 44 determines that the current situation meets the display requirements for the display item "Number of Agreeing Parties 2," it reads the display information number stored in the display information number column 53AE of the display information column 53ABB for "Number of Agreeing Parties 2" in the display information management table 53A for the "facilitator," and obtains the display information assigned with that display information number from the display information management table 55. In the example of FIG. 11, the display information is "Number of Agreeing Parties," that is, the number of other participants who agree with the content of the speaker at that time. Therefore, the display information determination unit 44 counts the number of participants who agree with the content of the speaker at that time during the immediately preceding data analysis period.

Specifically, the display information determination unit 44 refers to the sensing information management table 51 of each participant other than the speaker, and counts the total number of participants whose numerical value stored in the facial expression column 51B is "1" and whose participant ID of the current speaker is stored in the orientation column 51C, and whose nod flag stored in the nod column 51G of at least one record of the corresponding sensing information management table 51 from the time the current speaker started speaking, as the number of other participants who agree with the content of the current speaker's speech.

The display information determination unit 44 then determines the count result as the display information to be displayed on the display device 14A, 14B of the target participant ("facilitator"), and outputs this count result together with the participant ID of the target participant to the display processing unit 45.

The display item "Someone wants to speak" is a display item that displays the participant ID of a participant who looks like they want to speak, as described above. With reference to the display requirement management table 54, the display requirements for "Someone wants to speak" are that a participant other than the "facilitator" is speaking, and that there is a participant who looks like they want to speak. The display information determination unit 44 then determines whether or not the current situation meets these display requirements.

Specifically, the display information determination unit 44 determines whether any participant other than the "facilitator" is speaking, in the same way as for "Number of consenters: 2". If the determination returns a positive result, the display information determination unit 44 refers to the sensing information management table 51 of each participant other than the "facilitator" and the current speaker, and determines whether there is a participant whose most recent record in the corresponding sensing information management table 51 has a numerical value of "2" stored in the facial expression column 51B. If such a participant exists (if there is a participant who appears to want to speak), the display information determination unit 44 determines that the current situation meets the display requirements for the display item "Someone wishes to speak".

When the display information determination unit 44 obtains a determination result that the display requirements for the display item "Someone wants to speak" are met, it reads out the display information number stored in the display information number column 53AE of the display information column 53ABC for "Someone wants to speak" in the display information management table 53A for the "facilitator," and obtains the display information assigned with that display information number from the display information management table 55. In FIG. 11, the display information is the participant ID of the participant who looks like he or she wants to speak, so the display information determination unit 44 obtains the participant ID of that participant.

Then, the display information determination unit 44 determines the acquired participant ID as the display information to be displayed on the display devices 14A and 14B of the target participant, and outputs this participant ID together with the participant ID of the target participant to the display processing unit 45.

(2-2-3) When the target participant is a “Superior” and is “Speaking” On the other hand, when the attribute of the target participant is “Superior” and the current behavior of the target participant is “Speaking”, when referring to the display rule management table 53B for “Superior”, the corresponding display items are “Number of consenters: 1” and “Speaking time”.

In addition, since the display priority of each of these display items "Number of consenters 1" and "Speech duration" is "1", it is necessary to display the display information corresponding to the display devices 14A and 14B of the target participants for all display items that satisfy the display requirements. Therefore, the display information determination unit 44 determines whether or not the current situation satisfies the display requirements for each of these two display items, and determines the display information for display items that satisfy the display requirements.

For example, the display item "Many Agreeing Participants 1" is a display item that displays the number of other participants who agreed with the content of the target participant (here, the "boss") while the target participant was speaking, if that number is large, as described above. And, referring to the display requirement management table 54, it is specified as a display requirement for "Many Agreeing Participants 1" that there is a large number of other participants who agreed with the content of the target participant's speech.

In this embodiment, for "large number of agreeers 1", if the normalized sum of the number of participants who had a smiling expression during the data analysis period, the number of participants who looked at (turned their faces towards) the target participant during the data analysis period, and the number of participants who nodded during the data analysis period is equal to or greater than a preset threshold (hereinafter referred to as the first number of agreeers threshold), then it is considered that there are a large number of other participants who agree with the content of the target participant's speech.

The display information determination unit 44 then normalizes the number of participants other than the target participant who have "1" stored in the facial expression column 51B of the latest record in the corresponding sensing information management table 51. The display information determination unit 44 also normalizes the number of participants other than the target participant who have the participant ID of the target participant stored in the orientation column 51C of the latest record in the corresponding sensing information management table 51. Furthermore, the display information determination unit 44 normalizes the number of participants other than the target participant who have the nod flag set to "1" in the nod column 51G of the latest record in the corresponding sensing information management table 51.

The display information determination unit 44 then determines whether the sum of the values obtained by these normalizations is equal to or greater than the first consenter number threshold. If the sum is equal to or greater than the first consenter number threshold, the display information determination unit 44 determines that the current situation satisfies the display requirements for the display item "Number of consenters: 1".

When the display information determination unit 44 obtains a determination result that the display requirements for the display item "Many agreeers 1" are met, it reads out the display information number stored in the display information number column 53BE of the display information column 53BAA for "Many agreeers 1" in the display information management table 53B for "Superior", and obtains the display information assigned with that display information number from the display information management table 55. In FIG. 11, since "Number of agreeers" is the display information, the display information determination unit 44 counts the number of participants who agree with the content of the speaker at that time during the immediately preceding data analysis period.

Specifically, the display information determination unit 44 first calculates the total number of participants other than the target participant who have "1" stored in the facial expression column 51B of the latest record in the corresponding sensing information management table 51, the number of participants other than the target participant who have the participant ID of the target participant stored in the orientation column 51C of the latest record in the corresponding sensing information management table 51, and the number of participants other than the target participant who have the nod flag set to "1" in the nod column 51G of the latest record in the corresponding sensing information management table 51.

Next, the display information determination unit 44 subtracts the number of participants who are counted in duplicate from this total value. "Participants who are counted in duplicate" are participants who have "1" stored in the facial expression column 51B of the latest record in the corresponding sensing information management table 51, the participant ID of the target participant is stored in the orientation column 51C of the record, and the nod flag in the nod column 51G of the record set to "0". Also, participants who have "1" stored in the facial expression column 51B of the latest record in the corresponding sensing information management table 51, the participant ID of the target participant is not stored in the orientation column 51C of the record, and the nod flag in the nod column 51G of the record set to "1" are also included. Furthermore, participants who have "0" or "1" stored in the facial expression column 51B of the latest record in the corresponding sensing information management table 51, the participant ID of the target participant is stored in the orientation column 51C of the record, and the nod flag in the nod column 51G of the record set to "1" are also included. Furthermore, this also applies to participants for whom "1" is stored in the facial expression column 51B of the latest record in the corresponding sensing information management table 51, the participant ID of the target participant is stored in the orientation column 51C of that record, and the nod flag in the nod column 51G of that record is set to "1."

The display information determination unit 44 then determines the number of other participants who agree with the speech content of the target participant thus obtained as the display information to be displayed on the display devices 14A and 14B of the target participant (the "boss"), and outputs this number to the display processing unit 45 together with the participant ID of the target participant.

The display item "Speech Time" is a display item whose content is to display the speaking time of the target participant on the display devices 14A and 14B of the target participant, as described above. And when referring to the display requirement management table 54, it is stipulated that the display requirement for "Speech Time" is that the participant is speaking. Therefore, when the target participant is speaking, it is immediately determined that the display requirement for the display item "Speech Time" is met.

When the display information determination unit 44 obtains a judgment result that the display requirements for the display item "speech time" are met, it reads out the display information number stored in the display information number column 53BE of the display information column 53BAC for "speech time" in the display information management table 53B for "boss," and obtains the display information to which the display information number is assigned from the display information management table 55.

In the example of Figure 11, the display information is the elapsed time from when the target participant started speaking to the present, so the display information determination unit 44 obtains the speech time stored in the speech time column 51F of the latest record in the sensing information management table 51 for the target participant.

Then, the display information determination unit 44 determines the acquired speech time as the display information to be displayed on the display devices 14A and 14B of the target participant, and outputs this speech time together with the participant ID of the target participant to the display processing unit.

(2-2-4) When the target participant is a “superior” and is “not speaking and listening attentively” When the attribute of the target participant is “superior” and the current behavior of the target participant is “not speaking and listening attentively”, when referring to the display rule management table 53B for “superior”, the only corresponding display item is “interfering”.

The "Interjection" display item is a display item that displays a message encouraging the target participant to speak when the other participants have finished speaking, when the target participant looks as if they want to speak, as described above. With reference to the display requirement management table 54, the display requirements for "Interjection" are that the target participant looks as if they want to speak, and that the participants other than the target participant have finished speaking. The display information determination unit 44 then determines whether the current situation meets these display requirements.

Specifically, the display information determination unit 44 refers to the sensing information management table 51 of the target participant and determines whether or not "2" is stored in the facial expression column 51B of the latest record. Furthermore, the display information determination unit 44 refers to the sensing information management table 51 of each participant other than the target participant and determines whether or not there is a participant for whom the speech flag stored in the speech column 51D of the record immediately before the latest record is set to "1" and the speech flag stored in the speech column 51D of the latest record is set to "0". If the display information determination unit 44 obtains positive results from both of these two determinations, it determines that the current situation satisfies the display requirements for the display item "Speaking out".

When the display information determination unit 44 determines that the current situation meets the display requirements for the display item "Interference," it reads out the display information number stored in the display information number column 53BE of the display information column 53BBA for "Interference" in the display information management table 53B for "Superior," and obtains the display information to which the display information number is assigned from the display information management table 55.

In the example of FIG. 11, the message "Now's your chance to speak!" is the display information, so the display information determination unit 44 determines this message as the display information to be displayed on the display devices 14A and 14B of the target participant, and outputs the text data of this message together with the participant ID of the target participant to the display processing unit 45.

(2-2-5) When the target participant is a "boss" and "not speaking and not interested/doing side work" When the attribute of the target participant is "boss" and the behavior of the target participant is "not speaking and not interested/doing side work", there is no display item when referring to the display rule management table 53B for "boss". Therefore, in the case of "not speaking and not interested/doing side work", no information is displayed on the display devices 14A and 14B of the target participant regardless of the behavior of the target participant ("boss"). This is because the target participant has the attribute (position) of "boss".

(2-2-6) When the target participant is a “subordinate” and is “speaking” On the other hand, when the attribute of the target participant is “subordinate” and the current behavior of the target participant is “speaking”, when referring to the display rule management table 53C for “subordinate”, the corresponding display items are “Many people who agree: 2” and “Support”.

In addition, since the display priority of each of these display items "Many Agreeees 2" and "Support" is "1", it is necessary to display the display information corresponding to the display devices 14A and 14B of the target participants for all display items that satisfy the display requirements. Therefore, the display information determination unit 44 determines whether or not the current situation for each of these two display items satisfies the display requirements for that display item, and determines the display information for display items that satisfy the display requirements.

For example, the display item "Many Agreeing Parties 2" is a display item that displays the number of other participants who agreed with the content of the target participant (here, "subordinates") while the target participant was speaking, if that number is large, as described above. And when referring to the display requirement management table 54, it is stipulated as a display requirement for "Many Agreeing Parties 2" that there is a large number of other participants other than the target participant who agreed with the content of the target participant's speech.

In this embodiment, for "large number of agreeers 2", if the sum of the number of participants who had a smiling expression during the data analysis period, the number of participants who looked at (turned their faces towards) the target participant during the data analysis period, and the number of participants who nodded during the data analysis period is normalized and equal to or exceeds a preset threshold (a value smaller than the first number of agreeers threshold, hereinafter referred to as the second number of agreeers threshold), it is deemed that there are many other participants who agree with the content of the target participant's utterance. Note that the reason why the second number of agreeers threshold is smaller than the first number of agreeers threshold is to give confidence to participants with the attribute of "subordinate".

Then, the display information determination unit 44 judges whether or not the current situation satisfies the display requirements for the display item "Many Agreeing People 1", in the same way as in the case of "Many Agreeing People 1". Then, if the display information determination unit 44 obtains a judgment result that the display requirements for the display item "Many Agreeing People 2" are satisfied, it counts the number of other participants who agree with the content of the target participant's speech, in the same way as in "Many Agreeing People 1".

The display information determination unit 44 then determines the number of other participants who agree with the speech content of the target participant thus obtained as the display information to be displayed on the display devices 14A and 14B of the target participant ("subordinate"), and outputs this number to the display processing unit 45 together with the participant ID of the target participant.

The display item "Boost" is a display item whose content is to display a message encouraging the target participant to speak on the display devices 14A, 14B of the target participant when the target participant is speaking, as described above. When the display requirement management table 54 is referenced, it is stipulated that the display requirement for "Boost" is that the target participant is speaking. Therefore, when the target participant is speaking, it is immediately determined that the display requirement for the display item "Boost" is met.

If the display information determination unit 44 obtains a judgment result that the display requirements for the display item "support" are met, it reads out the display information number stored in the display information number column 53CE of the display information column 53CAB for "support" in the display information management table 53C for "subordinates," and obtains the display information to which the display information number is assigned from the display information management table 55.

In the example of FIG. 11, the message "Like!" is the display information, so the display information determination unit 44 determines this message as the display information to be displayed on the display devices 14A and 14B of the target participant, and outputs the text data to the display processing unit 45 together with the participant ID of the target participant.

(2-2-7) When the target participant is a “subordinate” and is “not speaking and listening attentively” When the attribute of the target participant is “subordinate” and the current behavior of the target participant is “not speaking and listening attentively”, when referring to the display rule management table 53C for “subordinate”, the corresponding display items are two display items, “interfering” and “influence/effect”.

And in the display item "not speaking and listening attentively," the display priority of the display item "interfering" is set to "1," and the display priority of the display item "not interested/doing side work" is set to "2."

The display information determination unit 44 therefore first determines whether the current situation satisfies the display requirements for the display item "opinion," and if the display requirements are met, determines the display information for that display item.

Specifically, the display information determination unit 44 determines whether the current situation of the target participant satisfies the display requirements for "interfering" in the same way as in the case of "interfering" by the "boss," and if it determines that the current situation satisfies the display requirements for the display item "interfering," determines that the display information to be displayed on the display devices 14A and 14B of the target participant is the message "Now's your chance to speak up!" and outputs the text data to the display processing unit 45 together with the participant ID of the target participant.

On the other hand, if the display information determination unit 44 determines that the current situation does not satisfy the display requirements for the display item "opinion," it determines whether or not the current situation satisfies the display requirements for the display item "impact/effect." If the display information determination unit 44 determines that the display requirements are satisfied, it determines the display information for that display item.

For example, the display item "Influence/Effect" is a display item that displays, on the display device of the target participant, the number of participants who have spoken on a topic initiated by the target participant and the current duration of the topic while the discussion on that topic continues, as described above. And when referring to the display requirement management table 54, it is stipulated as a display requirement for "Influence/Effect" that the topic initiated by the target participant is still ongoing.

In a conference, a "discussion" in which one or more participants speak continuously about a certain topic is usually carried out for a certain period of time or more, after which a certain period of quietness (a period during which none of the participants are speaking) has passed and then a "discussion" about the next topic begins. Therefore, by referring to the sensing information management table 51 of each participant, the most recent quiet period of a certain period of time or more (a period during which none of the participants are speaking) can be detected, and if the target participant speaks at the beginning of the quiet period, it can be determined that the topic initiated by the target participant is still being discussed.

The display information determination unit 44 then refers to the sensing information management table 51 of each participant, and detects a quiet period during which the speech flag stored in the speech column 51D of the sensing information management table 51 of all participants going back from the most recent record is "0". The display information determination unit 44 then determines whether the first person to speak after the end of the quiet period is the target participant.

Specifically, the display information determination unit 44 determines whether the speech flag stored in the speech column 51D of the first record in the sensing information management table 51 of the target participant after the end of the quiet period is "1". If the speech flag is "1", the display information determination unit 44 determines that the target participant is the one who started the current topic, and determines that the display requirements for the display item "influence/effect" are met.

If the display information determination unit 44 obtains a judgment result that the display requirements for the display item "impact/effect" are met, it reads out the display information number stored in the display information number column 53CE of the display information column 53CBB for "impact/effect" in the display information management table 53C for "subordinates," and obtains the display information to which the display information number is assigned from the display information management table 55.

In the example of FIG. 11, the number of participants in the topic and the duration of the topic are the display information, so the display information determination unit 44 calculates the number of participants and the duration, respectively.

Specifically, the display information determination unit 44 first refers to the sensing information management table 51 of the target participant, obtains the elapsed time stored in the elapsed time column 51A of the record at which the current topic started and the elapsed time stored in the elapsed time column 51A of the most recent record, and calculates the duration of the current topic by subtracting the former elapsed time from the latter elapsed time.

The display information determination unit 44 also refers to the speech flag in the speech column 51D of the sensing information management table 51 for each participant, and counts the number of participants whose speech flag is set to "1" in the speech column 51D of at least one record among the records from the record at the time the current topic started to the most recent record. This number is the number of participants who participated in the current topic.

The display information determination unit 44 then determines the number of participants in the current topic and the duration of the current topic obtained in this manner as the display information to be displayed on the display devices 14A and 14B of the target participant, and outputs the number of participants and the duration together with the participant ID of the target participant to the display processing unit 45.

(2-2-8) When the target participant is a “subordinate” and “not speaking and not interested/doing work from home” When the attribute of the target participant is “subordinate” and the current behavior of the target participant corresponds to “not speaking” and “not interested/doing work from home”, when referring to the display rule management table 53C for “subordinate”, the only corresponding display item is “non-listening time”.

The "Non-listening time" display item is a display item that displays the time (non-listening time) that a target participant is engaged in when they are in an action or state that suggests they are not interested in the meeting or are engaged in work other than that related to the meeting, as described above.

Referring to the display requirement management table 54, if the target participant is an online conference participant, the requirement for displaying "non-listening time" is that the target participant has not nodded even once during the most recent fixed period of time, and if the target participant is a face-to-face conference participant, the requirement for displaying "non-listening time" is that the face-to-face conference participant has not looked at the speaker (has not turned his or her face to the speaker) during the most recent fixed period of time.

Then, when the target participant is an online conference participant, the display information determination unit 44 refers to the sensing information management table 51 of the target participant and determines whether all of the nod flags stored in the nod column 51G of each record within the most recent fixed time period are "0". If the display information determination unit 44 obtains a negative result in this determination, it determines that the current situation of the target participant satisfies the display requirements for the display item "non-listening time".

When the target participant is participating in the conference face-to-face, the display information determination unit 44 refers to the sensing information management table 51 of each participant other than the target participant to identify all participants who have spoken within a certain period of time, for each time period. The display information determination unit 44 then refers to the sensing information management table 51 of the target participant to determine whether the participant ID of the speaker during that period of time has never been stored in the orientation column 51C of the record for each period of time within the last certain period of time. When the target participant obtains a positive result in this determination, the display information determination unit 44 determines that the current situation meets the display requirements for the display item "non-listening time".

When the display information determination unit 44 obtains a determination result that the current situation satisfies the display requirements for the display item "Non-listening time", it reads out the display information number stored in the display information number column 53CE of the display information column 53CBC for "Non-listening time" in the display information management table 53C for "subordinates", and obtains the display information assigned with that display information number from the display information management table 55. In the example of FIG. 11, since "Non-listening time" is the display information, the display information determination unit 44 obtains the most recent non-listening time of the target participant up to that point.

Specifically, when the target participant is an online conference participant, the display information determination unit 44 refers to the sensing information management table 51 of the target participant, and obtains the elapsed time stored in the elapsed time column 51A of the oldest record in which the nod flag value stored in the nod column 51G has been consecutively set to "0" going back from the most recent record. The display information determination unit 44 then subtracts the elapsed time obtained as described above from the elapsed time stored in the elapsed time column 51A of the most recent record to calculate the time during which the participant is not nodding as the non-listening time.

Then, the display information determination unit 44 determines the non-listening time calculated in this manner as the display information to be displayed on the display devices 14A and 14B of the target participant, and outputs this non-listening time together with the participant ID of the target participant to the display processing unit 45.

When the target participant is a face-to-face conference participant, the display information determination unit 44 refers to the sensing information management table 51 of the target participant, and obtains the elapsed time stored in the elapsed time column 51A of the oldest record in which the participant ID of a participant who was speaking during that time period is not consecutively stored in the orientation column 51C, going back from the latest record. The display information determination unit 44 then subtracts the elapsed time obtained as described above from the elapsed time stored in the elapsed time column 51C of the latest record, to calculate the time during which the target participant is not facing the speaker as the non-listening time.

Then, the display information determination unit 44 determines the non-listening time calculated in this manner as the display information to be displayed on the display devices 14A and 14B of the target participant, and outputs this non-listening time together with the participant ID of the target participant to the display processing unit 45.

(2-2-9) When the target participant is a “guest” and “talking” When the attribute of the target participant is a “guest” and the current behavior of the target participant is “talking”, the processing content of the display information determination unit is the same as that of a “subordinate” who is “talking”. Therefore, the explanation here is omitted.

(2-2-10) When the target participant is a “guest” and “not speaking and listening attentively” When the attribute of the target participant is a “guest” and the current behavior of the target participant is “not speaking and listening attentively”, when referring to the display rule management table 53D for “guest”, the only corresponding display item is the display item “interfering”.

Then, the display information determination unit 44 determines whether or not the current situation satisfies the display requirements for the display item "Interfering," in the same way as with "Interfering" for "Superior," and if the display requirements are met, determines the display information for that display item to be "Now is your chance to speak up," and outputs the determined display information to the display processing unit 45 together with the participant ID of the target participant.

(2-2-11) When the target participant is a "guest" and "not speaking and not interested/doing side work" When the attribute of the target participant is "guest" and the behavior of the target participant is "not speaking and not interested/doing side work", there are no display items when referring to the display rule management table 53D for "guest". Therefore, in the case of "not speaking and not interested/doing side work", no information is displayed on the display devices 14A and 14B of the target participant regardless of the behavior of the target participant. This is because the target participant has the attribute (position) of "guest".

(2-3) Processing Flow of Action/State Determination Unit and Display Information Determination Unit (2-3-1) Action/State Determination Process Next, the flow of a series of processes (hereinafter, referred to as action/state determination process) executed by the action/state determination unit 43 will be described with reference to Fig. 12. Note that the specific processing contents of the action/state determination unit 43 in each step of Fig. 12 have already been described in detail, so duplicated description will be omitted.

Figure 12 shows the flow of the action/state determination process executed by the action/state determination unit 43 at the above-mentioned determination cycle. The action/state determination unit 43 determines the action of each participant at that time according to the processing procedure shown in this Figure 12.

In practice, when the behavior/state determination unit 43 starts this behavior/state determination process, it first stores the various pieces of information about each participant provided by the image processing unit 40, the audio processing unit 41, and the nodding detection unit 42 in the sensing information management table 51 (Figure 4) for each corresponding participant (S1).

Specifically, the behavior/state determination unit 43 stores a numerical value corresponding to the facial expression of each participant notified by the image processing unit 40 in the facial expression column 51B of the corresponding sensing information management table 51. The behavior/state determination unit 43 also stores the participant ID of the other participant toward whom each participant is facing, notified by the image processing unit 40, in the orientation column 51C of the sensing information management table 51 for that participant.

The behavior/state determination unit 43 also sets the speech flag stored in the speech column 51D of the sensing information management table 51 of that participant to "1" if the participant is speaking based on the information notified by the audio processing unit 41 as to whether or not the participant is speaking, and sets the speech flag to "0" if the participant is not speaking. The behavior/state determination unit 43 also stores the speech volume and speech time of each participant notified by the audio processing unit 41 in the volume column 51E and speech time column 51F, respectively, of the sensing information management table 51 of that participant.

Furthermore, based on the information notified by the nodding detection unit 42 as to whether or not each participant is nodding, the behavior/state determination unit 43 sets the nodding flag stored in the nodding column 51G of the sensing information management table 51 for that participant to "1" if the participant is nodding, and sets the nodding flag to "0" if the participant is not nodding.

Next, the behavior/state determination unit 43 selects one participant who has not been processed after step S3, and determines whether the selected participant is speaking by referring to the speech column 51D of the sensing information management table 51 of the selected participant (hereinafter, in the explanation of Figure 12, this participant will be referred to as the selected participant) (S2).

If the behavior/state determination unit 43 obtains a positive result in this determination, it determines that the behavior/state of the selected participant at that time is "talking" (S4), and stores the determination result in the determined behavior/state management table 52 (Figure 5) in association with the participant ID of the selected participant (S10).

If the behavior/state determination unit 43 obtains a negative result in step S3, it refers to the sensing information management table 51 of the selected participant to determine whether the selected participant is seriously listening to the content of the speaker at that time (S5).

If the behavior/state determination unit 43 obtains a positive result in this determination, it determines that the behavior/state of the selected participant at that time is "not speaking and listening attentively" (S6), and stores the determination result in the determined behavior/state management table 52 in association with the participant ID of the selected participant (S10).

If the behavior/state determination unit 43 obtains a negative result in step S5, it refers to the sensing information management table 51 of the selected participant to determine whether the selected participant is not interested in the content of the speaker at that time or is performing work unrelated to the conference (S7).

If the behavior/state determination unit 43 obtains a positive result in this determination, it determines the behavior/state of the selected participant at that time as "not speaking and not interested/doing side work" (S8), and stores the determination result in the determined behavior/state management table 52 in association with the participant ID of the selected participant (S10).

In response to this, if the behavior/state determination unit 43 obtains a negative result in the determination in step S7, it determines that the selected participant's behavior/state at that time does not correspond to any of "talking," "not talking and listening attentively," or "not talking and not interested/doing side work" (S9), and stores the determination result in the determined behavior/state management table 52 in association with the participant ID of the selected participant (S10). In this case, "-" is stored in the determined behavior/information column 52B of the record corresponding to the selected participant in the determined behavior/state management table 52 (see the record with participant ID "0004" in Figure 5).

Then, the behavior/state determination unit 43 determines whether the processing of steps S3 to S10 has been completed for all participants (S11). If the behavior/state determination unit 43 obtains a negative result in this determination, it returns to step S2, and thereafter repeats the processing of steps S2 to S11 while sequentially switching the participant selected in step S2 to another participant who has not yet been processed in steps S3 and after.

This repeated process sequentially determines the behavior and status of each participant at that time, and the determined behavior and status are stored sequentially in the determined behavior and status management table 52.

Then, when the behavior/state determination unit 43 has finished determining the behavior/state of all participants at that time and has finished storing the determination results in the determined behavior/state management table 52, it obtains a positive result in step S11 and terminates this behavior/state determination process.

(2-3-2) Display Information Determination Process Next, the flow of a series of processes (hereinafter, referred to as the display information determination process) executed by the display information determination unit 44 will be described with reference to Fig. 13. Note that the specific process contents of the display information determination unit in each step of Fig. 13 have already been described in detail, so duplicated descriptions will be omitted.

Figure 13 shows the flow of the display information determination process executed by the display information determination unit 44 at the above-mentioned judgment period. The display information determination unit 44 determines the information to be displayed at that time on the display devices 14A and 14B of the necessary participants according to the processing procedure shown in Figure 13.

In practice, when the display information determination unit 44 starts this display information determination process, it first selects one participant who has not been processed after step S21 (S20). The display information determination unit 44 also obtains the attributes of the selected participant (hereinafter, in the explanation of FIG. 13, this participant will be referred to as the selected participant) from the participant attribute management table 50 (FIG. 3), and obtains the current behavior and status of the selected participant from the determined behavior and status management table 51 (FIG. 4) (S21).

Next, based on the attributes and current actions/state of the selected participant acquired in step S21, the display information determination unit 44 refers to the display rule management table 53 corresponding to the attributes of the selected participant, and identifies the display items for which information should be displayed on the display devices 14A and 14B of the selected participant depending on the situation (S22).

Then, in step S22, the display information determination unit 44 judges whether or not it has been possible to identify a display item for which information should be displayed on the display device 14A, 14B of the selected participant depending on the situation (S23). If the display information determination unit 44 is unable to identify a display item in this judgment (for example, in the case of "Boss" being "Not speaking and not interested/doing side work"), it proceeds to step S29.

In response to this, if the display information determination unit 44 obtains a positive result in the determination of step S23, it identifies all of the display items identified in step S22 that have the highest display priority (S24). Furthermore, for each display item identified in step S24, the display information determination unit 44 refers to the display requirement management table 54 (FIG. 10) and determines whether or not the current situation satisfies the display requirement for displaying information of that display item (S25). If the display information determination unit 44 obtains a negative result in this determination, it proceeds to step S29.

In response to this, if the display information determination unit 44 obtains a positive result in the determination of step S25, it refers to the display information management table 55 (FIG. 11) and identifies the information to be displayed for all display items for which a positive result has been obtained (S26). The display information determination unit 44 also reads out the identified information from the display information management table 55 or obtains it by calculation (S27).

The display information determination unit 44 then outputs the information acquired in step S27 to the display processing unit 45 together with the participant ID of the selected participant (S28), and then determines whether the processing from step S21 onwards has been executed for all participants (S29).

If the display information determination unit 44 obtains a negative result in this determination, it returns to step S20, and thereafter repeats the processing of steps S20 to S29 while sequentially switching the participant selected in step S20 to other participants who have not yet been processed in steps S21 and after.

By this repeated process, for all participants whose information should be displayed on the corresponding display devices 14A and 14B, the information to be displayed at that time and the participant ID of the participant are sequentially transmitted to the display processing unit 45. As a result, the information is displayed on the display devices 14A and 14B of these participants.

Then, when the display information determination unit 44 eventually completes the processing of steps S21 to S28 for all participants and obtains a positive result in step S29, it ends the display information determination process.

(3) Effects of this embodiment As described above, the conference support device 4 of this embodiment determines the current behavior and status of each participant in real time based on various information regarding the behavior and status of each participant collected by the sensor group 10A of the face-to-face participant system 2 and the sensor group 10B of each online participant system 3, and displays, for each participant, information corresponding to the attributes of that participant and the behavior and status of that participant at that time on the participant's display devices 14A, 14B as appropriate.

Therefore, according to the conference support device 4, information according to the participant's position and the participant's current behavior and state is presented to each participant in real time via the participant's terminal, making it possible to present useful information to each participant in real time.

(4) Other embodiments In the above embodiment, the display devices 14A and 14B are arranged on each participant side as terminals for presenting information (hereinafter, these are referred to as information presentation devices), and the necessary information is displayed on the display devices 14A and 14B to present to the necessary participants. However, the present invention is not limited to this. For example, robots may be arranged as such information presentation terminals instead of the display devices 14A and 14B on some or all of the participants' sides, and information may be presented to the participants by the movement of the robot rather than by display. Furthermore, a display device capable of displaying information may be provided on the robot, and information may be presented not only by the movement of the robot but also by display. Furthermore, a speaker may be provided on the robot so that the robot can speak, and messages and other information may be presented to the participants by voice or sound.

In the above embodiment, the attributes (positions) of participants in a conference are described as four types: "facilitator," "superior," "subordinate," and "guest." However, the present invention is not limited to this, and other attributes may be defined. For example, if the conference is a workshop, the attributes of participants may be two types: "lecturer" and "student."

The present invention relates to a conference support device and method, and can be applied to a conference support device that supports conferences including online participants.

1...conference support system, 2...face-to-face participant side system, 3...online participant side system, 4...conference support device, 11A, 11B...imaging device, 12A, 12B...sound collection device, 13A, 13B...vibration sensor, 14A, 14B...display device, 20...CPU, 40...image processing unit, 41...audio processing unit, 42...nodding detection unit, 43...action/state determination unit, 44...display information determination unit, 45...display processing unit, 50...participant attribute management table, 51...sensing information management table, 52...determined action/state management table, 53, 53A to 53D...display rule management table, 54...display requirement management table, 55...display information management table.

Claims (10)

A conference support device for supporting a conference, comprising:
a behavior/state determination unit that determines the behavior/state of each participant in real time based on the output of each sensor that acquires information regarding the behavior/state of each participant;
a presentation information determination unit that determines information to be presented to the participants based on the actions and states of the participants determined by the action and state determination unit and a previously set position of each of the participants;
a presentation processing unit that presents the information determined by the presentation information determination unit to the participant via a terminal provided on the corresponding participant's side. An item of information to be presented to the participant is set in advance for each combination of the position and the action/state;
The presentation information determination unit
The conference support device according to claim 1 , wherein, for each participant, information of the item corresponding to the position and the action/state of the participant is determined as information to be presented to the participant. For each of the items, requirements for presenting information of the item to the participants are set in advance;
The presentation information determination unit
The conference support device according to claim 2 , further comprising: determining, as information to be presented to the participants, information on the items that satisfy the requirements. When a plurality of items are set for one combination of the position and the action/state, a priority is set for each of the items;
The presentation information determination unit
4. The conference support device according to claim 3, wherein, when there are a plurality of said items that satisfy said requirements, information on said item having the highest priority is determined as information to be presented to said participant. The conference support device according to claim 1 , wherein at least a part of the information to be presented to the participants in accordance with the preset actions and states differs depending on the participants' positions. A conference support method executed by a conference support device that supports a conference, comprising:
a first step of determining in real time the behavior/state of each participant based on the output of each sensor that acquires information on the behavior/state of each participant;
a second step of determining information to be presented to the participants based on the determined actions and states of the participants and a preset position of each of the participants;
and a third step of presenting the determined information to the participant via a terminal provided on the corresponding participant's side. An item of information to be presented to the participant is set in advance for each combination of the position and the action/state;
In the second step, the conference support device
7. The conference support method according to claim 6, further comprising: determining, for each participant, information of the item corresponding to the position and the action/state of the participant as information to be presented to the participant. For each of the items, requirements for presenting information of the item to the participants are set in advance;
In the second step, the conference support device
The conference support method according to claim 7 , further comprising determining, as information to be presented to the participants, information on the items that satisfy the requirements. When a plurality of items are set for one combination of the position and the action/state, a priority is set for each of the items;
In the second step, the conference support device
9. The conference support method according to claim 8, wherein, when there are a plurality of said items that satisfy said requirements, information on said item having the highest priority is determined as information to be presented to said participant. The conference support method according to claim 6 , wherein at least a part of the information to be presented to the participants in accordance with the preset actions and states differs depending on the positions of the participants.

Images