JP2017201479A - Communication supporting system - Google Patents

Abstract

PROBLEM TO BE SOLVED: To provide a communication supporting system that can appropriately perform, without having to set rules in advance, presentation of information for supporting a communicative situation, such as a conference, or choice of an instruction for altering a communicative environment.SOLUTION: As the situation of a communication space is objectively kept track of in an encoded or numerical way using prescribed indicators by analyzing speech and attitudes of participants acquired by sensors at prescribed intervals of time, changes in the situation can be quantified in a time series. As an algorithm prior to reinforcement learning in which the actual reactions of participants after a communication supporting instruction selected by the system are reflected again as an indicator value and feedback is given to reward the communication supporting instruction immediately before is used, the accuracy of system choice for communication supporting instruction can be enhanced without having to prepare in advance learning rules involving many preparatory man-hours.SELECTED DRAWING: Figure 1

Description

  The present invention relates to a technique for enhancing communication output by providing artificial intelligence as a participant member in the field of communication and providing various information or giving instructions to change the communication space. This is a technique for improving the accuracy of provided information while encoding or digitizing a situation with a plurality of indices by analyzing speech and images, etc., and reflecting it in the learning process of artificial intelligence.

  As an example of means for gathering a plurality of participants to form communication, a business conference is the most representative, and is generally performed every day. However, even though the end time of the meeting is decided, it is also happening in daily life that the intended conclusion is not obtained by the end of the meeting. This is because, although depending on the complexity of the content of the conference, it is influenced by the fact that discussions between a plurality of participants sometimes diverge or converge or become silent.

  Some participants do not have the same level of knowledge about the contents of the meeting, or join the meeting from the middle. If a participant makes a speech without fully understanding the background of the meeting, the purpose of the discussion, and the matters decided so far, the content may not match the direction of the discussion, and the discussion may be irritated. . For this reason, it is impossible to obtain the expected conclusion on schedule, so it is necessary to provide information for participants to share similar knowledge before and during the meeting.

  In view of this, an idea support system that supports ideas during discussions by a plurality of participants and activates the place of discussion has been proposed (for example, Patent Document 1 below). In this idea support system, keywords that appear during the conference and words that support the idea are presented to the participants. In addition, there is a system that learns under what circumstances a change in conference state is recognized or a state change action is executed (for example, Patent Document 2 below).

Japanese Patent No. 51233591 JP 2014-85916 A

It is known that the devices and systems for supporting the ideas of the participants during the conference as described above have insufficient points when used in actual conferences.
For example, in the case of Patent Document 1, a speech analysis of utterances made during a meeting is performed, and a related word set (context-forming word) is calculated by calculating the appearance rate of words that appear in proximity as the relevance level. A set), and keywords and idea support words are extracted. Meeting participants who visually recognize these keywords and idea support words can understand and share the contents of the meeting, and can expand ideas that cannot be obtained only by the participants and activate the meeting space. It is supposed to be possible.

  However, it has not been verified whether conference participants have really spread their ideas with the idea support word provided from the idea support system, or whether the conference has succeeded with the intended outcome. A known idea support system such as Cited Reference 1 unilaterally provides participants with information that the system has determined to be useful, and is said to have the effect of activating the conference space. Not too much. In order to determine whether the use of the idea support system is truly beneficial, it is necessary to quantify and evaluate the remarks and attitudes of the participants who have received the information provided at subsequent meetings. Is important, and it is necessary to reflect the actual changes of participants in the information decision processing by the idea support system in a timely manner.

  The system described in the cited document 2 includes a conference state (for example, an active state, a boiled state, etc.) and an instruction for prompting improvement of the conference state (for example, message display for prompting breaks, Internet search results for frequent keywords) Display, etc.) from the participant, the participant requests what kind of situation the conference state wants to change, and accumulates the relationship with the actual action as a learning rule. It describes that it is possible to present appropriate information based on the learning rules when specified.

  However, in the case of the system of cited document 2, the state of the conference is grasped by the participant and input to the system, and therefore the input does not necessarily represent the actual conference state. For example, even if a participant recognizes that the conference is active because the speech continues without interruption, the reality may be a state-of-the-art discussion and a boiled state. The input may not reflect the meeting status. In addition, participants do not always input the conference status to the system in a timely manner, and if they are aware of the input, they may be distracted and unable to concentrate on the conference. Furthermore, the information that the system accumulates as a learning rule is, for example, a simple content such as displaying a message prompting a break or displaying a word with a high frequency of appearance when in a boiled state. It is a departure from providing useful information for guidance.

  Therefore, in order to solve the above-mentioned problems, the present invention can accurately present information for supporting a communication situation such as a meeting and select an instruction for changing the communication environment without constructing a rule in advance. The purpose is to provide a communication support system.

  In order to achieve the above object, a communication support system by a plurality of participants according to the present invention comprises at least one situation data acquisition means for grasping the situation of a communication space from at least one of audio data and image data, and the situation data And a server that controls any output medium in the communication space using the sensed data obtained by the acquisition means, and the server digitizes or encodes the sensed data at predetermined time intervals. In order to generate the state of the communication space as a state variable at time t and support communication in the state of the communication space at time t, based on the expected value of each support item from among a plurality of communication support items related to the state variable At least one support item After selecting and executing a command to execute the selected support item to the output medium, the expected value of the communication support item related to the state variable at time t + 1 generated based on the sensing data acquired by the situation data acquisition unit is used. The update of the expected value of the support item selected at time t is repeated.

  In the communication support system by a plurality of participants according to the present invention, the update of the expected value of the support item is based on a reinforcement learning algorithm, and the state variable is included in each sensed data obtained by the at least one situation data acquisition unit. It is a numerical value calculated from a set of each corresponding index or each index.

  Further, the plurality of indicators in the communication support system by a plurality of participants according to the present invention include a diversity indicating whether the communication content includes a plurality of viewpoints, a uniformity indicating the degree of communication involvement of each participant, and communication At least one of an activity level indicating whether it is performed alternately among a plurality of participants, an agreement level with respect to the speech or attitude of other participants, and an idea level indicating whether communication is performed using words of different categories The server selects the output medium according to a communication context, outputs an instruction to execute the selected support item for the selected output medium, and the output medium includes: Shared objects including one or more walls and tables in the communication space, and each reference Characterized in that it comprises at least one of personal information terminal including a user of the mobile terminal and an eyeglass-type display device.

  Furthermore, in order to achieve the above object, the table according to the present invention is a table used for communication support by a plurality of participants, and is a sound or image acquired using at least one situation data acquisition means. The server to which the sensing data related to is transmitted generates a state of the communication space as a state variable at time t by digitizing or encoding the sensing data at predetermined time intervals, and the state of the communication space at time t. In order to support communication in the state variable, at least one support item is selected from a plurality of communication support items related to the state variable based on an expected value of each support item, and after executing the selected support item, the situation data acquisition unit State at time t + 1 generated based on the sensing data acquired by Information extracted by the server according to the selected support item in the repetition of updating the expected value of the support item selected at time t using the expected value of the communication support item regarding the number Is displayed.

  According to the present invention, for each predetermined time interval, the actual utterance and attitude of a person who participated in a communication venue such as a meeting are analyzed, and the state of the communication space is expressed by a predetermined index or expressed as a numerical value. It is possible to quantify the changes in the situation on the spot in time series. In addition, the actual reaction of the participant after the communication support item selected by the system is issued is reflected again as the index value, and the reward based on the reflected index is fed back to the previous communication support instruction and the communication support item It is equivalent to learning automatically after evaluating whether the system has selected appropriate communication support according to the situation of the communication space in order to update the selection probability. For this reason, the selection accuracy of the communication support instruction by the system can be improved without implementing the learning rule in the system in advance.

  In particular, the selection of communication support items such as information provision in the present invention is configured to be determined by calculation based on the reinforcement learning model, and the appropriateness of the selection of communication support items is gradually refined as the number of selections is repeated. Since it is updated through the calculation of the expression, the subsequent selection strategy can be determined while the system itself learns without requiring an instruction from the participant. That is, since the selection in the present invention is not based on a learning method (so-called supervised learning) according to a rule incorporated in the system in advance, a model method in which communication support items such as information provision cannot be selected unless a learning rule is constructed in advance. Compared with the above, a great amount of man-hours required for constructing learning rules can be omitted.

  Furthermore, the output destination of the selected communication support item is specified from a plurality of output media according to the status of the communication space, the content of information to be provided, or the attributes of the participants. Since it is configured, information to be shared by a plurality of participants is distinguished from information necessary for each participant. Information that is truly useful to each participant and instructions to change the communication space are provided to the appropriate output medium for each participant without interfering with the progress of communication, and each participant who recognizes this activates communication. Can be used for.

It is the schematic which shows the spatial environment of the meeting assistance system by one Embodiment of this invention. It is a flowchart which shows the whole procedure of the process which the conference assistance system by one Embodiment of this invention performs. It is a process transition diagram showing the flow of the main process. It is a flowchart which shows the example of a procedure of the process which acquires meeting condition data. It is a detailed flowchart regarding the determination process of a meeting condition. It is a detailed flowchart flowchart regarding the determination process of reward amount. It is a detailed flowchart regarding the determination process of a recommendation. It is a figure which shows the example of the value (gain expected value) of each action with respect to the group (tuple) of a state variable. It is a figure which shows the example of the value (gain expected value) of each action with respect to the group (tuple) of a state variable. It is a figure which shows the example of the value (gain expected value) of each action with respect to the group (tuple) of a state variable. It is a figure which shows the example of the value (gain expected value) of each action with respect to the group (tuple) of a state variable. It is a flowchart which shows an example in the case of performing a personal recommendation based on the specificity of a word. It is a detailed flowchart regarding the recommendation presenting process.

  An embodiment of a communication support system according to the present invention will be described below with reference to the drawings. This embodiment is a conference support system in which the communication support system of the present invention is applied in a conference room of a company or the like. First, devices used in the conference support system will be described.

As shown in FIG. 1, there is a table 13 in the conference room, and participants are sitting on the chair 5 facing each other. Each participant is assigned a portable terminal 2 and is connected to the server 1 via a wireless or wired network (not shown). The mobile terminal 2 can transmit various instructions to the server 1 via the network in accordance with an operation input from the participant. Then, the server 1 extracts or outputs information useful for the participants based on audio and image information obtained from the space called the conference room, that is, information for activating the discussion and a command for changing the environment of the conference room. To do. Details of the specific processing will be described later.
The mobile terminal 2 is not particularly limited, but may be a desktop or laptop personal computer, a general-purpose computer device such as a workstation, a tablet terminal, a smartphone terminal, a PDA (Personal Digital Assistant) terminal, or the like. Can be used. Each participant can confirm information displayed on the screen of the mobile terminal 2.

  Each participant uses one or more microphones 4 built into or connected to the portable terminal 2. Voices spoken by each participant are collected by the microphone 4 and transmitted to the server 1. In this embodiment, the microphone 4 assigned to each participant is used in order to make it easy to identify who made the sound.

  Each participant may wear glasses 6 with an information display function belonging to the wearable terminal. The glasses 6 with an information display function have glasses identifiers corresponding to client identifiers for specifying the mobile terminals 2 used by each participant for communication with the server 1. The server 1 transmits information to the glasses 6 with an information display function worn by a specific participant as necessary, so that the participant can simultaneously display and display information while operating or browsing his / her mobile terminal 2. Information displayed on the functional glasses 6 can also be recognized. Note that other wearable terminals that replace glasses include smart watches, wristbands, rings, and the like, including the additional use of these. Furthermore, an arbitrary projector 7 or the like that is communicably connected to the server 1 may be installed in the conference room, and can be used as, for example, a common input / output screen for all participants to view all at once. .

  A camera 8 for capturing the facial expressions and actions of the participants is installed at an arbitrary location (for example, a ceiling) in the conference room. From the image taken by the camera 8, sitting posture (for example, forward / backward tilting, elbows, crossed legs, etc.), nodding motions, and facial expressions are analyzed to analyze emotions Guess the emotions. Any image analysis technique can be used for these detection and estimation methods. In addition, you may make it image with each camera 3 for every participant with the camera function with which the portable terminal allocated to each participant is provided.

  The server 1 connected to each device via a network is responsible for overall control of the conference support system. The server 1 transmits information to various output media that can be recognized by each participant. Examples of the output medium generally include a screen of the mobile terminal 2 assigned to each participant, and a display device or projector 7 that is shared and viewed by participants in a conference room. The conference support system of the present invention is configured to use not only such an existing information output mode but also the table 13 and the wall surfaces 9 and 10 in the conference room as a display. In addition, although details will be described later, the meaning of information displayed on the table 13 and the wall surfaces 9 and 10 is given to each. For example, the participants see the information displayed on the “table 13” and are important in the discussion. This is useful for confirming simple keywords, phrases, etc., and expanding the idea by looking at the information displayed on “Wall 9”. Note that the displayed information is not limited to text. For example, it may be a still image or a moving image with or without sound.

FIG. 2 is a flowchart showing an outline of the entire process executed in the conference support system. FIG. 3 shows each process as a process transition diagram.
After the start of the conference, the status of the conference room is grasped by a plurality of status data acquisition means (that is, sensors) such as the microphone 4 and the cameras 3 and 8 described above (step S201 in FIG. 2). This corresponds to “sensing process 31” in the process transition diagram of FIG.

  After the “sensing process 31”, the process proceeds to the “communication state perception process 32”. This is because the voice data and the image data obtained from the microphone 4 and the cameras 3 and 8 are analyzed, and the situation is now visualized by expressing numerically (step S202 in FIG. 2). . Since the sensing data representing the communication state is acquired through various sensors in the space called the conference room, the feature amount of the communication space can be extracted. It should be noted that, as a feature amount representing the actual conference situation, for example, a single indicator that conference participants are actively discussing and the activity level is high is not sufficient. Therefore, the meeting status is judged by a plurality of indicators. The contents of a plurality of indices representing the feature amount and the specification of the communication state using the plurality of indices will be described later.

  In the first process after the start of the conference, the reward process in step S203 in FIG. 2 is skipped (corresponding to the case where the reward process is No). This remuneration is a remuneration given to an action result in which the server 1 causes a change in the communication state such as keyword presentation, and is still before the action of the server 1 in the first process. Then, a transition is made from the “communication state perception process 32” to the “behavior selection process 34”. When executing the “behavior selection process 34”, the value (expected gain value) Q of the behavior a calculated in the “expected gain learning process 35” is used as a parameter. Further, the “expected gain learning process 35” calculates the value Q using the reward amount if the reward amount is obtained. The “communication support item” described in the claims corresponds to an action.

  The “behavior selection process 34” corresponds to the recommendation determination process shown in step S205 of FIG. The “behavior selection process 34” is to determine what action is appropriate for the server 1 to adopt based on a predetermined learning algorithm. The latest meeting situation represented by a plurality of indicators is determined. The action that can be a trigger to change is to select from a plurality of options. Specifically, information that is useful for making it diverge according to the conference situation, or conversely converging, is extracted as recommended (recommended) information, or for changing the conference room environment. Since a command is output, it is a recommendation determination process.

  The action selection by the server 1, that is, the determination of the recommendation, is based on a reinforcement learning algorithm known in the field of artificial intelligence. Learning in the present embodiment refers to the value of each action a when the server 1 tries to select the current meeting status (state s), and the value of each action related to the meeting status after execution of the selected action a. In other words, the value of each action a is updated based on a reward for a change in the meeting status, and thereby the suitability of the action selected by the server 1 and presented to the participant is improved. The recommendation information is roughly classified into information directed to each participant and information directed to all participants.

  Further, the server 1 selects an output medium suitable for presenting the determined recommendation information, and outputs an instruction for appropriately displaying it on the output medium (step S206 in FIG. 2). As a medium for outputting information in a space called a conference room, for example, there is a normal hardware device such as a display screen, and walls, floors, ceilings, furniture, etc. existing in the conference room can be output media similar to the display screen.

  While the conference end instruction is not input, the above-described processing is repeated. That is, the process returns to step S201, and audio data and image data from the microphone 4 and the cameras 3 and 8 are acquired. Further, when the repetitive process is being executed (that is, when the recommendation information has already been presented), the reward process of step S204 is executed. This reward is a positive reward if the communication status is improved for the action that the server 1 has selected specific recommendation information as useful information, and conversely no change is seen in the communication status. If it goes bad, it means returning negative rewards to the server. Therefore, the recommendation provided by the server when the meeting is about to start may not necessarily be beneficial because the reward is not sufficiently reflected, but the result of selection by the server 1 is selected next in the form of reward. Through iterative learning that influences the decision of the action to be taken, it will produce action choices that are highly accurate (ie, deemed more appropriate) over time.

  Thereafter, when an instruction to end the conference is input from a conference participant (for example, the person in charge of the moderator), and the server 1 receives the instruction, the utterance contents in the conference, recommended recommendation information, etc. are related to the client. The identifier and the information are stored in a format that can be grasped while being associated with the output medium (step S209). It goes without saying that the decision items discussed may also be saved.

In addition, after the conference, the evaluation from the conference participant may be accepted for the action a selected by the server 1 and used as an additional reward in the recommendation determination process in step S205. In the held meeting, questions such as information recommended by the server 1 and whether action commands for changing the environment of the conference room, which is a communication space, are useful are presented on an arbitrary output medium, and the participant performs a touch operation, etc. The answer is input at (step S208).
In the present embodiment, step S208 is executed after the conference ends, but may be executed immediately after the recommendation is presented in step S206. In this case, the evaluation from the participant is fed back whenever there is a recommendation by the server 1, and the convergence speed of reinforcement learning by the server 1 increases. Incorporating the evaluation from the participant into the reinforcement learning process of the server 1 contributes to further shortening the time required for selecting an appropriate action.

Hereinafter, each of the outline flows of the present invention shown in FIG. 2 will be described in detail.
FIG. 4 is a detailed flowchart regarding the process of acquiring the meeting status data shown in the step of FIG. As a preliminary preparation for actually starting sensing with the microphone 4 or the cameras 3 and 8, a conference participant (for example, a person in charge of a facilitator) inputs a conference ID for identifying which conference status data. (Step S401). The portable terminal 2, the microphone 4, the glasses 6 with an information display function, etc. used by each participant are associated with the client identifier of each participant (step S402). Furthermore, although the server 1 presents recommendation information on various output media, if it is considered as a conference participant, it will be confused if it does not recognize in advance what information will be presented on which output media. . Therefore, the association between the output medium and the output content to be presented is notified to the conference participants and set in advance on the server 1 side (steps S403 and S404).

  When the conference starts, data acquired by the microphone 4 and the cameras 3 and 8 in the conference room are transmitted to the server 1 via a wireless or wired network at predetermined time intervals (step S406). The server 1 records the transmitted data as sensing data in the memory (step S407). Although recording is performed at predetermined time intervals, data acquisition by the microphone 4 and the cameras 3 and 8 is actually performed continuously, and during a predetermined time interval (from time t-1 to time t). The acquired data may be recorded as sensing data at the timing of time t.

  In this embodiment, since the microphone 4 is assigned to each participant, the speaker is identified by the voice input device identifier or the client identifier of the mobile terminal 2 connected to the microphone 4, and the speech content is stored in association with the speaker. To do. In another embodiment, the server 1 notifies the server 1 that the participant will speak, or a part of the participant (for example, in charge of the chairman facilitator). The user) may make an input for transmitting as a signal to the server 1 who is currently speaking. In still another embodiment, the voices of a plurality of persons may be collected with a single microphone (for example, a desktop microphone arranged on a table). In this case, the method of identifying which participant is the main speaker uses any speaker recognition technology (for example, including voice quality and voiceprint recognition) that identifies a speaker from mixed speech, Identify the speaker.

FIG. 5 is a detailed flowchart regarding the conference status determination processing shown in step S202 of FIG.
First, when there is audio data acquired by the microphone 4, the server 1 identifies the participant who is actually in the conference room, what the participant is talking about, and what topic is being noted. Audio data is analyzed in order to understand the above. Specifically, “speech amount” and “speech interval” at (time t-1 to time t) are calculated (step S501) “speech amount” is the end of speech from each participant's speech start time. It may be based on the total time up to the time, or may be the number of characters that make up a word extracted by morphological analysis, which will be described later. It is one of the easiest indicators to show, and “speech interval” is a useful indicator for knowing the silence situation.

  Further, it is possible to identify that the speech has been switched from the speech of the participant A to the speech of the participant B by analyzing the audio data. This shows the fact that several participants were playing a conversation ball during the meeting, also called the amount of turn taking. The greater the amount of turn taking, the greater the number of speakers, so the higher the index of “activity”. Furthermore, “activity” can also be calculated using the average utterance time per participant at (time t-1 to time t). In addition, the participant with respect to the sum total of the speech amount of each participant The calculated value of the amount of speech of A is an index called “uniformity” that indicates whether or not all the participants are speaking almost uniformly. In addition, based on the frequency of words appearing from different fields or categories per participant (from time t-1 to time t), There is an index: A situation where many words are spoken from different fields or categories has a high degree of creativity.

  “Diversity” is another indicator. In short, this is an index based on the number of types of words, that is, whether the same word is used frequently or another word is used frequently in the conference. Using speech recognition technology, the utterance content is converted to text, then morphological analysis is performed and extracted as multiple words (steps S503, S504). Words that are less important in understanding the content of discussion such as particles and exclamations It is possible to extract a word as a keyword by deleting a phrase. As a result, in the case of a meeting where many of the same words are used, the content of the discussion indicates that “diversity” is lacking. In addition to determining by strict identity, as a result of synonym classification and cluster classification, each keyword may be expanded to be regarded as similar, and “diversity” may be determined. Note that the technique based on morphological analysis or the like in step S503 is within the scope of the existing technology and will be omitted in this specification, but will be fully understood by those skilled in the art.

  There is also an index called “degree of interaction”. When the conference participant touches the information presented on the table or the wall, the fact of touch is recognized, and the number of touches at (time t-1 to time t) is expressed as the “degree of interaction”.

  Each of the above indices is digitized or encoded. For example, as a simple example of digitization, each index is represented by five levels (1 to 5). In this way, by quantifying or encoding the status of the meeting with multiple indicators, what participants are talking about what, how actively each participant participates in the form of remarks It is possible to quantitatively grasp whether or not the topic (topic) that is attracting attention (step S506).

  Note that the method of estimating a topic from the content of a statement is not limited to this. For example, LDA (Latent Dirichlet Allocation) may be used. With the LDA method, a word does not exist independently and is latent. This is a language model that probabilistically obtains the topic of a word in a sentence under the assumption that words having the same topic and having the same topic are likely to appear in the same sentence. Since it is assumed that each word is generated from a hidden topic, it is possible to estimate a topic (topic) that is attracting attention from a word set obtained by analyzing the content of a statement. By using a topic estimation technique such as LDA, it is said that, for example, fruit apples and computer-related apples (R) can be distinguished.

  Next, when there is image data acquired by the camera, the face and figure of a person are recognized from the image data (step S507). For example, if you perform a nodding motion or detect participants with a forward leaning posture, you agree with the speech of other participants, while twisting your neck or gesturing your face to the left or right, If a participant in a backward leaning posture is detected, it is assumed that he / she has expressed a denial, so this is an index of “agreement”. It is possible to detect approval and denial from other actions and facial expressions. As described above, according to the present embodiment, in addition to extracting direct words such as approval and disapproval from audio data by analyzing image data, an index of “agreement” is calculated also from image data. Including that.

  The conference state can be set using a plurality of indices calculated by the above-described analysis of the audio data and the image data (step S508). For example, if the index values of “amount of speech”, “activity”, “uniformity”, and “diversity” at (time t-1 to time t) are large, the meeting at (time t-1 to time t) It is determined that the state is active and likely to be in the divergent mode, and in the opposite case, it is likely to be in the convergent mode. If the “speech interval” of a large number of participants is large, the conference state is silent, so there is a high possibility that it can be regarded as dead or boiled. This is a form of convergence mode. In this way, it is possible to automatically estimate whether the content of the discussion has converged or diverged, so that the purpose of the conference to be held is diverged if it is in the convergence mode. Target, or converge if divergent mode However, since several indicators may not necessarily accurately correspond to the divergent mode or the convergence mode, the participant can manually set the conference mode state manually. In the embodiment described later, an example in which the purpose of the conference is manually set is shown, and the mode state may be comprehensively determined by a combination of automatic estimation and manual setting. .

FIG. 6 is a detailed flowchart related to the reward amount determination processing shown in step S204 of FIG.
When a value such as “amount of speech”, “activity”, “uniformity”, “diversity” or the like, which is a feature amount representing the actual meeting status, is received (step S601), the reward amount is then determined. (Step S602). That is, the reward for the action most recently selected by the server is calculated using the change in the feature amount. Assuming that the current time is t, the feature quantities such as the degree of activity and the idea amount acquired in the most recent time interval (t-1, t) and the previous time interval (t-2, t-1) are used.
An example of a reward calculation algorithm is shown below.
Reward: = 0;
if ((t-1, t] utterance volume-(t-2, t-1] utterance volume)> 0 then add 1 to the reward;
if ((t-1, t) idea amount-(t-2, t-1] idea amount)> 0 then add 1 to the reward;
if ((t-1, t) turn taking amount-(t-2, t-1] turn taking amount)> 0 then 1 is added to the reward;
if ((t-1, t) interaction amount-(t-2, t-1] interaction amount)> 0 then 1 is added to the reward;
Etc. The above-described example shows only the addition reward. However, when the utterance amount or the like decreases, −1 is added to the reward to make a subtraction reward.

I will explain the rewards again.
The reward is, for example, every time the action command by the server 1 is repeated when the conference support system presents the participants and the conference room such as what information the conference support system has presented to the participants in the conference room. This is to provide the server 1 with clues for learning to be improved. The reward processing in the present embodiment is based on reinforcement learning that has been studied in the field of artificial intelligence.

  Reinforcement learning is based on learning control adapted to the external environment through trial and error, and is different from so-called supervised learning. In supervised learning, there is a rule that explicitly guides the appropriate output (events and actions that change the situation) for the state input that indicates the meeting room situation. Will be offered. Reinforcement learning, on the other hand, is a learning control method that aims for optimal output by autonomously and actively improving the number of trials and errors by using information called rewards as clues instead of teachers (ie, rules). is there. That is, the server 1 which is the action subject selects a certain action based on the situation data detected by various sensors in the meeting room, and the situation in the meeting room changes based on the action. A certain reward is given to the server 1 as the situation changes, and the server 1 is an algorithm that gradually learns to select a better action (decision decision) in response to this.

Exchange according to the following procedure is performed between the server 1 of the conference support system and the conference space. Hereinafter, (time t-1 to time t) is assumed to be time t, and (time t to time t + 1) is assumed to be time t + 1.
(1) at time t, and outputs an action a _t in accordance with the state s _t meeting space.
(2) on the selected action a _t the conference support system, meeting space the state changes to s _{t + 1,} reward r _t corresponding to the transition to the conference support system.
(3) Advance time t to time t + 1 and return to (1).
In the learning by the conference support system, the action a for the state s is sequentially determined for the purpose of maximizing the reward r.

A general update formula (recursion formula) of such reinforcement learning is, for example, as follows.

Q (s _{t + 1} , a _{t + 1} ) ← Q (s _t , a _t ) + α {r _{t + 1} + γmaxQ (s _{t + 1} , a) −Q (s _t , a _t )}

Q (s _t , a _t ) is a “value” for selecting the action a under a certain state s at time t. “Value” is also referred to as “expected gain” in the field of reinforcement learning. In the learning process by the server 1, the update formula selects that the action a having the highest value Q (s, a) is the optimum action at the time t in a certain state s. It is also executed at the subsequent time t _{t + 1} , meaning that this is repeated. The term with max in the update formula is the discount rate γ (0 <γ) when the action a having the highest value Q (known at that time) is selected under the state s _{t + 1.} ≦ 1) and α is a learning speed coefficient (0 <α ≦ 1).

This update equation, in which state the action a _t is changed to s _{t + 1,} the value Q (s _t, a _t) of action a in state s than the best in the next state s _{t + 1} by action a If the value Q (s _{t + 1} , a _{t + 1} ) of the action is larger, the value Q (s _t , a _t ) is increased, and if the value Q (s _t , a _t ) is smaller, the value Q (s _t , a _t ) is also decreased. It shows that Initially, the correct value Q (s, a) is not known for each combination of state s and action a. (The reason is that there is no teacher rule.) Therefore, the initial value Q (s, a) is randomly determined for all states and possible s, a pairs of actions that can be taken at that time. Keep it. This reinforcement learning makes it possible to bring the value Q (s, a) of a certain action a in a certain state s closer to the value of the best action in the next state. The difference depends on the discount rate γ and the reward r _{t + 1} in the formula, but basically, the best action value in a certain state propagates to the value of the action in the previous state. It is an algorithm.

Server, in order to utilize the reinforcement learning model described above, in step S601 of FIG. 6, each index (remarks amount at time t detected by the various sensors such as a microphone 4 and a camera 3, 8: P _a, activity: P _b , diversity: P _c , uniformity: P _d , count number tapped by participant for presented word: _Pe , count number tapped by participant for presented recommendation: P _f ) It takes values and defines state s and reward r. In the simplest example, the state s is defined by a set (P _a , P _b , P _c , P _d , P _e , P _f ) composed of each index value, and the statement amount P _a is 4 out of 5 levels. When the activity _Pb is 2 in 5 stages,..., The reward r value is calculated by adding or multiplying the values of the plurality of indices. At this time, for example, if the “diversity” index is more important than other indexes, a weighting factor greater than 1 is added to the “diversity” index value before simply adding or multiplying the multiple index values. Multiply it. In addition, since the conference mode state (divergence or convergence) is automatically estimated or manually set when calculating the index value, a reward calculation method considering the conference mode state may be used. For example, when the “uniformity” index at time t + 1 decreases in the diverging mode state, the reward is decreased by 1. When the “activity” index at time t + 1 increases in the convergence mode state, the reward is increased by one. Condition.

  Next, step S603 in FIG. 6 is processing for receiving a reward from a conference participant. As described above, basically, the reward r value is automatically calculated based on each index value generated based on the sensed data detected by various sensors in the conference room. When a conference participant has an impression that a recommendation (for example, presentation of specific information) is good, a reward from the conference participant may be additionally added. Conversely, if the meeting participants have the impression that the presented information is not appropriate in terms of the content and timing, a negative reward is given that makes the value lower than the automatically calculated reward r value, and subsequent reinforcement learning The information is reflected so that it is difficult to select the information in the selection by the algorithm.

And, finally, to calculate the total reward amount at time t (step S604), this is set to reward r _t in the update formula of reinforcement learning algorithm (step S605).

Next, FIG. 7 is a detailed flowchart regarding the recommendation determination process shown in step S205 of FIG.
The server 1 of the conference support system selects the action a under a certain state s at time t according to the learning algorithm based on the above-described reinforcement learning model. The representative action a is a recommendation of information It is. The server selects necessary information resources such as news, research papers, books, etc., and extracts specific information.

  First, a conference ID is used to identify whether the conference is a newly held conference or a conference that is a continuation of a conference that has already been held (step S701). Note that it is desirable to input the conference ID into the server 1 before starting the discussion. In the case of a new meeting, the initial value Q (s, a) used in the update formula of the reinforcement learning model is set to an arbitrary default value determined in a predetermined or random manner (step S703). If there is a conference ID that is the same as a conference ID that has already been held, setting it to a random default value will waste that the server 1 has learned based on past conference contents. Therefore, the value Q (s, a) and the reward r of the action a in the state s so far are read out and set as initial values so that the meeting is resumed due to past meeting contents and learning extension ( Step S702).

  Next, the purpose of the conference is set (step S704). The contents that server 1 recommends (presentation of information) depending on whether the current meeting is aimed at converging the discussion, whether it is aimed at diverging, or deepening And the instruction for changing the conference room environment) are different, the purpose of the conference is set in step S704. In the present embodiment, an input screen for selecting the purpose of the conference is displayed from the server 1 so that the conference participant can select, but the automatic setting without the participant's input is excluded. is not. For example, it may be automatically set so that a meeting for diverging or deepening discussions is targeted if the number of meetings held is small, and a meeting that converges if the number of meetings exceeds a predetermined number. It may also be decided according to the participant member's attribute information.For example, if the participant is a group of newcomers who have not joined the company for a long time, or if the conference is made up of participants with little experience, free and active communication is the first step. It may be possible to automatically set a conference with a goal of divergence, giving priority to the formation of

Next, the server 1 sets a plurality of actions according to the purpose of the conference set in step S704 (step S705). Specific examples include the following actions.
1. Quietly see the situation 2. Present the common sense association 3. Present the co-occurring monogoto 4. Present the article of the dictionary (Wikipedia) 5. Present the news article 6. Present the SNS (twitter) tweet 7. Presentation of images from social media 8. Presentation of related papers 9. Presentation of related books Needless to say, other actions may be set, and more actions or less actions may be set. May be set.

Assuming that the actions 1 to 9 above are, for example, a list of actions set for a meeting aimed at diverging, in each meeting aiming at convergence or deepening discussion. Set another action list. Since one action is selected from a group of actions suitable for the purpose of each meeting, the recommendation quality is improved. Some actions may be set in common for each purpose.
It is also possible to set a common action list for any meeting purpose, instead of setting a different action list for each purpose of the meeting. However, in the case of a common action list, the amount of calculation in the learning routine, which will be described later, increases exponentially, and the number of learning trial and error increases while the number of actions that are clearly not consistent with the purpose of the meeting increases. Considering the fact that unnecessary calculation processing occurs, and the ease of management of the action list, it is often preferable to set a different action list for each purpose.

  Next, specific processing for selecting the action that the server 1 determines to be most appropriate using the value (expected gain) of each action for the state variable tuple (tuple) is shown in FIG. This will be explained using D).

FIG. 8A shows the status of the conference room at time t, which means (time t-1 to time t), “activity”, “concept”, “uniformity”, “agreement”, “interaction”. This is an example in the case of using five indexes of “degree.” Further, as described below, “H” if the value of each index is greater than a predetermined threshold value set in advance, and “L” if the value is smaller. To "".
if Activity> P1 then “H” otherwise “L”
if idea amount> P2 then “H” otherwise “L”
if Uniformity> P3 then “H” otherwise “L”
if Agreement> P4 then “H” otherwise “L”
if interaction> P5 then “H” otherwise “L”
(Here, P1, P2,..., P5 are predetermined threshold values (constants).)
In another embodiment, other indicators such as “turn taking amount”, “diversity”, “speech amount”, “speech interval” may be used, and the number of indicators to be used is arbitrary.

In the case of the present embodiment, the state S of one conference room is represented by a tuple having “H” or “L” of five indices as a variable, so the number of possible states is the combination of variables. There are only numbers (2 ⁵ = 32). The left table in FIG. 8A indicates that each of the five index tuples corresponds to one of 32 states S (1) to S (32). In addition to making each index a tuple, each index may be represented by a numerical value, and a numerical value obtained by adding / subtracting / dividing each numerical value may be associated with each state S (i).

  The right table of FIG. 8A is the value (expected gain value) Q (i, j) for each action a selected by the server 1, i = 1 to 32, and j = 1 to 9. As described above, each action a in the present embodiment is as follows: 1. quietly observe the state; 2. present the common sense association; 3. present the co-occurring monogoto; 4. the dictionary content (Wikipedia) Presenting, 5. Presenting news articles, 6. Presenting SNS (twitter) tweets, 7. Presenting images from social media, 8. Presenting related papers, 9. Presenting related books.

  Since the server 1 is before the execution of learning based on the reward at the start of the meeting, a default value is set for the value (expected gain value) Q (i, j) for each action a. In the present embodiment, FIG. 8B shows setting as a default value.

  Next, referring to FIG. 8C showing the value (expected gain) Q (i, j) after learning is executed to some extent, the server 1 selects a specific action. Explain how. Now, the sign value of the index of time t based on the feature value obtained from each sensor is (activity, idea amount, uniformity, agreement degree, interaction amount) = (L, L, L, L, H) Then, the state of the conference room at time t corresponds to “state 16” in FIG. With respect to this state 16, the expected value of gain that can be obtained in the future by the server 1 selecting any one of the actions a from 1 to 9, that is, the value Q (i, j) of each action is shaded. Value.

The server selects an action with a probability corresponding to the ratio of expected gain values of the actions a _{1 to} a ₉ . That is, the probability that the action a _{1 of} “seen quietly” is selected is 0.0825 (= 3.57 / 43.22), and the action a ₂ of “present common sense association” is 0.1003 (= 4.34 / 43.22),. The action a ₉ “present a related book” is 0.0997 (= 4.31 / 43.22). Here, 43.22 is the sum total of each a ₁ (i = 1 to 9).
In the case of the present embodiment, an example in which the state S of the conference room state is expressed based on the index value is encoded as “H” or “L” in comparison with the threshold value. For example, you may make it represent with the numerical value which added / subtracted / divided each index digitized in five steps levels (1-5).

Furthermore, the server 1 selects a certain action at random with a probability ε using, for example, the Epsilon-Greedy algorithm. In the Epsilon-Greedy algorithm, the actions a ₁ and a ₂ are selected at random with a certain small probability ε, and the action having the maximum Q (i, j) with the probability 1−ε is selected. For example, if ε = 0.1, select the action with the highest Q (i, j) with a high probability (1-ε = 0.9) and select the action with a small probability (0.1). To do. In other words, in most cases, the appropriate action that should be selected based on high value (expected value of gain) is recommended, but it is generally not selected because of its low value. Try to select inappropriate behaviors that are likely to be selected with a small probability so that participants can recommend them. This avoids that learning does not progress due to local convergence, such as when the value Q (i, j) is always selected with the maximum value, the influence of the Q value given at random is increased. Because. Instead of selecting the action a having the maximum Q (i, j) value at an appropriate timing in the learning process, another action a ′ having a low value is selected.

  In another embodiment, instead of the Epsilon-Greedy algorithm, the action selection of the server 1 may be performed by the Softmax method or the like that similarly selects the action probabilistically. The Softmax method defines a predetermined function for a certain state s, regards this as a probability distribution for an action to be selected, and selects an action by, for example, an inverse function method. As a result, like the Epsilon-Greedy algorithm, as the Q value increases, the value of the Soxtmax function of the action also increases, and the action is easily selected. It should be noted that there is no limitation on applying other methods other than Epsilon-Greedy and Softmax functions, or applying a combination of a plurality of methods.

In the example of FIG. 8C, a case where “present a twitter tweet” of the action a ₆ having the maximum value among a plurality of actions that can be taken is selected (step S706). Moreover, the action to select does not necessarily have to be one. For example, “Present an image from social media” of the action a ₇ which is the action with the next highest value may be selected.

The server 1 that has selected the optimum action a ₆ outputs a command for executing the action a ₆ in the communication space called the conference room (step S707). Since the action a ₆ is “present a twitter tweet”, the server 1 presents the tweet information as recommendation information in the conference room, but the server 1 should recommend to all participants or only a specific conference participant It is determined whether or not to recommend, and the relevant output medium is selected to present recommendation information. For example, behavior a ₆ by the server 1 has selected the case of a high action personal dependence, and recommendation information to one person rather than the entire participants. The tweet of action a ₆ is highly dependent on the person sharing the tweet, and there may be matters that cannot be understood by the conference participants who do not share the tweet. Therefore, when the server 1 instructs the action a ₆ , a tweet is presented on the screen of the mobile terminal 2 or the wearable terminal (such as the glasses 6 with information display function) of only the conference participants sharing the tweet. Give instructions. If the action a ₂ “present common sense association” in the above example is selected as the action of maximum value, the shared display 7 installed in the conference room and all the members for the recommendation to all participants An instruction to be displayed on specific wall surfaces 9 and 10 that can be visually recognized is given.

  In addition to the above-mentioned personal dependence, the word included in action a is highly specific, and even if it is only included in the speech of a specific conference participant, it should be presented only to that participant. That's fine. FIG. 9 is a flowchart illustrating an example of a case where a personal recommendation is performed based on word specificity.

The personal recommendation is basically based on determining recommendation information based on the voice data and image data of each individual.
Therefore, first, when it is determined that the amount of speech of the individual is small (Yes in step S901), the topic (CT: Current Topic) that is currently being discussed is obtained from past conference data obtained based on the conference ID. It is identified whether it has appeared (step S902). If there is, the personal history data is searched, and the tf-idf value of the topic is calculated in order to determine whether or not there is a statement about the topic (step S903). The tf-idf value is used in the fields of information retrieval and document summarization, and is a value calculated based on two indicators: word frequency (term frequency) and inverse document frequency. . In other words, this is a technique for determining the importance of a word under the assumption that the more words appear, the more important the word is, and the importance of a word that appears across many documents is small.

  Therefore, the number of times the topic word appeared in the individual's past meeting data and the number of times the topic word appeared across multiple personal resources such as other meetings, emails and tweets were calculated and calculated. If the tf-idf value exceeds a predetermined level, it is determined that the individual has a high probability of knowing about the topic, and the server 1 does nothing (step S907). On the other hand, if the tf-idf value is equal to or lower than a predetermined level, it is determined that there is a high possibility that there is no knowledge about topics. In this case, the meaning and definition of the topic are searched from related resources (step S905), and output to the personal device of the individual (step S906). The personal device may be a general portable terminal 2, but in this embodiment, each participant wears glasses 6 with an information display function. Therefore, based on the tf-id value, the server determined that there is information that is necessary for other participants to participate actively in discussions, although it is unnecessary information for other participants. In this case, control can be performed so that information is displayed only on the glasses 6 with the information display function of the specific participant.

Here, the idea about selection of recommendation information is summarized.
(i) The conference support system of the present embodiment is not limited to supporting a single conference held on a certain day so that it can be successful, but can comprehensively support intermittent communication over a long period of time. The recommendations learned and presented by the server 1 for support at past meetings are utilized at future meetings. Therefore, appropriate recommendations based on learning accumulated based on the previous meeting are extracted.
(ii) The conference support system according to the present embodiment selects recommendation information after understanding the experiences and knowledge of individuals participating in communication at a conference. Information that should be supported for each individual is selected from the tf-idf value, etc. described above, using the contents of remarks on various media such as email and social media. Supporting accurate pinpoint information provision that differs from uniform information provision.
(iii) The conference support system according to the present embodiment selects recommendation information by using all information in a wide range of the world. In general, information related to phrases and topics, which are keywords and phrases, is extracted from resources prepared in advance. In the case of this embodiment, information that is not directly related to the content of the conference is also referred to and used for communication. For example, from any big data such as POS data, audience rating data, climate change data, stock price information, traffic jam information, etc., find out hidden laws and trends from fact data by any statistical analysis method including basket analysis, Unexpected correlation data that is not bound by existing stereotypes is extracted as recommendation information.

In this way, the choice of action a _t by the server 1 of the conference support system of the present embodiment described above, the entire indoor environment, such as such as play music or or in the conference room presenting a common information towards all participants In addition to changing, presenting specific information that can be beneficial to each individual based on the personal attributes of each conference participant (experience, knowledge, background, content of past comments, etc.) This means that the environment changes for each participant, such as adjusting the wind direction so that it is exposed to cold wind. It should be noted that the present invention includes not only providing text information as is done in existing conference support systems, but also issuing instructions for changing the environment of the communication field globally and locally.

Here, it returns to the flowchart shown in FIG. When the instruction of optimal action a _t with the maximum value at time t is output at step S707, the next server 1 acquires the feature quantity in the conference room of the time t + 1 from the various sensors (step S 708), the time t In the same manner as in the above, a tuple of state variables at time t + 1 is generated. As shown in the above-described example, in the conference room space that was in state 16 at time t, as a result of the server 1 instructing execution of action a ₆ , a set of state variables becomes (activity, idea amount, uniformity, It is assumed that the degree of agreement, the amount of interaction) = (H, H, L, H, H). This corresponds to the state 5 in FIG. Further, it is assumed that the reward becomes a total of +2 due to an increase in both the interaction amount and the turn taking amount according to a predetermined reward calculation algorithm.

At this time, the expected gain Q (16, 6) of action 6 in state 16 is
Q (s _{t + 1} , a _{t + 1} ) ← Q (s _t , a _t ) + α {r _{t + 1} + γmaxQ (s _{t + 1} , a) −Q (s _t , a _t )} Updated as follows: However, the learning count α = 0.3 and the discount rate γ = 0.9.
Among the actions a ₁ to a _{9 in} the state 5 shown in FIG. 8C, the value (expected gain) of the action a ₉ is the maximum at 7.96, so maxQ (5, a) = 7.96 in the above formula. Is set (step S709).

That is,
Q (16,6) ← Q (16,6) +0.3 {2 + 0.9max Q (5, a) −Q (16,6)} = 8.56 + 0.3 {2 + 0.9 * 7.96-8.56}, As a result of the selected action a _6, the state transitions to a new state 5 at time t + 1, and the expected gain value of state 16 at time t based on the expected gain value of each action in the new state is 8.74, which is a higher value. To be updated (step S710).

  The process between steps S706 to S710 is a routine to be repeated. The server 1 continues to present the recommendation until the end of the recommendation (for example, at the end of the conference, after a predetermined time has elapsed).

FIG. 10 is a detailed flowchart regarding the recommendation presenting process shown in step S206 of FIG.
As described above, in the present embodiment, the presentation of the recommendation information is not limited to the conventional information providing method in which text is displayed on the monitor in front of the eyes, and various information can be provided. There is a possibility that the entire space of the conference room is a medium for outputting information. Specifically, the wall, floor, ceiling, furniture, etc. of the space shown in FIG. 1 are used as the output medium. Of course, the above-mentioned glasses 6 with the information display function of the participants are also included as output media. The aim is to actively utilize the elements that make up the space such as furniture and walls in any space as a display medium, and to present unprecedented information to promote the change and convergence of human ideas.

  The server 1 determines whether the information to be recommended in the conference room is, for example, the context of the idea based on the purpose of the set conference, the estimated topic (topics), or the content of each selected action a. Whether the context is of interest or the original context is identified (step S1001). Next, the server 1 assigns each output medium according to the context of the recommendation information (step S1002). For example, the wall surface 9 is assigned as the context of the idea, and the keyword of the words that fly during the meeting is presented on the wall surface 9 in the hope that the server 1 will prompt a new idea. Another wall 10 is assigned as the context of interest. When the table 13 where the participants are facing is assigned to be a context of originality, an important concept is picked up from words that fly during the meeting and presented on the table 13. When a conference participant performs a touch operation on the recommended information presented, a signal corresponding to the touch position is sent back to the server. Next, the server selects various information resources related to the touched recommended information, and receives attention. Information related to the wall surface 10 as the context is displayed, and a new awareness is generated from the information. Further, the projector 7 may be used instead of the wall surfaces 9 and 10.

Further, as described above, when it is determined that there is a keyword that each participant does not know or is not familiar with in words that fly during the meeting, the definition is presented to the glasses 6 with information display function of the participant. The glasses 6 with the information display function of the participants are assigned as personal contexts of the participants together with the personal information terminal 2.
For example, if the tf-idf value of a word in the utterance of the participant X is larger than the threshold value TH3, it is determined that the word is a unique word for the participant X, and only the glasses 6 with the information display function of the participant X are informed. Make a presentation. On the other hand, if the tf-idf value is between TH2 smaller than TH3 (TH3 <tf-idf value <TH2), it is determined that it is an important word, so that other participants can be identified by presenting it in the table 13. To. If the tf-idf value is between TH1 smaller than the threshold TH2 (TH2 <tf-idf value <TH1), it is determined that the word is meaningful, and the wall 9 is presented so that other participants can be identified. To. Further, if the tf-idf value is smaller than the threshold value TH2 (TH1 <tf-idf value <0), it is not truncated as a meaningless word.

Further, in another embodiment, the action a performed under a certain state s of the server 1 is output as a command for changing the entire space of the conference room, not a recommendation of information. For example, if there is an acoustic device that outputs sound in the space, music selection and volume and direction adjustment are performed. In the converged state mode, conferences are often deadlocked, so uplifting music is output to the entire space, and in the divergent state mode, the color of the light from the illumination 11 is changed to a color that relaxes the feelings. It is conceivable that the intensity of the illumination 11 is lowered to make it slightly darker or the color of the wall surfaces 9 and 10 changes. In addition, adjusting the odor (type, amount, direction), wind (air amount and direction), temperature, humidity, and the like from the air conditioner 12 is also included in the action of the server 1.
In another embodiment, vital data such as pulse and body temperature is detected by a watch or wristband with a non-contact sensor or a heart rate sensor installed in the chair 5 or the like and used for adjusting temperature and humidity. May be. In still another embodiment, the height of the chair 5 on which the conference participant is sitting or the front table 13 may be changed, or the chair 5 or the floor may be vibrated. Moreover, if such a command to change the environment of the conference room is given to each participant based on the sensing data that is the image data and audio data of each participant, the conventional uniform simple mere information recommendation It can be expected to generate an opportunity for successful communication that could not be achieved.

  In the above-described example, the present invention is applied to a conference room that is a relatively narrow space in a company or the like, but is not limited thereto. It can be applied at any place where multiple people communicate. For example, the present invention may be applied to communication in a large space such as a lecture hall or a concert hall, or in an outdoor field rather than a closed space such as a room. It also includes application in virtual space, i.e., teleconferencing.

  In this embodiment, it has been described that the server 1 of the idea support system performs overall control. The role played by the server 1 corresponds to so-called artificial intelligence (AI). It is a comprehensive and multi-faceted communication support in which a space is an AI, sharing and communicating the ideas of multiple people, and consensus is formed as many options are created. Providing useful and timely recommended information for each participant while adapting to the content and state of the conference, especially the advantage that reinforcement learning performed as an AI can be designed without full prior understanding of the situation There is a point.

DESCRIPTION OF SYMBOLS 1 Server 2 Portable terminal 3 Camera 4 Microphone 5 Chair 6 Glasses with an information display function 7 Projector 8 Camera 9 Wall surface 10 Wall surface 11 Illumination 12 Air conditioning equipment 13 Table

Claims (10)

A support system for communication by a plurality of participants,
At least one situation data acquisition means for grasping the situation of the communication space by at least one of audio data and image data;
A server that controls any output medium in the communication space using the sensing data obtained by the situation data acquisition means,
The server, at predetermined time intervals,
The sensing data is digitized or encoded to generate the state of the communication space as a state variable at time t,
In order to support communication in the state of the communication space at time t, at least one support item is selected based on an expected value of each support item from a plurality of communication support items related to the state variable,
After outputting a command to execute the selected support item to the output medium, using the expected value of the communication support item related to the state variable at time t + 1 generated based on the sensed data acquired by the situation data acquisition unit, repeatedly updating the expected value of the support item selected at time t.
Support system that is configured as follows.   The support system according to claim 1, wherein updating of the expected value of the support item is based on a reinforcement learning algorithm.   3. The support system according to claim 1, wherein the state variable is a set of indices corresponding to each sensed data obtained by the at least one situation data acquisition unit or a numerical value calculated from each index.   The plurality of indicators include a diversity indicating whether the communication content includes a plurality of viewpoints, a uniformity indicating the degree of communication involvement of each participant, and an activity indicating whether the communication is alternately performed between the plurality of participants. 4. The degree of consent, the degree of consent to the speech or attitude of other participants, and the degree of idea indicating whether communication is performed using words of different categories. The support system described.   5. The server according to claim 1, wherein the server selects the output medium according to a communication context and outputs a command to execute the selected support item for the selected output medium. The support system described.   The output medium includes at least one of a shared object including one or more wall surfaces and a table in the communication space, and a personal information terminal including each participant's portable terminal and glasses-type display device. The support system according to any one of the above.   A shared object including one or more wall surfaces and a table in the communication space is an interface for transmitting a touch operation from a participant who has acted in response to the output of information on the surfaces of the wall surface and the table to the server. The support system according to claim 1, which functions as:   The plurality of communication support items are at least one of sound, light, wind, temperature, humidity, odor, and vibration toward a specific participant among the participants or a part or the whole of the communication space. The support system according to any one of claims 1 to 7, comprising changing an output amount and a type related to the output. A table used for communication support by a plurality of participants,
The server to which the sensing data related to the sound or image acquired using at least one situation data acquisition means is transmitted at every predetermined time interval.
The sensing data is digitized or encoded to generate the state of the communication space as a state variable at time t,
In order to support communication in the state of the communication space at time t, at least one support item is selected based on an expected value of each support item from a plurality of communication support items related to the state variable,
After the execution of the selected support item, the support selected at time t using the expected value of the communication support item related to the state variable at time t + 1 generated based on the sensed data acquired by the situation data acquisition means Update the expected value of the item,
A table that displays information extracted by the server according to the selected support item in repetition of the above.   The table according to claim 9, wherein when the participant performs a touch operation in response to information displayed on the table surface, the table is controlled to return a signal corresponding to a touch position to the server.

Images