JP2019169099A - Conference assistance device, and conference assistance system - Google Patents

Abstract

To solve the problem in which assistance so as to be a useful conference has not been performed, using information on conferences held in the past.SOLUTION: A conference assistance device 1 comprises: an information acquisition unit 4 that acquires conference room information and conference information with these information made confidential, from an information input unit to be installed in a conference room 2 where a conference is held; a conference usefulness determination unit 17 that applies the conference room information and the conference information to a conference usefulness determination model for determining usefulness of the conference held in the past, and outputs a determination result of the determined usefulness of the conference; and an advice selection unit 18 that outputs an advice selected by applying the determination result of the usefulness of the conference to an advice selection model to a robot terminal 104 to be installed in the conference room 2, and thereby makes the robot terminal 104 offer the advice to an attendee in person in the conference room 2.SELECTED DRAWING: Figure 1

Description

  The present invention relates to a conference support apparatus and a conference support system.

  In a meeting where a plurality of participants meet, the agenda is generally decided. In such a meeting, the quality of the meeting may be deteriorated by repeating the same statement or derailing the topic away from the agenda. Therefore, in order to improve the quality of the conference, a method for evaluating the activity of the conference has come to be used.

  In general, it is considered that the activity level of a conference can be evaluated using, for example, the total amount of utterances of all participants (hereinafter referred to as “total amount of utterances”) as an index, and support is provided to increase the activity level of the conference. Thus, the effectiveness of the conference can be improved. Conventionally, as a technique for activating a conference, for example, a technique disclosed in Patent Document 1 is known.

  In this patent document 1, the conference support system acquires “the total amount of utterances of participants in the conference hall A, recognizes the speech operation in the conference hall A based on the acquired total utterance amount, and determines the conference based on the recognition result. The conference activity level of the venue A is evaluated, and the calculated conference activity level is presented to the participants of the conference hall B. "

JP 2009-163431 A

  By the way, many conferences are set every day so that the conference room cannot be reserved, but not all conferences are functioning effectively. Therefore, conventionally, conference support has been performed based on information obtained during analysis of conference participants' utterances.

  However, this is a conference support that uses only the information of the conference being held, and information on conferences held in the past cannot be effectively utilized. This is because there is a problem such as information confidentiality and privacy, and the acquired information in the conference room (speech content, information for identifying a participant, etc.) could not be saved. For this reason, it was not possible to provide sufficient support for past conferences using past conference information.

  Patent Document 1 describes that a meeting support can be made in consideration of the relationship between meeting participants. However, in addition to meetings where attendees give their opinions to each other, there are times during which only one attendee keeps talking and the other attendees listen, such as a quality assurance meeting. There are also long meetings. Although the purpose of the conference is different as described above, simply providing similar support to all the conferences is merely a noisy conference and often does not meet the purpose of the conference. For this reason, the technology disclosed in Patent Document 1 cannot provide sufficient support for an effective conference.

  The present invention has been made in view of such a situation, and an object thereof is to support a conference that is currently held based on conferences held in the past.

  The conference support apparatus according to the present invention includes an information acquisition unit that conceals and acquires area information collected from an area where a conference is held and conference information input from a conference participant through an information terminal, and an information acquisition unit The validity of the conference determined by applying the area information and conference information acquired by the above to the conference validity determination model created by the area information and conference information acquired by the information acquisition unit in the conference held in the past. Apply the meeting validity judgment part that outputs the judgment result and the judgment result of the meeting validity to the advice selection model created by the area information and meeting information acquired by the information acquisition part in the meeting held in the past. The advice selected in this way is output to the information provider installed in the area, so that the information provider can provide advice to meeting attendees in the area. It includes a select unit.

According to the present invention, the currently held conference becomes effective by providing the conference attendee through the information providing unit with the advice selected using the model created based on the conference held in the past. Can help. In addition, since the area information and the conference information are concealed, it is possible to prevent a third party from knowing the detailed contents of the conference.
Problems, configurations, and effects other than those described above will be clarified by the following description of embodiments.

It is a block diagram which shows the structural example of the meeting assistance system which concerns on one embodiment of this invention. It is explanatory drawing which shows the example of the skeleton information which concerns on one embodiment of this invention. It is explanatory drawing which shows the example of a display of the conference start time input screen which concerns on one embodiment of this invention. It is explanatory drawing which shows the example of the meeting information stored in the meeting related information database which concerns on one embodiment of this invention. It is explanatory drawing which shows the example of the waveform diagram of the person action information which concerns on one embodiment of this invention. It is explanatory drawing which shows the example of a display of the conference end time input screen which concerns on one embodiment of this invention. It is explanatory drawing which shows the example of the meeting information stored in the meeting related information database which concerns on one embodiment of this invention. It is a flowchart which shows the process example of the meeting assistance system which concerns on one embodiment of this invention. It is a block diagram which shows the hardware structural example of the computer which concerns on one embodiment of this invention.

  Hereinafter, embodiments for carrying out the present invention will be described with reference to the accompanying drawings. In the present specification and drawings, components having substantially the same function or configuration are denoted by the same reference numerals, and redundant description is omitted.

  FIG. 1 is a block diagram illustrating a configuration example of the conference support system 30. The solid-line arrows shown in the figure represent the flow of data and information when various learning is performed after the conference ends after the data and information acquired during the conference are stored in the conference-related information database 20. . On the other hand, broken arrows represent the flow of data and information used for real-time processing for providing a conference attendee attending a currently held conference with advice for validating the conference. In addition to the conference where a plurality of conference attendees gather in one conference room 2, the conference according to the present embodiment is a meeting with a small number of people, and conference attendees at remote locations attend using the video conference system. Includes meetings.

  The conference support system 30 includes the conference support device 1, the information input unit 3, and the robot terminal 104. The information input unit 3 is installed in the conference room 2 where the conference is held, and conference room information and conference information are input from the information input unit 3. For example, the information input unit 3 includes a camera device 100, a microphone 101, a sensor 102, and an information terminal 103 operated by a conference attendee. The conference room information is a collective term for information output from the camera device 100, the microphone 101, and the sensor 102 to the conference support device 1, and is used as an example of area information.

  For example, a humanoid robot terminal 104 is also installed in the conference room 2. When providing advice to meeting attendees, this ROBO terminal 104 displays meeting room information and meeting information on a display or projects them on a projector to visualize these information to meeting attendees. To do. The conference room 2 is an example of an area where a conference is held. In addition to the conference room 2, the conference may be held in an open space or the like. When a conference is held in an open space, information may be input to the conference support device 1 from the camera device 100, the microphone 101, or the like installed at a location away from the open space.

The camera device 100 outputs a video (video data) taken in the conference room 2 to the conference support device 1.
The microphone 101 outputs the voice (voice data) collected in the conference room 2 to the conference support apparatus 1.

  The sensor 102 is, for example, a temperature sensor, a humidity sensor, an illuminance sensor, a thermo sensor, or the like. When a temperature sensor is used as the sensor 102, temperature information in the conference room 2 is output to the conference support apparatus 1. When a humidity sensor is used as the sensor 102, humidity information in the conference room 2 is output to the conference support apparatus 1. When an illuminance sensor is used as the sensor 102, illuminance information in the conference room 2 is output to the conference support apparatus 1. When a thermosensor is used as the sensor 102, body temperature information of the conference attendee is output to the conference support apparatus 1. Thus, various information output from the sensor 102 is output to the conference support apparatus 1 as sensor data.

  The information terminal 103 is operated by a conference attendee to input conference information at the start and end of the conference. The conference information is a collective term for various types of information regarding the conference attended by the conference attendees, and details thereof will be described later with reference to FIGS. 4 and 7. The conference information input to the information terminal 103 is output to the conference support apparatus 1. As the information terminal 103, for example, a PC (Personal Computer), a smartphone, or the like is used.

  The robot terminal 104 is used as an example of an information providing unit that provides advice to a conference attendee for validating an ongoing conference. As the information providing unit, an information terminal such as a tablet capable of displaying advice in characters may be used. However, depending on the seat positions of the attendees of the conference, it may be difficult to see the advice displayed on the information terminal such as a tablet. For this reason, a humanoid robot terminal 104 may be used as the information providing unit. When the robot terminal 104 provides audio or video advice, gestures and expressions that face the conference attendees are used. If the Robo terminal 104 provides advice to the meeting attendees by an operation close to that of a human, it is considered that the attendees of the meeting can easily accept the advice from the Robo terminal 104 rather than giving advice to each other.

  For example, the conference support apparatus 1 may be installed in a local environment by a general PC or the like, or may be a Web server constructed on a cloud accessible via a network. The conference support apparatus 1 includes an information acquisition unit 4, an advice selection learning unit 15, a conference effectiveness learning unit 16, an advice selection unit 18, and a conference effectiveness determination unit 17. The conference support apparatus 1 includes a conference related information database (DB: DataBase) 20, an advice selection model database 21, and a conference validity determination model database 22.

  The information acquisition unit 4 conceals and acquires the conference room information and the conference information from the information input unit 3. The information acquisition unit 4 includes a video acquisition unit 10, a person action extraction unit 11, a sound information acquisition unit 12, a sensor data acquisition unit 13, and a conference information acquisition unit 14.

The video acquisition unit 10 acquires video of a conference held in the conference room 2 taken by the camera device 100.
The person action extraction unit 11 converts the meeting attendee in the video acquired by the video acquisition unit 10 into skeleton information, and the person action information of the conference attendant extracted from the skeleton information is stored in the meeting related information database 20 together with time information. Store. This person action information includes not only the skeleton information but also the actions of the conference attendees extracted by the person action extraction unit 11 based on the skeleton information. The actions of the attendees of the conference are labeled, for example, “sitting”, “standing”, “raising hands”, etc., and stored in the conference related information database 20.

Here, the skeleton information will be described with reference to FIG.
FIG. 2 is an explanatory diagram illustrating an example of skeleton information.
On the upper side of FIG. 2, an example of the skeleton information 41 of the conference attendant standing near the whiteboard is shown. The skeleton information is, for example, information obtained by processing a meeting attendee into a humanoid silhouette for each frame of a video and connecting dots indicating human characteristic parts (head, joints, etc.) with lines. Since the skeletal information is represented by connecting dots indicating human features with lines, the direction and behavior of the attendees are indicated by the movement of the lines.

  In the lower part of FIG. 2, examples of the skeleton information 42 and 43 of sitting conference attendees are shown. For example, it can be seen from the skeletal information 42 that the conference attendee is sitting on a chair in the conference room 2 and placing his hand on the desk. In addition, it can be seen from the skeleton information 43 that the attendees are talking with their hands raised.

  By converting the conference attendees into skeleton information in this way, it is possible to manage how the conference attendees acted in the conference while protecting the personal information of the conference attendees. In this way, by converting the meeting attendees included in the video into the skeleton information by the person action extracting unit 11, a third party cannot individually identify the meeting attendees. This protects the personal information of the meeting attendees.

Returning again to the description of FIG.
The sound information acquisition unit 12 acquires the sound emitted at the conference collected by the microphone 101, and includes the volume of the sound emitted by the conference attendant and the sound generation position obtained based on the acquired sound. The sound information is stored in the conference related information database 20. The sound source position and sound volume are used to read the information exchange status between the conference attendees. Moreover, since the sound information acquisition unit 12 does not acquire the utterance content for each conference attendee, the psychological burden on the conference attendee can be reduced. And the sound information acquisition part 12 stores a sound source position and volume information in the meeting relevant information database 20 with time information.

  The sensor data acquisition unit 13 acquires sensor data output from the sensor 102 that detects the state of the conference room 2. The sensor data is used, for example, for grasping changes in room temperature and humidity in the conference room 2. And the sensor data acquisition part 13 stores sensor data in the meeting relevant information database 20 with time information.

  The conference information acquisition unit 14 acquires conference information input from the information terminal 103 by the conference attendee at the start and end of the conference, and stores this conference information in the conference related information database 20 together with the time information. The conference information includes conference start time information (see FIG. 3 to be described later) input by the conference attendee at the start of the conference and conference end information (see FIG. 7 to be described later) input at the end of the conference. is there. The conference start time information includes, for example, the purpose of the conference, the title of the conference attendee, gender, start date and time, and the like. The information at the end of the meeting includes, for example, the impression of the meeting, the advice that the meeting attendee thinks necessary, the advice provided from the robot terminal 104, and the like.

  The conference related information database 20 stores conference room information and conference information acquired by the information acquisition unit 4 in a concealed manner. Among the information stored in the conference related information database 20, the conference room information is, for example, information acquired by the person action extraction unit 11, the sound information acquisition unit 12, and the sensor data acquisition unit 13, that is, person action information and sound information. And sensor data. The person action information, sound information, and sensor data are stored as images in the conference related information database 20 (see FIG. 4 described later). However, sensor data may be excluded from the conference room information.

  The advice selection learning unit 15 learns conference room information and conference information in conferences held in the past, read from the conference related information database 20, and creates an advice selection model for selecting advice. At this time, the advice selection learning unit 15 uses the conference end time information stored in the conference related information database 20 to select an advice selection model used when selecting an advice necessary for an effective conference in the future. Create using machine learning such as deep learning.

  For this reason, the advice selection learning unit 15 provides the goal achievement level 515, the advice 516 considered necessary for achieving the goal, and the advice input by the conference attendee on the conference end input screen (see FIG. 6 described later). The advice selection model is created using information such as the time 517, the provided advice 518, and the provided advice effect 519. This advice selection model is considered to be necessary for meeting purpose 502 inputted through the information terminal 103 at the start of the meeting, meeting impression 513, meeting atmosphere 514, degree of achievement 515, and achievement of the purpose inputted through the information terminal 103 at the end of the meeting. The advice 516, the provision timing 517 of the advice, the provided advice 518, and the provided advice effect 519 are different depending on the learning results.

  Further, the advice selection learning unit 15 creates a new advice selection model by appropriately combining these pieces of information. For this reason, the advice selection learning unit 15 feeds back the conference information input through the information terminal 103, learns the content of the advice to be provided to the conference attendee, and the timing when the robot terminal 104 provides the advice, and selects the advice. Create a model. Then, the advice selection learning unit 15 stores an advice selection model that is a learning result of the advice selection in the advice selection model database 21.

  The conference effectiveness learning unit 16 learns conference room information (personal behavior information, sound information, sensor data) and conference information read from the conference related information database 20, and is used to determine the effectiveness of the conference. Create an effectiveness assessment model. Here, the conference information learned by the conference effectiveness learning unit 16 includes conference start time information and conference validity included in the conference end time information. Here, the effectiveness of the conference is information including a conference impression 513, a conference atmosphere 514, a goal achievement level 515, etc. with respect to the conference attended by the conference attendee on the conference end input screen (see FIG. 6 described later). is there.

  The conference effectiveness learning unit 16 is concealed with the characteristics of personal information and confidential information, and the information stored in the conference related information database 20 and the conference attendee input from the conference end input screen. Based on effectiveness, conference effectiveness learning is performed. In this conference effectiveness learning, the conference effectiveness learning unit 16 creates a conference effectiveness determination model for determining the effectiveness of the conference, for example, using machine learning such as deep learning. This meeting validity determination model differs, for example, as the learning result corresponding to the meeting name, the purpose of the meeting, and the meeting attendee input through the information terminal 103 at the start of the meeting. In addition, a new learning effectiveness determination model is created by appropriately combining these pieces of information.

  Therefore, the conference effectiveness learning unit 16 feeds back conference information input through the information terminal 103 at the end of the conference, learns the effectiveness of the conference, and creates a conference validity determination model. Then, the conference effectiveness learning unit 16 stores a conference effectiveness determination model, which is a learning result for determining the effectiveness of the conference, in the conference effectiveness determination model database 22. The conference validity determination model database 22 stores a plurality of conference validity determination models created for each past conference.

  The conference validity determination unit 17 applies the conference room information (personal action information during meeting, sound information, sensor data) and conference information acquired by the information acquisition unit 4 to the conference validity determination model. Outputs the validity judgment result of. Here, the conference validity determination model is created based on the conference room information and the conference information acquired by the information acquisition unit 4 at a conference held in the past. Accumulated in the judgment model database 22. For this reason, the conference validity determination unit 17 reads out from the conference validity determination model database 22 a conference validity model suitable for the currently held conference, by combining various information acquired from the information acquisition unit 4. Then, the conference validity determination unit 17 outputs the determination result of the conference validity obtained by executing the conference validity model to the advice selection unit 18 and the robot terminal 104.

  Here, the conference validity determination unit 17 determines the effectiveness of the conference even during the conference by using the information acquired from the information input unit 3 by the information acquisition unit 4 during the conference, and the conference validity. Are output to the advice selection unit 18 and the robot terminal 104. As a result, the robot terminal 104 can provide the meeting validity determination result to the meeting attendee during the meeting. Here, the determination result of the validity of the conference provided by the robo terminal 104 indicates, for example, by a numerical value whether or not the current conference is valid. For this reason, the attendees of the meeting will behave and speak to increase the effectiveness of the meeting.

  The advice selection unit 18 selects the advice by applying the determination result of the validity of the currently held conference output from the conference validity determination unit 17 to the advice selection model, and outputs this advice to the robot terminal 104. To do. Here, the advice selection model is created based on the conference room information and the conference information acquired by the information acquisition unit 4 at the conference held in the past, and many advice selection models are accumulated in the advice selection model database 21. Has been. For this reason, the advice selection unit 18 reads out an advice selection model suitable for the currently held conference from the advice selection model database 21 based on the determination result of the effectiveness of the conference, and selects the advice. Therefore, the robot terminal 104 can provide advice suitable for the currently held conference to the conference attendees in the conference room 2. Then, according to the advice, the attendees of the conference can improve the effectiveness of the conference, for example, by taking a break or by uttering little by little.

FIG. 3 is an explanatory diagram illustrating a display example of a conference start time input screen.
The conference start input screen is displayed on the information terminal 103 at the start of the conference. At the start of the conference, the conference attendee operating the information terminal 103 inputs the conference start information through the conference start input screen. The conference start time information input through the information terminal 103 is acquired by the conference information acquisition unit 14 and stored in the conference related information database 20.

  The conference start input screen includes items that allow the user to input a conference name 501, a conference purpose 502, a conference start date and time 503, the number of conference attendees 504, and a conference attendee seat 505. If the conference date and time, location, attendees, etc. are registered in advance in a conference system (not shown) managed separately from the conference support system 30, the information transferred from the conference system can be used for the conference. You may make it display on the input screen at the time of a meeting start in the state in which the date of an event, an event place, an attendee, etc. were embedded.

  Here, the seat position of the conference attendant in the conference room 2 is designated as the conference attendee seat 505. Then, only the position information of the meeting attendee is input so as not to identify the meeting attendee. For example, “K” shown in the figure represents a meeting attendee who is in a managerial position, and “H” represents a meeting attendee in a general position. Note that only the gender of the conference attendee may be input to the conference attendee seat 505. For example, if the conference attendee is a male, “M” is input, and if the conference attendee is a female, “W” is input. Moreover, you may input combining the position information and gender of a meeting attendee. In this case, “KM” is displayed if the attendee of the meeting is male and has a managerial position. If the attendee is a woman and has a regular position, “HW” is displayed.

After the necessary information is input on the conference start input screen, the conference is held.
Here, the meeting information stored in the meeting related information database 20 will be described with reference to FIG.
FIG. 4 is an explanatory diagram showing an example of the conference related information 200a stored in the conference related information database 20. The conference related information 200a is conference start time information input from the conference start time input screen.

  The conference related information 200a is a table that stores conference information collected for each conference every time the conference is held. The conference related information 200a includes fields for storing personal action information, sound information, and sensor data in addition to a field for storing conference start time information input from the conference start time input screen.

In the field for storing the conference start information, the conference name 501, the conference purpose 502, the conference start date and time 503, the number of conference attendees 504, and the conference attendee seat 505 shown in FIG. 3 are stored.
In the fields for storing the personal action information, the sound information, and the sensor data, waveform diagrams of the personal action information, the sound information, and the sensor data that change with time are stored.

Here, among the three types of waveform diagrams stored in the conference related information 200a, the waveform diagram of the human behavior information will be described with reference to FIG.
FIG. 5 is an explanatory diagram illustrating an example of a waveform diagram of human behavior information. This waveform diagram shows that a conference is started in a conference (meeting started on January 13, 2018 at 10:30) specified by the first record in the field for storing conference start information in the conference related information 200a. Later, it shows how the attendees acted by 1 hour.

  The waveform chart shown in FIG. 5 shows that among the four meeting attendees shown in the attendee seat field of FIG. 4, the upper left attendee is “KM1”, the upper right attendee is “KM2”, and the lower right hand corner. It is a graph which shows the person action information of "KM3" and the attendant of lower left as "HW". Each graph represents, for example, the movement of the right hand for each conference participant obtained from the skeleton information.

  It is shown that the attendee KM1 represented by the solid line is moving best when 20 minutes have passed since the start of the conference. In addition, it is shown that the attendee KM2 represented by a two-dot chain line moves best when 40 minutes have elapsed since the start of the conference. For this reason, it is considered that the attendees KM1 and KM2 are making gestures or explaining in front of a whiteboard installed in the conference room 2, for example. On the other hand, the attendee KM3 represented by the broken line and the attendee HW represented by the alternate long and short dash line do not move much from the start to the end of the conference. For this reason, it is considered that the attendees KM3 and HW are listening to other attendees.

  Such a waveform diagram is also stored in sound information and sensor data as shown in FIG. The four types of graphs shown in the sound information waveform diagram identify meeting attendees for each line type of the graph, similar to the graphs shown in the human behavior information waveform diagram. The four types of graphs shown in the waveform diagram of the sensor data identify temperature, humidity, etc. for each line type of the graph. For this reason, it becomes possible to determine the effectiveness of the conference without specifying the content uttered in the conference by combining the person action information, the sound information, and the sensor data.

  4 is plotted in real time during a meeting, for example, and is converted into image data and stored in the meeting related information database 20 at the end of the meeting. Only by looking at these graphs stored in the conference related information database 20, the content of the conference can be concealed because the third party cannot know what was spoken in the conference. When the advice selection learning unit 15 and the conference effectiveness learning unit 16 read these graphs from the conference related information database 20, simply specify the time from the start of the conference and analyze the effectiveness of the conference at that time. Is possible. Thus, even if the content of the conference is concealed, the advice selection learning unit 15 and the conference effectiveness learning unit 16 can learn.

FIG. 6 is an explanatory diagram showing a display example of the input screen at the end of the conference.
The input screen at the end of the conference is displayed on the information terminal 103 at the end of the conference. At the end of the conference, the conference attendee operating the information terminal 103 inputs the conference end information through the conference end input screen. Information at the end of the conference input through the information terminal 103 is acquired by the conference information acquisition unit 14 and stored in the conference related information database 20.

  Of the input screen at the end of the conference, items 511 and 512 display the conference name 501 and the conference purpose 502 input on the input screen at the start of the conference at the start of the conference. The conference end input screen includes a conference impression 513 as a conference result, a conference atmosphere 514, an objective achievement level 515, an advice 516 that the conference attendee considers necessary for achieving the objective, a timing 517 of providing the advice, and the robot terminal 104. Is provided with items that can input the advice 518 provided by the user, the effect 519 of the provided advice, and the seat position 520 of the information input person.

  The conference impression 513, the conference atmosphere 514, the goal achievement level 515, the advice provision time 517, and the advice effect 519 provided from the ROBO terminal 104 during the conference are all displayed on the input screen when the conference attendee By moving the displayed slider to the left and right, it is possible to input a numerical value between “0” and “100”. For example, if the conference impression 513, the conference atmosphere 514, and the provided advice effect 519 are bad, a value close to “0” is input, and if it is good, a value close to “100” is input. A numerical value close to “0” is input if the goal achievement level 515 is near unachieved, and a numerical value close to “100” is input if it is close to achievement. If the advice provision time 517 is close to the start, a value close to “0” is input, and if close to the end, a value close to “100” is input.

The advice 516 that the conference attendee considers necessary for achieving the purpose is an item that is text-inputted by the conference attendee.
The advice 518 provided from the robot terminal 104 that the conference attendee considers effective for the conference is selected by the advice selection unit 18 and the advice provided from the robot terminal 104 to the conference attendee is displayed.

  Note that the input items on the conference end input screen shown in FIG. 6 are examples. For example, a plurality of advices may be registered as the advice 516 required for achieving the purpose. A plurality of meeting attendees may input information at the end of the meeting.

Next, the conference information stored in the conference related information database 20 will be described with reference to FIG.
FIG. 7 is an explanatory diagram illustrating an example of the conference related information 200b stored in the conference related information database 20. The conference related information 200b is conference end time information input from the conference end time input screen.

  The conference related information 200b is a table that stores conference end time information input from the conference end time input screen when the conference ends. The conference related information 200b includes a field for storing conference end time information input from the conference end time input screen shown in FIG. 6 in addition to the conference name and conference end date and time embedded at the end of the conference. Of each information stored in the conference related information 200b, the conference impression, the conference atmosphere, and the goal achievement level are the conference impression 513, the conference atmosphere 514, and the goal achievement level on the conference end input screen shown in FIG. The numerical value input from 515 is stored. Further, the advice provision time stored in the conference related information 200b is stored with a numerical value inputted at the advice provision time 517 on the conference end time input screen or a word converted from this numerical value.

Next, a processing example of the conference support system 30 will be described.
FIG. 8 is a flowchart illustrating a processing example of the conference support system 30.

  First, the conference support system 30 stands by until the conference attendee inputs conference start information through the information terminal 103 (S1). If the conference start information is not input (NO in S1), this process is terminated.

  On the other hand, when the conference start time information is input (YES in S1), the information acquisition unit 4 determines whether the conference room information (information in the conference room 2) input from the information input unit 3 can be acquired. (S2). As described above, the conference room information input from the information input unit 3 is video, sound information, and sensor data in the conference room 2. If the conference room information cannot be acquired (NO in S2), this process is terminated.

  On the other hand, if the information acquisition part 4 can acquire conference room information (YES of S2), it will progress to the process of step S3. At this time, the information acquisition unit 4 acquires the skeleton information of the conference attendee from the video in the conference room 2. The information acquisition unit 4 acquires sound information, sensor data, and conference start time information. The conference room information and conference start information acquired by the information acquisition unit 4 are stored in the conference related information database 20.

  Next, the conference validity determination unit 17 determines the effectiveness of the conference using the conference room information and the conference start time information (S3). Then, the conference validity determination unit 17 outputs the determination result of the conference validity determined using the conference validity determination model read from the conference validity determination model database 22 to the robot terminal 104 and the advice selection unit 18. .

  Next, the advice selecting unit 18 selects an advice suitable for the conference using the advice selection model read from the advice selection model database 21 with the determination result of the effectiveness of the conference as an input (S4). Then, the advice selection unit 18 outputs the selected advice to the robot terminal 104.

  The robot terminal 104 provides the conference attendee with a determination result of the validity of the currently held conference and advice (S5). For example, when the advice selection unit 18 selects “break” as advice, the robot terminal 104 prompts the conference attendee to shift to a break by voice and action. This makes it possible to improve the situation of the conference because the conference participants smoothly transition to the break as compared with the case where the attendees are encouraged to transition to the break.

  Next, the conference support system 30 stands by until the conference attendee inputs information at the end of the conference through the information terminal 103 (S6). If the information at the end of the conference is not input (NO in S6), the conference is being held, and thus the processes in steps S2 to S6 are repeated as appropriate. Here, the timing at which the processes of steps S2 to S6 are performed is learned by the advice selection learning unit 15 based on, for example, the advice provision timing 517 and the provided advice effect 519 input as the conference end information. This is the time during the meeting calculated from the advice selection model. A suitable time for providing advice is calculated from the advice selection model. Then, by repeating the processes of steps S2 to S6, a plurality of advices are provided to the meeting attendees while the meeting is being held.

  When the conference ends, the conference end information is input by the conference attendee through the conference end input screen (YES in S6). When the conference end information is input, the advice selection learning unit 15 performs advice selection learning using the information read from the conference related information database 20, and creates an advice selection model (S7). Then, the advice selection learning unit 15 stores the created advice selection model in the advice selection model database 21.

  Next, the conference effectiveness learning unit 16 performs conference effectiveness learning using the information read from the conference related information database 20, and creates a conference effectiveness determination model (S8). Then, the conference effectiveness learning unit 16 stores the created conference effectiveness determination model in the conference effectiveness determination model database 22 and ends this processing.

Next, the hardware configuration of the computer C configuring the conference support apparatus 1 will be described.
FIG. 9 is a block diagram illustrating a hardware configuration example of the computer C.
The computer C is hardware used as a so-called computer. The computer C includes a CPU (Central Processing Unit) C1, a ROM (Read Only Memory) C2, and a RAM (Random Access Memory) C3 connected to the bus C4. Furthermore, the computer C includes a nonvolatile storage C5 and a network interface C6.

  The CPU C1 reads out the program code of software that realizes each function according to the present embodiment from the ROM C2, and executes it. In the RAM C3, variables, parameters and the like generated during the arithmetic processing are temporarily written. For example, a program having functions of the information acquisition unit 4, the advice selection learning unit 15, the conference effectiveness learning unit 16, the conference effectiveness determination unit 17, and the advice selection unit 18 in the conference support apparatus 1 is stored in the ROM C2. Then, the function of each unit is realized by the CPU C1 appropriately reading and executing the program from the ROM C2.

  As the nonvolatile storage C5, for example, an HDD (Hard Disk Drive), an SSD (Solid State Drive), a flexible disk, an optical disk, a magneto-optical disk, a CD-ROM, a CD-R, a magnetic tape, a nonvolatile memory, or the like is used. It is done. In the nonvolatile storage C5, in addition to the OS (Operating System) and various parameters, a program for causing the computer C to function is recorded. For example, the conference related information database 20, the conference validity determination model database 22 and the advice selection model database 21 in the conference support apparatus 1 are constructed in the nonvolatile storage C5. Note that the ROM C2 and the non-volatile storage C5 permanently record programs and data necessary for the operation of the CPU C1, and are computer-readable non-transitory storing the programs executed by the computer C. Used as an example of a recording medium.

  For example, a network interface card (NIC) or the like is used as the network interface C6, and various types of data can be transmitted and received between devices via a LAN (local area network) connected to a terminal, a dedicated line, or the like. is there. For this reason, the information acquisition part 4 acquires various information from the information input part 3 via the network interface C6. Further, the conference validity determination unit 17 outputs the determination result of the conference validity to the robot terminal 104 via the network interface C6, and the advice selection unit 18 outputs the advice via the network interface C6.

  In the conference support system 30 according to the embodiment described above, the information acquisition unit 4 provided in the conference support device 1 displays various types of information acquired from the information input unit 3 provided in the conference room 2 as conference related. Store in the information database 20. Then, the conference support apparatus 1 creates an advice selection model based on the information stored in the conference related information database 20, and also creates a conference validity determination model. During the conference, the advice selected by the conference support apparatus 1 based on the advice selection model and the determination result of the conference validity determined based on the conference validity determination model are sent from the robot terminal 104 to the conference attendee. Provided.

  Accordingly, the conference support apparatus 1 can provide support for the attendees to increase the effectiveness of the conference. In addition, the conference attendee can improve the conference by improving the conference according to the advice provided from the robot terminal 104 during the conference and the determination result of the conference validity.

  Also, at the start of the conference, conference start information is input by the conference attendee operating the information terminal 103, and at the end of the conference, the conference end information is input by the conference attendee. When the conference attendee inputs information at the end of the conference, an advice 516 considered by the conference attendant himself or an effect 519 on the advice provided from the robot terminal 104 can be input. Thereby, the conference support apparatus 1 can receive feedback of the advice that the conference attendee who actually attended the conference needs and thinks is effective.

  Therefore, it is possible to continue by re-learning the conference validity judgment model and the advice selection model using the evaluation result regarding the effectiveness judgment and advice of the conference itself by the conference attendee entered at the end of the conference by the conference attendee. Can reflect the evaluation of the attendees. And the meeting assistance apparatus 1 can perform the meeting assistance for making a meeting smooth by continuing updating the timing which provides advice during holding of a meeting, and the content of advice to the optimal state.

  Further, the conference information stored in the conference related information database 20 is visualized and provided to the conference attendee during or after the conference. Here, as shown in FIGS. 2 and 4, the person behavior information of the attendees of the conference held in the past is stored as an image based on the skeleton information. Also, as shown in FIG. 4, the sound information and sensor data are also stored in the form of images instead of making individual information identifiable. For this reason, personal action information, sound information, and sensor data are concealed, and personal information cannot be specified.

  By visualizing and providing the human behavior information, sound information, and sensor data thus concealed to the robot terminal 104, etc., the attendees of the conference how the conference was held and whether the conference was activated You can feel that. In addition, meeting attendees can understand that each piece of information acquired during the meeting is concealed and personal information is protected, thus reducing the resistance to each piece of information being acquired during the meeting. Can do.

The present invention is not limited to the above-described embodiments, and various other application examples and modifications can of course be taken without departing from the gist of the present invention described in the claims.
For example, the above-described embodiment is a detailed and specific description of the configuration of the apparatus and the system in order to explain the present invention in an easy-to-understand manner, and is not necessarily limited to one having all the described configurations. Moreover, it is also possible to add, delete, or replace another configuration with respect to part of the configuration of the present embodiment.
Further, the control lines and information lines indicate what is considered necessary for the explanation, and not all the control lines and information lines on the product are necessarily shown. Actually, it may be considered that almost all the components are connected to each other.

  DESCRIPTION OF SYMBOLS 1 ... Conference support apparatus, 2 ... Conference room, 3 ... Information input part, 4 ... Information acquisition part, 10 ... Image | video acquisition part, 11 ... Person action extraction part, 12 ... Sound information acquisition part, 13 ... Sensor data acquisition part, DESCRIPTION OF SYMBOLS 14 ... Meeting information acquisition part, 15 ... Advice selection learning part, 16 ... Meeting effectiveness learning part, 17 ... Meeting effectiveness determination part, 18 ... Advice selection part, 20 ... Meeting related information database, 21 ... Advice selection model database 21 ... Conference validity determination model database, 30 ... Conference support system, 100 ... Camera device, 101 ... Microphone, 102 ... Sensor, 103 ... Information terminal, 104 ... Robot terminal

Claims (8)

An information acquisition unit that conceals and acquires area information collected from an area where a conference is held, and conference information input from an attendee of the conference through an information terminal;
The area information and the conference information acquired by the information acquisition unit are applied to the conference validity determination model created by the area information and the conference information acquired by the information acquisition unit in a meeting held in the past. A conference validity determination unit that outputs a determination result of the effectiveness of the conference determined
The advice selected by applying the determination result of the effectiveness of the conference to the advice selection model created by the area information and the conference information acquired by the information acquisition unit in the conference held in the past, A meeting support apparatus comprising: an advice selecting unit that outputs the advice from the information providing unit to a conference attendant in the area by outputting to an information providing unit installed in the area. Furthermore, the area information acquired by concealment by the information acquisition unit and the conference related information database in which the conference information is stored,
Learning the area information and the conference information read from the conference related information database, and creating a conference validity determination model, a conference effectiveness learning unit,
A conference effectiveness determination model database for storing the conference effectiveness determination model,
The conference validity determination unit outputs the determination result of the conference validity determined using the conference validity determination model read from the conference validity determination model database to the information providing unit and the advice selection unit. The conference support apparatus according to claim 1. Further, learning the area information and the conference information read from the conference related information database, and creating the advice selection model for selecting the advice provided to the conference attendee during the conference An advice selection learning section;
An advice selection model database for storing the advice selection model,
The conference support apparatus according to claim 2, wherein the advice selection unit selects the advice using the advice selection model read from the advice selection model database. The information acquisition unit includes the purpose of the conference input from the information terminal at the start of the conference, and the validity of the conference attended by the conference attendee input from the information terminal at the end of the conference. Conference information acquisition that stores the conference information in the conference related information database including the effect on the advice provided by the information providing unit during the conference and the advice that the conference attendee considers effective for the conference The conference support device according to claim 3, further comprising a unit. The advice selection learning unit learns the content of the advice to be provided to the conference and the timing at which the information providing unit provides the advice by using the conference information input through the information terminal as feedback, and the advice Create a selection model,
The conference support apparatus according to claim 4, wherein the conference effectiveness learning unit learns the effectiveness of the conference by using the conference information input through the information terminal as feedback, and creates the conference effectiveness determination model. . The information acquisition unit
A video acquisition unit that acquires the video of the conference from a camera device;
Converting the meeting attendee in the video into skeleton information, and storing the skeleton information and the person action information of the meeting participant extracted from the skeleton information in the meeting related information database;
Sound obtained in the conference is acquired from the microphone, and the sound information including the volume of the sound emitted by the conference attendee and the position where the sound is generated, which is obtained based on the sound, is stored in the conference related information database. A sound information acquisition unit;
A sensor data acquisition unit that stores sensor data acquired from a sensor that detects the state of the area in the conference related information database;
The meeting support apparatus according to claim 5, wherein the area information includes the person behavior information, the sound information, and the sensor data, and is stored as an image in the meeting related information database. An information input unit that is installed in an area where a conference is held, and that inputs area information collected from the area, and conference information that is input from an attendee of the conference through an information terminal;
An information acquisition unit for concealing and acquiring the area information and the conference information from the information input unit;
The area information and the conference information acquired by the information acquisition unit are applied to the conference validity determination model created by the area information and the conference information acquired by the information acquisition unit in a meeting held in the past. A conference validity determination unit that outputs a determination result of the effectiveness of the conference determined
Applying the determination result of the effectiveness of the conference to the advice selection model created by the area information and the conference information acquired by the information acquisition unit at the conference held in the past, and outputting the selected advice An advice selection section;
A conference support system comprising: an information providing unit that is installed in the area and that provides the advice output from the advice selection unit to conference attendees in the area. The conference support system according to claim 7, wherein the information providing unit is a humanoid robot terminal that visualizes and provides the area information and the conference information.