WO2022123688A1

WO2022123688A1 - Scenario control method, scenario control device, and scenario control program

Info

Publication number: WO2022123688A1
Application number: PCT/JP2020/045856
Authority: WO
Inventors: 充裕後藤; 済央野本; 哲小橋川; 史朗小澤
Original assignee: 日本電信電話株式会社
Priority date: 2020-12-09
Filing date: 2020-12-09
Publication date: 2022-06-16

Abstract

The present invention provides a scenario control method for changing a scenario in a presenter system that presents slide contents by controlling the operation of a presenter agent on the basis of the scenario. This scenario control method comprises: a first step of estimating a current audience state regarding the interest and attitude of the entire audience toward a presentation in an online meeting on the basis of face information of the audience acquired during the presentation and the contents of posts posted by the audience during the presentation; a second step of determining an ideal audience state for increasing the interest of the entire audience toward the presentation and concentrating the attitude of the entire audience on the basis of the current audience state and the degree of progress of the scenario; a third step of determining a correction scenario for transitioning the current audience state to the ideal audience state; and a fourth step of changing the scenario to the correction scenario.

Description

Scenario control method, scenario control device, and scenario control program

The present invention relates to a scenario control method, a scenario control device, and a scenario control program.

A presenter system that automatically performs a presentation by combining a slide and a presenter agent is known (see Patent Documents 1 and 2). In the presenter system, in order to let the presenter agent convey the content of the presentation to the audience, the utterance content of each slide constituting the presentation is created as a scenario. In addition, in order to emphasize and supplement the utterance content and the utterance script in which the utterance content is transcribed, non-verbal movements such as face orientation and arm movement are also created as scenarios. Based on a scenario that includes utterance content and non-verbal movement, the presenter agent appeals the slide content to the audience by uttering the slide content with gestures and displaying the utterance script on the screen. ..

JP-A-2019-144732 Japanese Unexamined Patent Publication No. 2020-86774

In Patent Document 1, in the above presenter system, a scenario is created while assuming the audience state in advance. Therefore, if there is a mistake in the assumption of the audience status, even if the presenter agent is controlled according to the scenario, the appealing power of the presentation is low. In Patent Document 2, the content of the presentation is dynamically changed according to the state of the audience. However, it is difficult to apply to an online conference environment because it is intended for an audience in a real space environment.

The present invention has been made in view of the above circumstances, and an object of the present invention is to provide a technique capable of improving the appealing power of a presentation in an online conference environment.

The scenario control method of one aspect of the present invention is a scenario control method for changing the scenario in a presenter system that controls the operation of the presenter agent based on the scenario to present the slide contents, and is acquired during the presentation at an online conference. Based on the facial information of the audience and the content posted by the audience during the presentation, the first step of estimating the current audience state regarding the interest and attitude of the entire audience toward the presentation, and the current present. Based on the audience status and the progress of the scenario, the second step of determining the ideal audience status that raises the audience's interest in the presentation and concentrates the attitude of the entire audience, and the current present. A third step of determining a correction scenario for transitioning the audience state to the ideal audience state, and a fourth step of changing the scenario to the correction scenario are performed.

The scenario control device of one aspect of the present invention is a scenario control device that changes the scenario in a presenter system that controls the operation of the presenter agent based on the scenario to present the slide contents, and is acquired during the presentation at an online conference. Based on the facial information of the audience and the content posted by the audience during the presentation, an estimation unit that estimates the current audience state regarding the interest and attitude of the entire audience toward the presentation, and the current audience state. And, based on the progress of the scenario, the first decision-making part that determines the ideal audience state that raises the interest of the entire audience in the presentation and concentrates the attitude of the entire audience, and the current audience. A second determination unit for determining a correction scenario for transitioning the state to the ideal audience state, and a change unit for changing the scenario to the correction scenario are provided.

The scenario control program of one aspect of the present invention is a scenario control program that causes a computer to execute the above scenario control method.

According to the present invention, it is possible to provide a technique capable of improving the appeal of a presentation in an online conference environment.

FIG. 1 is a diagram showing an overall configuration of a presenter system. FIG. 2 is a diagram showing a scenario example of each slide. FIG. 3 is a diagram showing an example of a state transition rule. FIG. 4 is a diagram showing an example of a scenario correction rule. FIG. 5 is a diagram showing an example of scenario correction history information. FIG. 6 is a flow chart showing a characteristic processing operation of the scenario control device. FIG. 7 is a diagram showing images of facial expression analysis and chat posting analysis. FIG. 8 is a diagram showing an example of an audience state. FIG. 9 is a diagram showing an example of determining the audience state of the transition destination. FIG. 10 is a diagram showing an example of a correction scenario between each state. FIG. 11 is a diagram showing an example of a correction scenario at the time of transition to the same state. FIG. 12 is a flow chart showing a specific example of the processing operation of the scenario control device. FIG. 13 is a flow chart showing a specific example of the selection process of the correction scenario. FIG. 14 is a flow chart showing a specific example of the scenario change process. FIG. 15 is a diagram showing a modified example of the scenario. FIG. 16 is a diagram showing a modified example of the presenter system. FIG. 17 is a diagram showing a hardware configuration example of the scenario control device.

Hereinafter, embodiments of the present invention will be described with reference to the drawings. In the description of the drawings, the same parts are designated by the same reference numerals and the description thereof will be omitted.

[Outline of the invention]
The present invention applies a presenter system by combining slides and a presenter agent to an online conference system, and dynamically creates a scenario (speech content, non-verbal movement) for controlling the operation of the presenter agent acting on behalf of a person's presentation. It is a technology that improves the appeal of presentations in an online conference environment by changing it.

Specifically, in order to effectively convey the content of the presentation to the audience, the behavior of the presenter agent is dynamically changed according to the current audience status of the audience participating in the online conference. In other words, the scenario for controlling the behavior of the presenter agent is dynamically changed according to the audience state (audience interest / interest, audience attitude).

In particular, the present invention creates an ideal audience state in which the audience has a high level of interest and can concentrate on and understand the main points of the presentation according to the current audience state and the progress of the scenario (progress of the presentation). Change the scenario so that it transitions. Further, in the present invention, the current audience state is determined using the audience face information (face orientation, facial expression, etc.) and the chat posted content, and based on the determination result, the progress of the scenario and the utterance timing are determined. Determine the ideal audience condition.

By providing such features, it is possible to realize a presentation that is easy for the audience of the online conference to understand without setting a complicated scenario in advance, and it is possible to improve the appealing power of the presentation in the online conference environment.

[Presenter system configuration]
FIG. 1 is a diagram showing an overall configuration of a presenter system according to the present embodiment. The presenter system acquires and predefines the user terminal 6 used by the audience of the online conference, the presentation control device 5 that controls the display of slides and the speech / operation of the presenter agent 100 based on the scenario, and the audience state. It includes a scenario control device 1 that changes a running scenario based on a rule. The user terminal 6, the presentation control device 5, and the scenario control device 1 are connected to a communication network capable of intercommunication.

[User terminal configuration]
The user terminal 6 is an information processing terminal used by an audience participating in an online conference. The user terminal 6 includes a Web camera, a display, a data communication function, a data input / output function, and the like. During the online conference, a slide for presentation, a presenter agent 100 for presenting the slide contents, and the like are displayed on the display of the user terminal 6. The number of user terminals 6 is at least one. The user terminal 6 may be a personal computer provided by a person concerned or a third party for an online conference, or may be a mobile terminal prepared by the audience.

[Configuration of presentation control device]
As illustrated in FIG. 1, the presentation control device 5 includes, for example, a scenario execution unit 50, a scenario storage unit 51, a scenario notification unit 52, and an instruction control unit 53.

The scenario execution unit 50 has a function of reading a scenario from the scenario storage unit 51 and executing the scenario. Further, the scenario execution unit 50 has a function of executing the changed scenario when the scenario is changed by the scenario control device 1.

The scenario storage unit 51 has a function of storing a scenario for controlling the operation of the presenter agent 100. In the scenario, as illustrated in FIG. 2, utterance contents for explaining each slide by voice and non-verbal movements such as facial expression, face orientation, and arm movement are set for each slide. ing.

The scenario notification unit 52 has a function of notifying the transition destination determination unit 40 of the scenario control device 1 of the scenario execution status (scenario progress, presentation progress).

The instruction control unit 53 has a function of controlling the operation (utterance content, non-verbal operation) of the presenter agent 100 displayed on the user terminal 6 based on the execution of the scenario by the scenario execution unit 50.

[Scenario control device configuration]
The scenario control device 1 is a device that changes the scenario being executed by the presentation control device 5 while estimating the current audience state of the audience watching the presentation in the online conference in real time from the audience face image and the posted chat content. Is.

As illustrated in FIG. 1, the scenario control device 1 includes, for example, a face image acquisition unit 10, a face image determination unit 11, a facial expression analysis information storage unit 12, a chat acquisition unit 20, and a chat analysis unit 21. , Audience state estimation unit 30, transition destination determination unit 40, state transition rule storage unit 41, correction content selection unit 42, scenario correction rule storage unit 43, scenario correction unit 44, and correction history storage unit 45. , Equipped with.

The face image acquisition unit 10 has a function of receiving an audience face image taken during an online conference from the user terminal 6.

The face image determination unit 11 reads facial expression analysis information from the facial expression analysis information storage unit 12, analyzes the facial expression and face orientation of the audience in the audience facial image using the facial expression analysis information, and makes a presentation. It has a function to analyze the interests and trends of interests of the entire audience.

The facial expression analysis information storage unit 12 has a function of storing facial expression analysis information in which an image feature amount for determining whether the audience is in a positive state or a negative state is described.

The chat acquisition unit 20 has a function of receiving chat data posted by an audience during an online conference from a user terminal 6.

The chat analysis unit 21 has a function of analyzing the contents of chat data and analyzing the tendency of the attitude of the entire audience toward the presentation.

The audience state estimation unit 30 is based on the tendency of the entire audience's interest / interest by the face image determination unit 11 and the tendency of the entire audience's attitude by the chat analysis unit 21, and the audience's overall interest / interest and attitude toward the presentation. It has a function to estimate and calculate the current audience status of.

The transition destination determination unit 40 reads the state transition rule from the state transition rule storage unit 41, and uses the state transition rule to notify the current audience state and the scenario execution status notified from the scenario notification unit 52 of the presentation control device 5. Based on the above, it has a function to determine the ideal transition destination audience state that can improve the interest / interest and attitude of the entire audience.

The state transition rule storage unit 41 has a function of storing a state transition rule for determining the audience state of the transition destination. As illustrated in FIG. 3, the state transition rule is set with a scenario execution status and a transition destination state indicating the audience state of the transition destination.

The modification content selection unit 42 reads a scenario modification rule from the scenario modification rule storage unit 43, and selects a modification scenario used to transition the current audience state to the transition destination audience state from the scenario modification rule. Be prepared. Further, the correction content selection unit 42 reads the scenario correction history information from the correction history storage unit 45, refers to the scenario correction history information, and if the same state transition occurs a plurality of times, the same state transition is used in the past. It has a function to select a correction scenario different from the selected correction scenario.

The scenario correction rule storage unit 43 has a function of storing a scenario correction rule in which a correction scenario for transitioning the current audience state to the transition destination audience state is set. As illustrated in FIG. 4, the scenario modification rule includes the ID of the scenario for modification, the current state indicating the current audience state, the transition destination state indicating the audience state of the transition destination, the scenario modification content, and the modification target. And are set. The modification target is set to one or both of the utterance content and the non-verbal behavior that make up the scenario.

The scenario correction unit 44 has a function of changing the scenario stored in the scenario storage unit 51 of the presentation control device 5 and the scenario read from the scenario storage unit 51 by the scenario execution unit 50 into a correction scenario.

The correction history storage unit 45 has a function of storing the number of times the correction scenario selected from the scenario correction rule is used as scenario correction history information. As illustrated in FIG. 5, the scenario correction history information stores the ID of the correction scenario and the number of times the correction scenario is used.

[Structural features and characteristic processing operations of the scenario control device]
The present invention estimates the audience state of an audience who participates in an online conference system and is watching a presentation in real time, and dynamically changes the scenario of the presenter agent based on the estimated current audience state. Achieve a highly appealing presentation.

Therefore, the structural features of the scenario control device 1 shown in FIG. 1 include the face image determination unit 11 and the chat analysis unit 21 that calculate the characteristics of the audience state, and the current audience state regarding the interest, interest, and attitude of the entire audience. Audience state estimation unit (estimation unit) 30 that estimates the current audience state, and a transition destination determination unit (first) that determines what state the current audience state should be transitioned to based on the current audience state and the scenario execution status. (Decision unit) 40, a modification content selection unit (second determination unit) 42 that determines a scenario for modification based on a predefined scenario modification rule and a history of modification of a scenario performed in the past, and It is provided with a scenario correction unit (change unit) 44 for changing a running scenario to a correction scenario.

Subsequently, the characteristic processing operation of the scenario control device 1 based on the above-mentioned structural characteristics will be described. FIG. 6 is a flow chart showing a characteristic processing operation of the scenario control device 1.

Step S1;
First, the face image determination unit 11 analyzes the positive / negative state of each audience from the facial expressions and face orientations of each audience face image acquired from the plurality of user terminals 6, respectively, and positive / negative of the entire audience. The state is determined (see FIG. 7).

For example, from the equation (1), the face image determination unit 11 sets Si = “1” if the facial expression of the i-th audience fst _i (1 ≦ i ≦ N) is in a positive state, and if it is in a negative state, it sets _Si = “1”. S _i = "0", and if it is difficult to calculate the positive / negative state, S _i = "0". Then, the face image determination unit 11 obtains _All from the equation (2), which is the sum of the positive / negative states of all the audiences. After that, the face image determination unit 11 determines from the equation (3) that if _All > (N / 2), the entire audience is in a positive state, and in other cases (the number of positive states and negative states is the same). (Including cases), determine the entire audience as negative. In the positive state, the audience as a whole is highly interested in the presentation. In the negative state, the audience as a whole has low interest in the presentation.

Next, the chat analysis unit 21 analyzes the concentration / divergence state of each audience from the posted content and the amount of sentences of each chat posted by each audience during the online conference, and determines the concentration / divergence state of the entire audience. Judgment (see FIG. 7).

For example, the chat analysis unit 21 acquires chats posted at regular intervals. Next, the chat analysis unit 21 calculates the amount of posted text Cl _j (1 ≦ j ≦ M) for each acquired chat j and the degree of similarity Cs _j between the chat j and the slide content. The posted sentence amount Cl _j is a value (0 ≦ Cl ≦ 1) obtained by normalizing the number of content words included in the chat j with the number of words in all chats acquired at regular intervals. The similarity Cs _j is a value (0 ≦ Cs ≦ 1) indicating the degree of similarity between the content word of the chat j and the content word of the question target slide. Next, the chat analysis unit 21 calculates the score S _j for each chat j from the product of the calculated posted sentence amount Cl _j and the similarity Cs _j , and totals all the scores at regular intervals from the equation (4). Obtain the SC _all . Finally, from the equation (5), the chat analysis unit 21 determines that the entire audience is in the concentrated state when SC _all > threshold Th, and in other cases, the entire audience is determined to be in the divergent state. The larger the amount of posted text Cl _j and the higher the degree of similarity Cs _j with the slide, the more concentrated listening is represented.

Step S2;
Next, the audience state estimation unit 30 estimates and calculates the current audience state regarding the entire audience's interest / interest and attitude toward the presentation based on the positive / negative state determination result and the concentration / divergence state determination result. do.

For example, the audience state estimation unit 30 classifies the interest and interest of the entire audience into high (positive state) and low (negative state), and the attitude of the entire audience is concentrated (concentrated state) and divergent (divergent state). ), The current audience state is estimated from the four audience states (see FIG. 8). If it is a positive / concentrated state, it is a state 1 in which the person is interested in the presentation and wants to hear the main points and details of the presentation. If it is a positive / divergent state, it is assumed that the person is interested in the presentation but does not know the main points and details of the presentation. In the negative / concentrated state, the interest of the presentation is low, but the state 3 in which the presentation is obligatory to be listened to. If it is a negative / divergent state, it is assumed that the person is not interested in the presentation and is in a state of being distracted other than the presentation 4.

In this embodiment, all of interest / interest and attitude are binary, but a multi-value other than the binary may be used. For example, by extending the audience state model as a multi-valued state determination other than binary, it is possible to deal with finer changes in the audience state.

Step S3;
Next, the transition destination determination unit 40 is an ideal transition destination that raises the interest and interest of the entire audience in the presentation and concentrates the attitude of the entire audience based on the current audience status and the scenario execution status. Determine the audience status.

For example, when the current audience state is the state 4, the transition destination determination unit 40 determines any of the states 1 to 3 as the transition destination audience state. In particular, when determining state 1 as the transition destination audience state, if the scenario execution status is the first half of the scenario (immediately after the start of the presentation), first the state to state 2 is to increase the interest and interest of the entire audience in the presentation. Priority is given to the transition, and then the state transition to state 1 is performed. In other words, after raising interest / interest, the attitude shifts to a focused attitude (see FIGS. 9 (a) and 9 (b)). On the other hand, if the scenario execution status is the latter half of the scenario (after a certain period of time has elapsed), the state 3 is given priority first in order to concentrate the attitude of the entire audience toward the presentation, and then the state transition to the state 1 is performed. In other words, after having a focused attitude, the interest / interest is increased (see FIGS. 9 (a) and 9 (c)).

Step S4;
Next, the modification content selection unit 42 determines a modification scenario for transitioning the current audience state to the transition destination audience state. For example, the modification content selection unit 42 acquires the modification scenario corresponding to the current audience state and the transition destination audience state from the scenario modification rule. At this time, the modification content selection unit 42 updates the number of times the modification scenario is used.

The correction scenario describes transition conditions such as "repeat the same explanation" and "eye contact operation" between the states of the state transition diagram shown in FIG. Transition conditions are set according to each state transition of "state transition source-state transition destination". Nonverbal movements are physical movements such as facial expressions, face orientation, and arm positions / directions. The utterance content is sound information / text information such as the utterance content uttered by the presenter agent, the utterance script, the intonation / interval of the voice, and the sound effect at the same time as the voice.

If the same state transition as the state transition that transitions the current audience state to the transition destination audience state has been performed in the past, even if the same correction scenario as the correction scenario used in the past is used. , The appeal of the presentation may not improve. In this case, the correction content selection unit 42 refers to the number of times the correction scenario is used, and selects a correction scenario different from the past correction scenario so as not to make a uniform scenario correction. For example, as illustrated in FIG. 11, when the state transition from the state 4 to the state 3 is performed for the second time, a “pointing operation” different from the first “attention guidance to the slide” is selected.

Step S5;
Finally, the scenario correction unit 44 changes the scenario being executed by the presentation control device 5 to the correction scenario.

[Specific example of processing operation of scenario control device]
FIG. 12 is a flow chart showing a specific example of the processing operation of the scenario control device 1.

Step S101;
First, the face image acquisition unit 10 receives each audience face image from the plurality of user terminals 6. Then, the face image determination unit 11 determines the positive / negative state of each audience based on the facial expression analysis information of the facial expression analysis information storage unit 12, and calculates the positive / negative state of the entire audience for the presentation.

Step S102;
Next, the chat acquisition unit 20 receives the chat data of the chat exchanged on the online tool from the user terminal 6. Then, the chat analysis unit 21 calculates the amount of posted text of the chat data and the degree of similarity between the posted content and the slide content, and calculates the concentration / divergence state of the entire audience with respect to the presentation.

Step S103;
Next, the audience state estimation unit 30 determines the current audience state regarding the audience's interest / interest and attitude toward the presentation based on the positive / negative state of the entire audience and the tendency of the concentration / divergence state of the entire audience. Estimate calculation. For example, one state is determined from the four states 1 to 4 illustrated in FIG.

Step S104;
Next, the transition destination determination unit 40 determines whether or not the state transition is necessary based on the estimation result of the current audience state. When the current audience state is state 1, the entire audience is watching the presentation with interest and interest, and it is determined that the state transition is unnecessary and the process is terminated. When the current audience state is the state 2 to the state 4, it is determined that the state transition is necessary, and the process proceeds to step S105.

Step S105;
Next, the transition destination determination unit 40 determines whether or not there is one candidate for the transition destination state to be transitioned from the current audience state. If the current audience state is state 4, the transition destination states are a plurality of states 1 to 3, and the process proceeds to step S106. If the current audience state is state 2 or state 3, the transition destination state is only state 1, and the process proceeds to step S107.

Step S106;
When there are a plurality of candidates for the transition destination state, the transition destination determination unit 40 acquires the scenario execution status from the presentation control device 5 and either is the first half of the presentation (from the start to the policy change position) or the second half of the presentation (policy). Based on whether it ends from the conversion position), the corresponding transition destination state is read from the state transition rule (FIG. 3) of the state transition rule storage unit 41, and this is used as a candidate for the transition destination state.

Step S107;
Next, the transition destination determination unit 40 determines the candidate for one transition destination state grasped in step S105 or the candidate for the transition destination state grasped in step S106 as the transition destination audience state.

Step S108;
Next, the correction content selection unit 42 selects all the correction scenarios that are candidates from the scenario correction rule (FIG. 4) of the scenario correction rule storage unit 43 based on the current audience state and the transition destination audience state. The final correction scenario is selected so as not to overlap with the past correction results recorded in the scenario correction history information (FIG. 5) of the read and correction history storage unit 45 as much as possible. A specific example of the correction scenario selection process will be described later.

Step S109;
Finally, the scenario correction unit 44 changes the scenario being executed by the presentation control device 5 with the correction scenario, and updates the scenario correction history information of the correction history storage unit 45. A specific example of the scenario change process will also be described later.

[Correction scenario selection process]
FIG. 13 is a flow chart showing a specific example of the selection process of the correction scenario.

Step S108-1;
First, the correction content selection unit 42 corrects all the correction scenarios corresponding to the combination of the current audience state and the transition destination audience state from the scenario correction rule (FIG. 4) of the scenario correction rule storage unit 43. Get as a candidate for.

Step S108-2;
Next, the modification content selection unit 42 determines whether or not the number of candidates for the modification scenario is one. If the number of candidates for the modification scenario is one, the process proceeds to step S108-6. If there are a plurality of candidates for the correction scenario, the process proceeds to step S108-3.

Step S108-3;
When there are a plurality of candidates for the correction scenario, the correction content selection unit 42 uses the scenario correction history information (FIG. 5) of the correction history storage unit 45 to indicate that each candidate's correction scenario has been used so far. Get the number of uses respectively.

Step S108-4;
After step S108-3, the modification content selection unit 42 discards the candidate whose number of uses is not the minimum among the candidates for the plurality of modification scenarios, and selects the candidate whose number of uses is the smallest.

Step S108-5;
After step S108-4, the correction content selection unit 42 determines whether or not the number of remaining correction scenario candidates is one. If the number of remaining correction scenario candidates is one, the process proceeds to step S108-6. If the number of remaining correction scenario candidates is a plurality, the process proceeds to step S108-7.

Step S108-6;
When it is determined in step S108-2 or step S108-5 that the number of candidates for the modification scenario is one, the modification content selection unit 42 selects the candidate for the modification scenario as the modification scenario to be changed.

Step S108-7;
When it is determined in step S108-5 that the number of candidates for the correction scenario is a plurality, the correction content selection unit 42 randomly changes one candidate from the remaining candidates for the correction scenario. Select as.

[Scenario change processing]
FIG. 14 is a flow chart showing a specific example of the scenario change process.

Step S109-1;
First, the scenario correction unit 44 acquires the correction target of the correction scenario selected in step S108 from the scenario correction rule (FIG. 4) of the scenario correction rule storage unit 43.

Step S109-2;
Next, the scenario correction unit 44 determines the type of the acquired correction target. If the type of the modification target is non-verbal operation, the process proceeds to step S109-3. If the type of the correction target is the utterance content, the process proceeds to step S109-4. If the types of the modification targets are all (both non-verbal actions and utterance contents), the process proceeds to step S109-5.

Step S109-3;
When the type of the modification target is non-verbal operation, the scenario modification unit 44 replaces the non-language operation set in the scenario before modification with the non-language operation of the scenario modification content, or the non-language operation thereof. Add the scenario correction contents to.

Step S109-4;
When the type of the correction target is the utterance content, the scenario correction unit 44 adds the utterance content of the scenario correction content to the utterance content set in the scenario before the correction.

Step S109-5;
When the types of modification targets are all, the scenario modification unit 44 changes the running scenario so as to repeat the explanation of the same slide. An example of changing the scenario is shown in FIG. FIG. 15A is an operation example for raising interest / interest. As shown in FIG. 15 (a-1), a sound effect is added to the utterance content. As shown in FIG. 15 (a-2), the utterance content is emphasized by repeating the same utterance content. This makes it possible to increase the interest and interest of the audience in the slide contents.

FIG. 15 (b) is an operation example for promoting an attitude. As shown in FIG. 15 (b-1), the important points of the slide are pointed with a hand or a finger. As shown in FIG. 15 (b-2), making eye contact with the audience gives the audience the impression of speaking and encourages them to listen more intensively. As shown in FIG. 15 (b-3), by paying attention to the important points of the slide and guiding the attention to the slide, the slide contents are carefully urged to listen to the explanation. This can enhance the audience's attitude towards the slide content.

Step S109-6;
Finally, the scenario correction unit 44 updates the scenario correction history information of the correction history storage unit 45 regarding the correction scenario that was the change target. For example, the scenario correction unit 44 increments the number of times the correction scenario is used.

The processing operation of the scenario control device 1 has been explained above. The scenario control device 1 periodically executes the above processing operation, estimates the current audience state, raises the interest and interest of the entire audience in the presentation, and concentrates the attitude of the entire audience. Change the scenario. Based on the scenario of the scenario storage unit 51, the presentation control device 5 dynamically changes the scenario in the above-mentioned modified scenario while the scenario execution unit 50 instructs the instruction control unit 53 to control the scenario based on the scenario. Further, the presentation control device 5 notifies the scenario control device 1 of the scenario execution status required for scenario correction one by one via the scenario notification unit 52.

[Effect of this embodiment]
In the present embodiment, the scenario control device 1 estimates the audience state during the remote presentation on the online conference system, and changes the scenario based on the estimation result. This makes it possible to provide a more appealing presentation in the presenter system in the remote presentation.

[Modified example of this embodiment]
In the present embodiment, the presenter system in which the presenter agent is operated in the virtual space in the computer has been described as a premise (see FIG. 16A). In addition, as long as the three elements of slide content, oral content (utterance content), and gesture motion (non-verbal motion) can be expressed and executed, the entity of the presenter agent may be either real space or virtual space. For example, as shown in FIG. 16 (b), a tangible presenter agent 100 may be used. In this case, the presenter agent 100 is present for each audience.

[others]
The present invention is not limited to the above embodiment. The present invention can be modified in a number of ways within the scope of the gist of the present invention.

The scenario control device 1 of the present embodiment is, for example, as shown in FIG. 17, a CPU (Central Processing Unit, processor) 901, a memory 902, a storage (Hard Disk Drive, Solid State Drive) 903, and a communication device 904. And, it can be realized by using a general-purpose computer system including an input device 905 and an output device 906. The memory 902 and the storage 903 are storage devices. In the computer system, each function of the scenario control device 1 is realized by the CPU 901 executing a predetermined program loaded on the memory 902.

The scenario control device 1 may be mounted on one computer. The scenario control device 1 may be implemented by a plurality of computers. The scenario control device 1 may be a virtual machine mounted on a computer. The program for the scenario control device 1 can be stored in a computer-readable recording medium such as an HDD, SSD, USB (Universal Serial Bus) memory, CD (Compact Disc), or DVD (Digital Versatile Disc). The scenario control program for the scenario control device 1 can also be distributed via the communication network.

1: Scenario control device 10: Face image acquisition unit 11: Face image determination unit 12: Face expression analysis information storage unit 20: Chat acquisition unit 21: Chat analysis unit 30: Audience state estimation unit 40: Transition destination determination unit 41: Status Transition rule storage unit 42: Correction content selection unit 43: Scenario correction rule storage unit 44: Scenario correction unit 45: Correction history storage unit 5: Presentation control device 50: Scenario execution unit 51: Scenario storage unit 52: Scenario notification unit 53: Instruction control unit 6: User terminal 7: Display panel 8: Camera 100: Presenter agent 901: CPU
902: Memory 903: Storage 904: Communication device 905: Input device 906: Output device

Claims

It is a scenario control method that changes the scenario in the presenter system that controls the operation of the presenter agent based on the scenario and presents the slide contents.
The first estimation of the current audience status regarding the audience's overall interest and attitude toward the presentation based on the facial information of the audience acquired during the presentation at the online conference and the content posted by the audience during the presentation. Steps and
Based on the current audience status and the progress of the scenario, a second step of determining an ideal audience status that raises the audience's interest in the presentation and concentrates the attitude of the audience as a whole.
A third step of determining a correction scenario for transitioning the current audience state to the ideal audience state, and
The fourth step of changing the scenario to the correction scenario, and
Scenario control method to do.
In the first step,
A claim that estimates the current audience state from four audience states in which the interest of the entire audience for the presentation is divided into high and low, and the attitude of the entire audience toward the presentation is further divided into intensive and divergent. The scenario control method according to 1.
In the second step,
If the progress of the scenario is the first half of the scenario, the state that raises the interest of the whole audience to the presentation is preferentially determined, and if the progress of the scenario is the second half of the scenario, the attitude of the whole audience to the presentation. The scenario control method according to claim 1 or 2, wherein the state of concentrating is preferentially determined.
In the third step,
A claim for determining a modification scenario different from the modification scenario determined in the past when the same state transition as the state transition for transitioning the current audience state to the ideal audience state has been performed in the past. The scenario control method according to any one of 1 to 3.
The correction scenario is
The scenario control method according to any one of claims 1 to 4, which is a scenario for modifying one or both of the spoken voice and the non-verbal operation of the presenter agent.
A scenario control device that changes the scenario in a presenter system that controls the operation of the presenter agent based on the scenario to present the slide contents.
An estimation unit that estimates the current audience status regarding the audience's overall interest and attitude toward the presentation based on the facial information of the audience acquired during the presentation at the online conference and the content posted by the audience during the presentation. When,
Based on the current audience status and the progress of the scenario, the first decision-making part that determines the ideal audience status that raises the audience's interest in the presentation and concentrates the attitude of the entire audience. ,
A second determinant that determines a correction scenario that transitions the current audience state to the ideal audience state.
A change part that changes the scenario to the correction scenario,
A scenario control device.
A scenario control program that causes a computer to execute the scenario control method according to any one of claims 1 to 5.