JP2020086774A - Apparatus, method and program for controlling scenario - Google Patents

Abstract

To provide an apparatus for controlling a scenario adapted to assist in effective content presentation for a variety of audiences.SOLUTION: An apparatus for controlling a scenario according to a first embodiment of the present invention comprises an acquiring section that acquires information representative of a situation of an audience viewing a content, an extracting section that extracts a feature amount of the audience from an image, an estimating section that estimates a viewing state of the audience as any one of a plurality of states including a first state based on the feature amount of the audience and takes the estimated state as a current state, a determining section that if the current state is not the first state, determines any one different from the current state among the plurality of states as a destination-of-transition state, and a selecting section that selects any one out of at least one available correction contents for a presentation scenario of the content, associated with a combination of the current state and the destination-of-transition state.SELECTED DRAWING: Figure 1

Description

The present invention relates to control of a scenario that describes a presentation mode of content, for example.

In recent years, a content presentation technique that combines digital signage and a communication robot is known. For example, it is possible to contribute to labor saving of various information services, for example, by causing the robot to speak an explanation or to make a gesture in synchronization with video contents such as a slide displayed on the display.

In such a content presentation technique, a presentation mode of content, generally, a slide displayed on a display, dialogue (contents of speech) that a robot speaks while displaying the slide, and non-verbal actions such as gestures that the robot takes. , A presentation scenario (hereinafter, simply referred to as a scenario) describing "," is created in advance. That is, during the slide display, the robot speaks the dialogue described in the scenario and takes the non-verbal movement described in the scenario. Scenarios are ideally created to convey the key points and details of the slides to the audience, but the attributes of the audience, such as age, gender, knowledge, preferences, etc., are diverse and effective for all audiences. It is not easy to create a scenario that presents various contents.

Non-Patent Document 1 proposes a digital signage that displays interactive advertising content that responds to the position and distance of a person. Further, Non-Patent Document 2 proposes to detect a driver's consciousness state while driving and to present an easy-to-understand alarm to the driver based on the information.

Chen et al., "Digital Signage Interactively Responding to Human Situation", 2014, 76th National Convention of Information Processing Society of Japan Hatsuo Yamazaki et al., "Development of driver status monitor and examination of warning presentation method of driving support system", IEEE Trans. IA, Vol. 125, No. 11, 2005

The present invention aims to support effective content presentation to diverse audiences.

A scenario control device according to a first aspect of the present invention includes an acquisition unit that acquires information that represents the state of an audience who views content, and an extraction that extracts a feature amount of the audience from the information that represents the state of the audience who views the content. And an estimation unit that estimates the viewing state of the audience as one of a plurality of states including the first state based on the feature amount of the audience, and the estimated state is the current state, and the current state is Presentation of content associated with a combination of a current state and a transition destination state, and a determination unit that determines any one of the plurality of states different from the current state as the transition destination state if the state is not the first state A selection unit for selecting any one of at least one available correction content for the scenario.

That is, the scenario control device estimates the viewing state of the audience based on the image of the audience, and selects the correction content for the scenario in which the presentation mode of the content is described in order to transition the estimated viewing state. Therefore, according to this scenario control device, even when the content is presented to the audience who is not suitable for the scenario prepared in advance, the correction content can be selected to be suitable for the audience.

In the scenario control device according to the first aspect, the audience feature amount includes a first feature amount indicating the interest/degree of interest of the audience content, and a second feature amount indicating the concentration degree of the audience on the content. In addition to the first state, the plurality of states include a second state, a third state, and a fourth state, and the estimation unit determines that the first feature amount is equal to or larger than the first threshold value. , The second feature amount is equal to or more than the second threshold value, the audience viewing state is estimated to be the first state, and the estimating unit determines that the first feature amount is equal to or more than the first threshold value. When the second feature amount is less than the second threshold value, the viewing state of the audience is estimated to be the second state, and the estimation unit determines that the first feature amount is less than the first threshold value and the second feature amount is less than the second threshold value. When the feature amount is equal to or larger than the second threshold value, the viewing state of the audience is estimated to be the third state, and the estimation unit determines that the first feature amount is less than the first threshold value and the second feature amount. If is less than the second threshold, the viewing state of the audience may be estimated as the fourth state.

This scenario control device (hereinafter, referred to as a scenario control device according to the second aspect of the present invention) classifies the plurality of states into two axes of interest/interest level and concentration level, and thus improves the viewing state of the audience. Therefore, it is possible to narrow down the state elements that need to be improved in order to select appropriate correction content.

In the scenario control device according to the second aspect, when the present state is the fourth state and the progress status of the presentation scenario is less than the third threshold, the determining unit changes the second state to the transition destination. If the present state is the fourth state and the progress of the presentation scenario is equal to or greater than the third threshold, the determining unit determines the third state as the transition destination state. Good.

According to this scenario control device, until the progress of the scenario reaches the third threshold, priority is given to the state transition for drawing the interest of the audience, and after the progress of the scenario reaches the third threshold. Priority is given to state transitions in order to bring out the concentration of the audience.

In the scenario control device according to the second aspect, the determining unit sets the second state to the transition destination state when the current state is the fourth state and the total length of the presentation scenario is less than the fourth threshold value. You may decide. According to this scenario control device, when the total length of the scenario is short, priority is given to the state transition for eliciting the interest of the audience regardless of the progress of the scenario.

In the scenario control device according to the first or second aspect, the selection unit has the smallest cumulative selection count among at least one available correction content associated with the combination of the current state and the transition destination state. You may choose one.

According to this scenario control device, since various correction contents are uniformly selected, it is possible to find a scenario in which the audience has a good reaction and make the audience's viewing state transition to a state defined as good. Further, as a result, the scenario is modified in a non-uniform manner, so that it is possible to suppress a decrease in the effectiveness of the scenario modification due to familiarization with the presentation mode of the content.

The scenario control device according to the first or second aspect may further include a correction unit that corrects the content presentation scenario based on the selected correction content. According to this scenario control device, even when the content is presented to an audience who is not suitable for the prepared scenario, the scenario can be dynamically modified to be suitable for the audience.

A scenario control method according to a third aspect of the present invention is a computer-implemented scenario control method, which comprises acquiring information indicating a state of an audience who views content, and a state of an audience who views content. Extracting the feature amount of the audience from the represented information, and estimating the viewing state of the audience as one of the plurality of states including the first state based on the feature amount of the audience, and estimating the estimated state as the current state. A state, and if the current state is not the first state, determine any one of a plurality of states different from the current state as the transition destination state, and combine the current state and the transition destination state. Selecting any one of the associated at least one available modification to the content presentation scenario.

That is, the scenario control method estimates the viewing state of the audience based on the image of the audience, and selects the correction content for the scenario in which the presentation mode of the content is described in order to transition the estimated viewing state. Therefore, according to this scenario control method, even when the contents are presented to an audience who is not suitable for the scenario prepared in advance, the correction content can be selected to be suitable for the audience.

A scenario control program according to a fourth aspect of the present invention includes computer-readable instructions for causing a computer to function as the scenario control device according to the first or second aspect. According to this scenario control program, the scenario control device according to the first or second aspect can be realized by software.

According to the present invention, it is possible to support effective content presentation to various audiences.

The block diagram which illustrates the contents presentation system containing the scenario control device concerning an embodiment. Explanatory drawing of the content presentation aspect by the content presentation system of FIG. Explanatory drawing of the scenario which describes the content presentation aspect by the content presentation system of FIG. Explanatory drawing of the audience feature extraction process performed by the audience feature extraction part of FIG. Explanatory drawing of the audience state estimated by the audience state estimation part of FIG. The figure which shows an example of the candidate of the transition destination state determined when the present state is presumed to be the state 4 of FIG. The figure which shows another example of the candidate of the transition destination state determined when the present state is estimated to be the state 4 of FIG. The figure which illustrates the state transition rule table preserve|saved at the state transition rule memory|storage part of FIG. The figure which illustrates the correction content for implement|achieving the state transition which improves each state other than the state 1 of FIG. Explanatory drawing of "utilization of a sound effect" which is an example of the correction content mentioned in FIG. Explanatory drawing of "repeat the same description" which is an example of the correction content mentioned in FIG. Explanatory drawing of "pointing operation" which is an example of the correction content quoted in FIG. Explanatory drawing of "eye contact operation|movement with respect to an audience by eye-gaze control" which is an example of the correction content mentioned in FIG. Explanatory drawing of "attention guidance to a slide by a line-of-sight control" which is an example of the correction content mentioned in FIG. The figure which illustrates the correction content rule table preserve|saved at the correction content rule memory|storage part of FIG. The figure which illustrates the correction history table preserve|saved at the correction history storage part of FIG. 3 is a flowchart illustrating the operation of the scenario control device in FIG. 1. The flowchart which illustrates the detail of step S310 of FIG. The flowchart which illustrates the detail of step S320 of FIG. The figure which shows the example of a changed completely type of a presenter. The figure which shows another modification of a presenter.

Hereinafter, a description of embodiments will be given with reference to the drawings. In the following, the same or similar reference numerals will be given to the same or similar elements as the elements already described, and redundant description will be basically omitted.

(Embodiment)
The scenario control device according to the embodiment can be incorporated in, for example, a content presentation system in which a digital signage and a communication robot are combined. Note that, as will be described later, such a content presentation system is only an example. For example, the digital signage does not necessarily have to be realized by a physical display, and is provided in a VR (Virtual Reality)/AR (Augmented Reality)/MR (Mixed Reality) space (hereinafter simply referred to as a virtual space). It may be realized by a virtual display. Further, the communication robot may be replaced with a pointing device or a virtual agent existing in a virtual space.

The content presentation system may include the scenario control device 100 according to the embodiment, the camera 10, the display 20, the robot 30, and the presentation control device 200, as illustrated in FIG. 1.

The display 20 and the robot 30 are in charge of presenting content. Specifically, as illustrated in FIG. 2, the display 20 displays video content, for example, a slide, and the robot 30 is displayed on the display 20 by utilizing nonverbal actions such as utterances and gestures. Explain the video content to the audience.

The camera 10 is installed so as to capture an image of the audience at the time of presenting the content, particularly the vicinity of the face area of the audience. In the following description, it is assumed that hardware different from the robot 30 such as a web camera is prepared as the camera 10, but a camera mounted on the display 20 or the robot 30 may be used. In addition to the visible light camera that acquires general image data, a depth camera combined with an infrared sensor is used to measure the features (coordinates of feature points such as eyes and nose) and skeleton data of the face area of the audience. It is also possible to obtain a feature amount (coordinates of each joint such as a shoulder and a neck) and acquire the face direction and the point of interest of the audience. Alternatively, the line-of-sight direction of the audience may be acquired using a line-of-sight measurement camera or the like. In short, not only the image data but also any information indicating the state of the audience at the time of presenting the content can be used, but in the following description, it is assumed that the image data is used.

The presentation control device 200 controls the display content of the display 20, the utterance content of the robot 30, and the non-verbal operation according to the scenario. Note that FIG. 1 is merely an example, and some or all of these control targets may be controlled by separate control devices.

As illustrated in FIG. 3, the scenario is an element of the video content (a slide in the presentation, or an animation (part) set to the slide, but it may be a scene in the case of a moving image, for example) and the element. Describe the utterance content and non-verbal actions that are presented to explain. That is, the presentation control device 200 displays the animation 1-1 of the slide 1, for example, on the display 20 according to the scenario of FIG. , And perform a non-verbal action 1-1, for example, a pointing action that indicates part or the whole of the display 20, and then display the next animation 1-2 of the slide 1 on the display 20. In the meantime, the robot can be made to utter the utterance content 1-2 and take the non-verbal action 1-2. Further, the presentation control device 200 can display the slide 2 on the display 20 and cause the robot to speak the utterance content 2 and perform the non-verbal motion 2 according to the scenario of FIG. 3, for example.

The scenario control device 100 is executed by the presentation control device 200 to estimate the viewing state of the audience based on the image of the audience captured by the camera 10, and to shift the estimated viewing state to a state defined as better. Dynamically modify the scenario. Thereby, as will be described later, the presentation mode of the content such as the utterance content and the non-verbal motion of the robot 30 adaptively changes depending on the reaction of the audience. Therefore, even when presenting content to an audience who is not suitable for the prepared scenario, the scenario control device 100 corrects the scenario to suit the audience, and the main points and details of the content are provided to the audience. It is possible to communicate effectively.

Next, a hardware configuration example of the scenario control device 100 will be described. Note that the presentation control device 200 can adopt the same hardware configuration as the scenario control device 100.

The scenario control device 100 can be, for example, a computer. In this case, the scenario control device 100 performs information processing such as dynamic correction of a scenario (for example, CPU (Central Processing Unit), FPGA (Field Programmable Gate Array), ASIC (Application Specific Integrated)), and the like. including. Further, the scenario control device 100 includes a memory that temporarily stores a program executed by the processor to realize such processing, data used by the processor, and the like.

The scenario control device 100 can further use a communication I/F (interface) for connecting to an external device such as the presentation control device 200 via, for example, a network. The communication I/F may be built in the scenario control device 100 or external to the scenario control device 100.

The scenario control device 100 can further use an auxiliary storage device for storing data. The auxiliary storage device may be built in the scenario control device 100 or external to the scenario control device 100. The auxiliary storage device is preferably a non-volatile storage medium such as an HDD (Hard Disk Drive), an SSD (Solid State Drive), or a flash memory. Alternatively, the auxiliary storage device may be a file server connected to the scenario control device 100 via a network.

The scenario control device 100 can further use an input I/F for receiving external input data such as image data. The input I/F may be built in the scenario control device 100 or external to the scenario control device 100.

The scenario control device 100 and the presentation control device 200 may be separate devices as illustrated in FIG. 1, or may be the same device. When the scenario control device 100 and the presentation control device 200 are separate devices, they do not necessarily have to be arranged close to each other. For example, the scenario control device 100 may be a cloud server connected to the presentation control device 200 as one or a plurality of clients located all over the country via a network such as the Internet. That is, the content presentation system may be realized by a cloud-compatible interaction control technology such as R-env: Renmai (registered trademark).

Next, a functional configuration example of the scenario control device 100 will be described. As illustrated in FIG. 1, the scenario control device 100 includes an image acquisition unit 101, an audience feature extraction unit 102, an audience state estimation unit 103, a transition destination determination unit 104, a state transition rule storage unit 105, A correction content selection unit 106, a correction content rule storage unit 107, a correction history storage unit 108, and a scenario correction unit 109 are included.

The image acquisition unit 101 acquires an image (data) obtained by capturing an image of an audience viewing content with the camera 10. The image acquisition unit 101 sends the acquired image to the audience feature extraction unit 102. The image may be a moving image or a still image. However, as will be described later, since the audience feature extraction unit 102 needs a time-series image to extract a feature amount, a plurality of still images are required in the latter example. The image acquisition unit 101 can correspond to, for example, the communication I/F and/or the input I/F described above.

The audience feature extraction unit 102 receives the image from the image acquisition unit 101 and extracts the feature amount of the audience from the image. Specifically, the audience feature extraction unit 102 extracts a feature amount indicating the interest of the audience with respect to the content, the degree of concentration, and the like from the image based on the orientation, size, and movement of the face area of the audience. The audience feature extraction unit 102 sends the extracted feature amount to the audience state estimation unit 103. The audience feature extraction unit 102 can correspond to, for example, the above-described processor.

For example, as described below, the audience feature extraction unit 102 includes a second feature amount that indicates a first feature amount that indicates the interest of the audience in the content and a second feature amount that indicates the concentration degree of the audience in the content. The dimensional feature amount may be extracted. The audience feature extraction unit 102 may extract one-dimensional or three-dimensional or more feature amounts.

The first feature amount may be calculated based on, for example, the size of the face area of the audience whose face area faces forward, for example, the face of the audience facing the display 20 and/or the robot 30. An audience with a positive face area may have been interested in the content at least when the image was captured. The size of the face area depends on the size of the original face of the audience, but is large in the distance from the audience to the camera 10 (which is supposed to be located near the display 20 and the robot 30). Dependent. This distance becomes smaller as the audience gets closer to the content so that the content is easier to view. Further, even if the distance from the audience to the camera 10 is the same, if the number of audiences increases, the total size of the face area increases. In this way, the size of the face area of the audience whose face area is positive can indicate the interest in the content of the audience.

Specifically, the audience feature extraction unit 102 detects the face area included in the image. Note that the audience feature extraction unit 102 may detect as many face areas as possible, but an upper limit may be set for the number of detections, or face areas smaller than a predetermined area may be ignored. Then, the audience feature extraction unit 102 calculates the orientation of each of the detected face areas. Here, the orientation of the i-th face area is θ _i . i is an arbitrary integer.

Next, the audience feature extraction unit 102 corrects the calculated orientation of the face area as necessary. For example, as shown in FIG. 4, the point where the audience desires to turn his/her face (hereinafter referred to as a reference point) and the position of the camera 10 may not match. In such a case, the audience feature extraction unit 102 can obtain the corrected orientation by subtracting the angle difference θ _d between the camera position and the reference point from the orientation of each face area. If the corrected orientation of the i-th face area is θ′ _i , then θ′ _i =θ _i −θ _d . When the camera position and the reference point are the same, θ _d =0.

The audience feature extraction unit 102 determines whether or not the face area is forward depending on whether or not the corrected orientation of each face area is within a predetermined range. For example, the audience feature extraction unit 102 determines that the i-th face area is facing forward when φ ₁ ≦θ′ _i ≦φ ₂ is satisfied. Here, φ ₁ and φ ₂ may be threshold values determined based on the boundary of the orientation of the face where the display 20 and the robot 30 can be visually recognized from the assumed position of the audience, and satisfy φ ₁ <φ ₂ . In addition, not only the face orientation is acquired on the basis of images, but also the face orientation is obtained from the skeletal data acquired using the depth camera from the coordinate positions of both shoulder joints and the orientation of the spinal joint, and the gaze measurement camera is used to determine the forward facing Alternatively, it may be obtained from the direction of the line of sight of the audience that has been acquired.

The audience feature extraction unit 102 calculates the area of each face area that is determined to be positive, and obtains the maximum value and the sum of the calculated areas. Further, the audience feature extraction unit 102 obtains the area ratio by dividing this total sum by this maximum value. Then, the audience feature extraction unit 102 can repeat the acquisition of the image and the calculation of the area ratio over a certain period of time, and can calculate the sum of the area ratios over the certain period of time as the first feature amount. In addition, as described above, when determining the coordinate position of both shoulder joints, the direction of the spinal joint, or the forward direction from the line of sight of the audience, the number of people may be calculated as the first feature amount instead of the area ratio. .. Here, the number-of-people ratio is obtained by dividing the number of audiences determined to be positive by the total number of audiences.

The second feature amount may be calculated based on the movement of the face area of the audience. For example, an audience who is gazing at the video content displayed on the display 20 (there is little blur in the face area) or a nod audience is expected to concentrate on the content. On the other hand, it is expected that an audience who shakes their heads to the left or right, or an audience who moves their faces too much is more distracting than the content or simply distracted.

Specifically, the audience feature extraction unit 102 tracks the face area of each audience from the time-series images over a certain period of time. Then, the audience feature extraction unit 102 recognizes the movement of the face of the audience based on the movement of the face area of each audience. The audience feature extraction unit 102 may recognize a facial action using, for example, a gesture recognizer. This gesture recognizer may include, for example, a learning model created by supervised learning using a large amount of face area motion data for learning and its motion labels. It should be noted that not only the motion of the face area is acquired based on the image, but the motion may be recognized from each coordinate of the skeleton data acquired using the depth camera.

The degree of concentration is assigned in advance to each motion label, which is the recognition result of the motion of the face. For example, a high degree of concentration may be assigned to “gaze”, “nodding”, etc., and a low degree of concentration may be assigned to “left/right swing”, “large movement”, and the like. Here, the degree of concentration may be multivalued, or may be a binary value that means “concentration” and “divergence”.

The audience feature extraction unit 102 creates a histogram of the recognition results (action labels) over the entire audience, and obtains the recognition result that is the most frequent value. Then, the audience feature extraction unit 102 can extract the degree of concentration assigned to this mode value as the second feature amount. When there are a plurality of mode values, the audience feature extraction unit 102 can extract the minimum value of the degree of concentration assigned to these mode values or another statistic as the second feature amount.

Alternatively, the audience feature extraction unit 102 may extract the degree of concentration assigned to the recognition result for each audience as the degree of concentration of the audience, and use a statistic such as the average of the degree of concentration as the second feature amount.

The audience state estimation unit 103 receives the audience feature amount from the audience feature extraction unit 102, and estimates the audience viewing state as one of the plurality of states based on the feature amount. The estimated viewing state is referred to as a current state in the description below. The audience state estimation unit 103 sends a value indicating the current state, for example, a state ID, to the transition destination determination unit 104. The audience state estimation unit 103 can correspond to, for example, the above-described processor.

Although the plurality of states can be defined in various ways, four states shown in FIG. 5, for example, can be defined based on the above-described two-dimensional feature amount. State 1 refers to a state in which the audience has a high interest/interest in the content and concentration, state 2 refers to a state in which the audience has a high interest/interest in the content but a low concentration, and state 3 refers to the content in the audience. A state of low interest/high interest but high concentration, state 4 indicates a state of low interest/interest and focus on the content of the audience. In the example of FIG. 5, the first feature amount and the second feature amount are each divided into two ranges for simplification, but one or both may be divided into three or more ranges. ..

According to the example of FIG. 5, the audience state estimation unit 103 estimates the current state as state 1 when the first feature amount and the second feature amount are equal to or more than the first threshold value and the second threshold value, respectively. However, when the first feature amount is equal to or more than the first threshold value and the second feature amount is less than the second threshold value, the current state is estimated to be the state 2, and the first feature amount is the first When it is less than the threshold value and the second feature amount is equal to or more than the second threshold value, the current state is estimated to be the state 3, and the first feature amount and the second feature amount are respectively the first threshold value and the second threshold value. If it is less than the threshold of, the current state is estimated to be state 4.

In the example of FIG. 5, the state 1 is defined as the best state, and the state 4 is defined as the poorest state. That is, if the current state is estimated to be state 1, it is expected that the content will be effectively presented to the audience by the current scenario, and thus the scenario need not be modified. On the other hand, if the current state is estimated to be other than state 1, there is room for improvement in the interest and/or concentration of the audience's content, and therefore the scenario control device 100 decides to correct the scenario aiming at state 1. Become.

The transition destination determination unit 104 receives a value indicating the viewing state (current state) of the audience from the audience state estimation unit 103. The transition destination determination unit 104 first determines whether or not the current state is a state that does not require state transition, for example, state 1 in FIG. When the current state is a state that needs to undergo state transition, the transition destination determination unit 104 determines one that is defined as better than the current state as the transition destination state. The transition destination determination unit 104 sends a value indicating the current state and the transition destination state, for example, a state ID, to the correction content selection unit 106. The transition destination determination unit 104 can correspond to, for example, the above-described processor.

According to the example of FIG. 5, the state 1 is the state that does not need to make the state transition, and the states 2 to 4 are the states that need to make the state transition. When the current state is the state 2 or the state 3, the transition destination determination unit 104 may determine the state 1 as the transition destination state. On the other hand, when the current state is the state 4, the state transition illustrated in FIG. 6 is state 4→state 2, then the state transition 2→state 1, and the state transition illustrated in FIG. 7 is state 4→state 3 and then the state transition. A state transition of 3→state 1 can be selected. Which state transition has priority may be fixed or random, but may be strategically determined as described below, for example.

For example, while the presentation, which is an example of video content, has various configurations, some types of presentations include topics that have impact and familiar topics in the first half of the presentation to attract the audience. Slides are arranged, and in the latter half of the conclusion section, slides containing messages and the like to be finally conveyed to the audience by the presentation are arranged. Therefore, for example, in the first half, the scenario is corrected by giving priority to attracting interest in the presentation, and in the second half, the scenario is corrected by giving priority to focusing on the presentation. Speaking and/or non-verbal actions in harmony with the composition can be used to skillfully elicit the interest, attention and concentration of the audience.

Therefore, for example, when there are a plurality of candidates for the transition destination state (the current state is state 4 according to the example of FIG. 5), the transition destination determination unit 104 can refer to the progress status of the scenario. Here, the progress status can be derived, for example, by dividing the execution position of the scenario by the total length of the scenario. The execution position of the scenario represents the position of the element during playback of the video content. For example, the slide number being played, identification information such as the element number of the animation being set for the slide, etc., the playback of the video content. The identification information such as the scene number, the current reproduction time of the video content, the elapsed time from the start of presentation of the content, and the like may be used, and can be notified by (the execution position notification unit 202 of) the presentation control device 200. Further, the total length of the scenario may be, for example, the total number of slides, the total playback time of the video content, the time from the start of presentation of the content to the end thereof. The transition destination determination unit 104 determines the state 2 as the transition destination state when the progress status of the scenario is less than the threshold value (hereinafter, also referred to as policy transition threshold value for convenience), and the progress status of the scenario is equal to or greater than the policy transition threshold value. In some cases, the state 3 may be determined as the transition destination state. The policy change threshold is, for example, 1/2 and can be described as at least a part of the state transition rule stored in the state transition rule storage unit 105.

If the total length of the scenario is within the range defined as short, which state transition is to be prioritized may be fixed. For example, the policy change threshold may be set to 1 when priority is given to state transitions that bring out the interest of the audience, and the policy change threshold may be set to 0 when priority is given to state transitions that bring out the concentration of the audience. If the total length of a scenario is shorter than a certain scenario length threshold and it is determined that the total length of the scenario is short, if the priority is given to state transitions that elicit the interest of the audience regardless of the progress of the scenario, the policy is The conversion threshold can be derived as follows.

In the above formula, Th _P represents the policy change threshold, L represents the total length of the scenario, and TH _L represents the scenario length threshold.

The state transition rule storage unit 105 stores the state transition rule in the state transition rule table format shown in FIG. 8, for example. The state transition rule stored in the state transition rule storage unit 105 is read by the transition destination determination unit 104 as needed. The state transition rule storage unit 105 can correspond to, for example, the above-mentioned memory and/or auxiliary storage device.

The correction content selection unit 106 receives values indicating the current state and the transition destination state from the transition destination determination unit 104, and is executed from the presentation control device 200 (the scenario notification unit 203 thereof) and the scenario currently being executed. You will be notified of the scenario. The modification content selection unit 106 refers to the modification content rule stored in the modification content rule storage unit 107 and reads out at least one scenario modification content associated with the combination of the current state and the transition destination state. Here, the modification content rule describes the scenario modification content and the modification target that can be used for the combination of the current state and the transition destination state. The read scenario correction content corresponds to a candidate scenario correction content to be applied. The correction content selection unit 106 selects one from the candidates, a value indicating the selected scenario correction content, a value indicating a scenario position to be corrected, and (a part of) the scenario specified by the scenario position. To the scenario correction unit 109. The modification content selection unit 106 can correspond to, for example, the above-described processor.

Here, for each combination of the current state and the transition destination state, that is, for each target state transition, an effective approach for realizing the state transition, that is, a scenario correction content may be different. As illustrated in FIG. 9, in the transition from the state 4 to the state 2, the scenario in which the robot 30 performs “use of sound/LED”, “repeat the same description”, “beckoning action”, etc. Although it may be effective to correct the above, in the transition from the state 4 to the state 3, the robot 30 is caused to perform “attention guidance to slide by eye control”, “pointing action”, etc. May be effective. Further, since the attributes of the audience are various, not all such scenario correction contents are always effective for realizing the targeted state transition. Therefore, the correction content selection unit 106 selects a scenario correction content by trial and error to find a scenario in which the audience has a good reaction, and shifts the audience's viewing state to a state defined as good.

“Use of sound effect/LED (Light-Emitting Diode)” in FIG. 9 means, for example, as shown in FIG. 10, outputting a sound effect/lighting an LED when the robot 30 speaks at a target scenario position. Can be As a result, the effect of stimulating the hearing/visual sense of the audience and attracting the attention can be expected. The LED may be mounted on the robot 30, for example.

Here, the target scenario position means a scenario position where the scenario is corrected, and is, for example, an element that is presented during or after the presentation (typically the next) among the elements of the video content. possible. Specifically, the target scenario position may be a slide being played or later, an animation being played or being set for a slide being played, a scene being played or being played later, or the like. .. Further, the target scenario position can be determined depending on the scenario correction content. For example, if the non-verbal operation or the utterance content is changed for the element of the video content being reproduced, the behavior of the robot 30 may change in the middle of the explanation, which may make the audience feel uncomfortable. When the modification content to be changed is selected, the element to be reproduced next to the element of the video content being reproduced can be set as the target scenario position. On the other hand, for example, when “repeating the same description” described later is the correction content, the target scenario position may be an element of the video content being reproduced, or an element after that. ..

“Repeating the same description” in FIG. 9 means, for example, as shown in FIG. 11, repeating the contents of the scenario at the target scenario position, for example, duplicating and inserting immediately before or after the scenario position. obtain. It should be noted that the scenario of the repeated part may be further added with utterance contents such as "Don't you understand a little?" and "I will say it again". As a result, the effect of emphasizing the content at the target scenario position and attracting the attention of the audience can be expected.

The “beckoning action” in FIG. 9 may be causing the robot 30 to beckon. As a result, in addition to the effect of attracting the attention of the audience who is already in front of the display 20, an effect of attracting a new audience from the surroundings can be expected.

The “illustrated motion” in FIG. 9 means, for example, moving the hand and/or arm of the robot 30 to enclose a portion of interest in the video content or attach a left hand and a right hand to the portion to be compared. Can be. As a result, it is possible to expect an effect that the points to be emphasized in the video content are clearly communicated to the audience and the audience is made aware of the important points.

The “pointing operation” in FIG. 9 may be moving the hand and/or arm of the robot 30 to point a finger or hand to a notable portion in the video content, as shown in FIG. 12, for example. .. As a result, it is possible to expect an effect that the points to be emphasized in the video content are clearly communicated to the audience and the audience is made aware of the important points.

The “emphasis on important points in para-language” in FIG. 9 means, for example, not the utterance content itself of the robot 30 at the target scenario position, but the volume at the time of utterance is increased, the pitch is changed, intonation is added, and the pause is performed. Can be longer. As a result, the utterance content of the robot 30 at the target scenario position is emphasized, and an effect of encouraging the listener to listen to the utterance content of the robot 30 can be expected.

The “attention guidance to the slide by the line-of-sight control” in FIG. 9 may be moving the head of the robot 30 to direct the eyes on the display 20, as shown in FIG. 14, for example. This can be expected to have the effect of allowing the audience to gaze at the display 20 in synchronism.

The “eye contact operation to the audience by the line-of-sight control” in FIG. 9 may be moving the head of the robot 30 to direct the audience to look at it, as shown in FIG. 13, for example. As a result, the robot 30 speaks so as to speak to the audience, and an effect of encouraging the audience to listen to the speech content of the robot 30 can be expected.

It should be noted that the scenario correction content of FIG. 9 is merely an example, and the scenario correction content illustrated as being usable for realizing a certain state transition may be used for realizing a different state transition. ..

The correction content selection unit 106 may refer to the correction history stored in the correction history storage unit 108, for example, in order to select an effective scenario correction content. In this revision history, the cumulative number of times of scenario revision contents included in the revision rule is described for each revision rule. It should be noted that the cumulative selection count can be reset, for example, each time the presented content changes, or each time the presentation of the same content ends. In addition to the cumulative number of times, additional information at the time of modification may be recorded in the modification history. For example, the time at which the correction is performed, the scenario contents before the correction, the statistical information of the first and second feature amounts before and after the correction (for example, the average value of the rate of change of each feature amount before and after the correction) ), etc. may be recorded. The modification content selection unit 106 may select, for example, the scenario modification content with the smallest cumulative selection count. Further, the correction content selection unit 106 selects the correction content at the same time, time zone (such as am and pm), and/or day of the week using the additional information at the time of correction described above, and selects the content of the scenario before correction. Considering this, the scenario correction contents may be selected so that the same scenario correction contents are not repeatedly selected for the same pre-correction scenario at the same time, time zone, and/or day of the week. As a result, various scenario correction contents are uniformly selected, so that it is possible to find a scenario in which the audience has a good reaction and to shift the audience's viewing state to a state defined as good. Further, as a result, the scenario is non-uniformly modified, so that it is possible to suppress a decrease in the effectiveness of the scenario modification due to the audience becoming accustomed to the utterance content and/or non-verbal movements of the robot 30.

The modification content selection unit 106 may select the scenario modification content based on the content of the scenario being executed. For example, the correction content selection unit 106 may not select the scenario correction content for which a non-verbal operation that matches or is similar to the non-verbal operation of the content of the scenario being executed is a correction target. By not selecting such scenario correction contents, an effect of preventing the non-verbal movement of the robot 30 from becoming monotonous can be expected. Further, the scenario to be corrected may be analyzed and corrected to an appropriate non-verbal action according to the utterance content.

The modification content rule storage unit 107 stores the modification content rule in the format of the modification rule table shown in FIG. 15, for example. In the modification content rule table of FIG. 15, a combination of an ID for identifying each modification content rule, a current state and a transition destination state in which the modification content rule can be used, and a scenario modification that is the details of the modification content rule The content and its correction target are associated. The modification content rule stored in the modification content rule storage unit 107 is read by the modification content selection unit 106 as necessary. The modification content rule storage unit 107 may correspond to, for example, the above-described memory and/or auxiliary storage device.

The revision history storage unit 108 stores the revision history in the format of the revision history table shown in FIG. 16, for example. In the correction history table of FIG. 16, an ID for specifying each of the correction content rules (of which, the scenario correction content) is associated with the cumulative selection count of the scenario correction content. The item of ID in the correction history table of FIG. 16 may be common to the item of ID in the correction content rule table of FIG. The correction history stored in the correction history storage unit 108 is read by the correction content selection unit 106 as needed. The correction history stored in the correction history storage unit 108 is updated (incremented) by the scenario correction unit 109 each time the scenario is corrected. The correction history storage unit 108 may correspond to, for example, the above-mentioned memory and/or auxiliary storage device.

The scenario correction unit 109 receives a value indicating the scenario correction content selected by the correction content selection unit 106, for example, an ID from the correction content selection unit 106, and further, a value indicating a scenario position to be corrected, and this scenario position. Receives the scenario specified by (also called the unmodified scenario). The scenario correction unit 109 corrects the scenario before this correction based on the scenario correction content and the scenario position. The scenario correction unit 109 sends the post-correction scenario to the presentation control apparatus 200 by, for example, transmission means (not shown) (for example, communication I/F). For example, the corrected scenario may be sent to the scenario execution unit 201 of the presentation control apparatus 200 to operate the presentation control apparatus 200 according to the modified scenario, or may be sent to the scenario storage unit 205 and the modified scenario is recorded and then presented. The control device 200 may be operated. The scenario correction unit 109 also updates the correction history stored in the correction history storage unit 108. More specifically, the scenario correction unit 109 increments the cumulative selection count of the applied scenario correction contents. The scenario correction unit 109 can correspond to the above-described processor, for example.

Specifically, when the correction target of the scenario correction content is the utterance content, the scenario correction unit 109 utters the utterance indicated by the scenario correction content with respect to the utterance content described in the scenario corresponding to the target scenario position. Content (additional dialogue, or utterances (eg, use of sound effects, emphasis on paralinguistics, etc.) may be added. For additional dialogue, utterances such as "Don't you understand?" and "I will explain again" are added before the utterance contents of the scenario before correction. In the use of sound effects, the sound effects such as "Jean" and "Pean" are simultaneously played at the same time as the utterance of the utterance content of the scenario before correction. In para-language important point emphasis, the volume of the utterance contents of the scenario before correction is increased, or a certain period of time is set before the utterance.

In addition, when the correction target of the scenario correction content is a non-verbal operation, the scenario correction unit 109 indicates the non-verbal operation described in the scenario corresponding to the target scenario position as the non-verbal operation ( (E.g., beckoning action, illustrated action, pointing action, attention guidance to the slide by eye gaze control, eye contact action to the audience by eye gaze control, etc.) or described in the scenario corresponding to the target scenario position. The non-verbal operation indicated by the scenario correction content may be added to the non-verbal operation. For example, if the scenario correction is a non-verbal action that attracts the audience's attention such as a beckoning action, instead of replacing it, add it before the non-verbal action described in the scenario corresponding to the target scenario position. May be. Further, when the correction targets of the scenario correction contents are all, the scenario correction unit 109 duplicates the scenario corresponding to the target scenario position and repeats the same description, for example, immediately before or after the scenario position. Can be inserted into. At the time of copying, before the utterance content in the scenario, add the words "Repeat again.", "Is it a little difficult?" Details of the relationship (correction operation) with the new scenario defined in the scenario modification content, and the new scenario (speech that represents repetition and non-verbal action corresponding to the speech) defined in the scenario modification content separately from the target scenario You may specify.

Next, a functional configuration example of the presentation control device 200 will be described. As illustrated in FIG. 1, the presentation control device 200 includes a scenario execution unit 201, an execution position notification unit 202, a scenario notification unit 203, a presentation control unit 204, and a scenario storage unit 205.

The scenario execution unit 201 reads a scenario from the scenario storage unit 205 and controls execution of the scenario. The scenario execution unit 201 sequentially interprets an element (for example, a slide or an animation set on the slide) of the video content described in the scenario, the utterance content and the non-verbal movement of the robot 30 presenting the element, It may be video data displayed on the display 20, utterance content data of the robot 30 (text data that can be processed by TTS (Text-to-Speech), and data (option) indicating the utterance method, or the voice data itself. It is also possible) to obtain motion control data for controlling non-verbal motions of the robot 30 and send them to the presentation control unit 204. The scenario execution unit 201 also sends a value indicating the execution position of the scenario to the execution position notification unit 202. Furthermore, the scenario execution unit 201 sends the scenario being executed to the scenario notification unit 203. When the scenario stored in the scenario storage unit 205 is corrected by (the scenario correction unit 109 of) the scenario control device 100, the scenario execution unit 201 operates according to the corrected scenario. The scenario execution unit 201 can correspond to the above-mentioned processor, for example.

The execution position notification unit 202 receives a value indicating the execution position of the scenario from the scenario execution unit 201, and notifies the scenario control device 100 (the transition destination determination unit 104 thereof) of the execution position. The execution position notification unit 202 can correspond to the above-mentioned communication I/F.

The scenario notification unit 203 receives the scenario being executed from the scenario execution unit 201 and notifies the scenario control device 100 (the correction content selection unit 106 thereof) of this. The scenario notification unit 203 can correspond to the above-mentioned communication I/F.

The presentation control unit 204 receives the video data of the display 20, the utterance content data of the robot 30, the operation control data of the robot 30, and the like from the scenario execution unit 201. The presentation control unit 204 gives video data to the display 20 in a timely manner, and gives utterance content data and/or control data to the robot 30 in a timely manner. The presentation control unit 204 can correspond to, for example, the above-described processor and communication I/F.

The scenario storage unit 205 stores the scenario. The scenario stored in the scenario storage unit 205 is read by the scenario execution unit 201 as needed. The scenario storage unit 205 can correspond to, for example, the above-mentioned memory and/or auxiliary storage device.

Next, an operation example of the scenario control device 100 will be described with reference to FIGS. 17 to 19. Note that the operation illustrated in FIG. 17 is repeatedly performed during the presentation of the content, but may be performed regularly, for example, every slide, or may be performed irregularly.

First, the image acquisition unit 101 acquires image data of an image of an audience from the camera 10 (step S301). The audience feature extraction unit 102 extracts the feature quantity of the audience, for example, the above-described first feature quantity and second feature quantity from the image data acquired in step S301 (step S302).

The audience state estimation unit 103 estimates the audience's viewing state as one of a plurality of states, for example, the above-described states 1 to 4 based on the feature amount extracted in step S302 (step S303).

The transition destination determination unit 104 determines whether the current state estimated in step S303 is a state that requires state transition, for example, the state 2 or the state 3 described above (step S304). If it is determined in step S304 that the current state is a state that requires state transition, the process proceeds to step S305, and if not (for example, the current state=state 1), the process ends.

In step S305, the transition destination determination unit 104 determines whether or not there is one transition destination state candidate. For example, when the current state is the state 2 and the state 3 described above, only the state 1 is the candidate for the transition destination state. On the other hand, when the current state is the above-mentioned state 4, the candidates for the transition destination state are the state 2 and the state 3. If it is determined in step S305 that there is one transition destination state candidate, the process proceeds to step S307, and if not, the process proceeds to step S306.

In step S306, the transition destination determination unit 104 refers to the state transition rule stored in the state transition rule storage unit 105 and the scenario execution position notified by the execution position notification unit 202. Then, the process proceeds to step S307.

In step S307, the transition destination determination unit 104 determines the transition destination state. Specifically, since the number of candidates for the transition destination state is one when the process does not go through step S306, the transition destination determination unit 104 determines the candidate as the transition destination state. On the other hand, in the case of passing through step S306, the transition destination determination unit 104 applies the state transition rule referred to in step S306 to the scenario progress status based on the execution position of the scenario also referred to in step S306. Any one of the plurality of candidates can be determined as the transition destination state. For example, the transition destination determination unit 104 determines the state 2 as the transition destination state when the scenario progress status is less than the policy transition threshold value, and the state 2 when the scenario progress status is equal to or greater than the policy transition threshold value. 3 can be determined as the transition destination state.

The modification content selection unit 106 selects any one of at least one scenario modification content associated with the combination of the current state estimated in step S303 and the transition destination state determined in step S307. (Step S310). A detailed concrete example of step S310 will be described later with reference to FIG.

The scenario correction unit 109 corresponds to the scenario correction content selected in step S310 and the scenario before the correction specified by the target scenario position determined by the correction content selection unit 106, for example, the slide displayed next. The utterance content and/or the non-verbal action is modified (step S320), and the process ends. A detailed concrete example of step S320 will be described later with reference to FIG.

Hereinafter, a detailed specific example of step S310 in FIG. 17 will be described with reference to FIG. The process of FIG. 18 starts from step S311.
In step S311, the modification content selection unit 106 refers to the modification content rule stored in the modification content rule storage unit 107, and identifies the current state estimated in step S303 and the transition destination state determined in step S307. At least one scenario modification content associated with the combination is obtained.

The correction content selection unit 106 determines whether or not there is one scenario correction content acquired in step S311 (step S312). If it is determined in step S312 that there is one scenario correction content, the process proceeds to step S316, and if not, the process proceeds to step S313.

In step S313, the correction content selection unit 106 refers to the correction history stored in the correction history storage unit 108, and acquires the cumulative selection count of each scenario correction content acquired in step S311. Then, the modification content selection unit 106 discards the scenario modification content having the smallest cumulative selection count acquired in step S313 (step S314).

The correction content selection unit 106 determines whether or not there is one scenario correction content remaining after step S314 (step S315). If it is determined in step S315 that there is only one scenario correction content remaining, the process proceeds to step S316, and if not, the process proceeds to step S317.

In step S316, the modification content selection unit 106 selects the only scenario modification content that remains at that time, and the process ends. On the other hand, in step S317, the correction content selection unit 106 randomly selects one of the plurality of scenario correction content remaining at that time, and the process ends.

Hereinafter, a detailed specific example of step S320 in FIG. 17 will be described with reference to FIG. The process of FIG. 19 starts from step S321.
In step S321, the scenario correction unit 109 acquires the correction target of the scenario correction content selected in step 310. The scenario correction unit 109 determines whether the correction target acquired in step S321 is a non-verbal operation, utterance content, or all.

If it is determined in step S321 that the correction target is a non-language action, the process proceeds to step S323. In step S323, the scenario correction unit 109 replaces the non-verbal action described in the target portion of the scenario before the revision with the non-verbal action indicated by the scenario revision content selected in step S310, or before the revision. The non-verbal operation indicated by the scenario correction content selected in step S310 is added to the non-verbal operation described in the target portion of the scenario. For example, if "pointing action" is described as a non-verbal action in the target part of the scenario before modification, and the non-verbal action indicated by the scenario modification content is "eye contact action to the audience by eye control". At the time of executing the portion, the robot 30 performs the eye contact operation without performing the originally planned non-verbal pointing operation. In step S323, if the non-verbal action described in the target portion of the scenario before modification is the same as or similar to the non-verbal action indicated by the scenario modification content selected in step S310, The process may return to step S310 after excluding the selected scenario correction content.

If it is determined in step S321 that the correction target is the utterance content, the process proceeds to step S324. In step S324, the scenario correction unit 109 adds, to the utterance content described in the target portion of the scenario before correction, the utterance content indicated by the scenario correction content selected in step S310, for example, the utterance method or additional dialogue. To add. For example, "I will explain XX from now on" is described as the utterance content in the target part of the scenario before modification, and the utterance content indicated by the scenario modification content is "important in para-language (volume increase)". If it is "point emphasis", the robot 30 will utter the originally planned utterance content "I will explain XX from now on" at a louder volume than usual. ..

If it is determined in step S321 that all correction targets are present, the process proceeds to step S325. In step S325, the scenario correction unit 109 duplicates the target portion of the scenario before correction, modifies a part of the scenario as needed, and then adds it. This allows the same description to be repeated.

In step S326, the scenario correction unit 109 updates the correction history stored in the correction history storage unit 108 in order to increment the cumulative selection count of the scenario correction content selected in step 310, and the process ends.

As described above, the scenario control device according to the embodiment estimates the viewing state of the audience based on the image of the audience viewing the content, and transitions the estimated viewing state to a state defined as better. , Dynamically modify the scenario in which the presentation mode of the content is described. That is, the scenario control device adaptively changes the utterance content, non-verbal motion, etc. of the presenter of the content such as a communication robot depending on the reaction of the audience. Therefore, according to this scenario control device, even when the content is presented to an audience who is not suitable for the prepared scenario, the scenario can be modified to be suitable for the audience. In short, according to the scenario control device, it is possible to support the content presentation so that the main points and details of the content can be effectively transmitted regardless of the attributes of the audience.

(Modification)
In FIG. 1, the content presentation system in which the digital signage and the communication robot are combined has been described. However, the scenario control device according to the embodiment can be applied without being limited to a specific content presentation technique.

For example, the presenter of the content is not limited to the physical robot existing in the physical space such as the robot 30. The presenter may be a virtual agent 31 displayed on the display 20 so as to be superimposed on the video content 21 or separated from the video content 21, as illustrated in FIG. In this case, the utterance content of the virtual agent 31 may be realized as audio output with the video content or text displayed with the video content, and the non-verbal operation of the virtual agent 31 may be realized as a gesture video. Such a content presentation technique may be applicable to, for example, a car navigation system. As a further modification, the video content 21 and the virtual agent 31 may be displayed in the virtual space.

Alternatively, the presenter may be the pointing device 32 illustrated in FIG. The pointing device 32 in FIG. 21 is, for example, a combination of a laser pointer and a robot arm capable of grasping the laser pointer and moving the laser irradiation position to an arbitrary position on the display 20, but this is different. A pointing device as a presenter may be realized by hardware.

Also, the presenter may not be able to speak or perform non-verbal actions. For example, the presenter may be able to speak but not be able to take non-verbal actions, or conversely, the presenter may be able to take non-verbal actions but not be able to speak. .. Furthermore, the presenter may be capable of both utterance content and non-verbal actions, but only one may be controllable.

The embodiments described above are merely illustrative for facilitating the understanding of the concept of the present invention, and are not intended to limit the scope of the present invention. The embodiment can add, delete, or change various components without departing from the scope of the present invention.

Although some functional units have been described in the above embodiments, these are merely examples of implementation of each functional unit. For example, a plurality of functional units described as being mounted on one device may be mounted on a plurality of separate devices, or conversely, may be mounted as being mounted on a plurality of separate devices. It is also possible that the implemented functional units are implemented in one device.

The various functional units described in each of the above embodiments may be realized by using a circuit. The circuit may be a dedicated circuit that realizes a specific function or a general-purpose circuit such as a processor.

At least a part of the processing of each of the above-described embodiments can be realized by using a processor mounted on a general-purpose computer as basic hardware. The program that realizes the above processing may be stored in a computer-readable recording medium and provided. The program is stored in the recording medium as an installable file or an executable file. The recording medium may be a magnetic disk, an optical disk (CD-ROM, CD-R, DVD, etc.), a magneto-optical disk (MO, etc.), a semiconductor memory, or the like. The recording medium may be any one as long as it can store the program and can be read by a computer. Further, the program that implements the above processing may be stored in a computer (server) connected to a network such as the Internet and may be downloaded by a computer (client) via the network.

10... Camera 20... Display 21... Video content 30... Robot 31... Virtual agent 32... Pointing device 100... Scenario control device 101... Image acquisition unit 102... Audience feature extraction unit 103... Audience state estimation unit 104... Transition destination determination unit 105... State transition rule storage unit 106... Correction content selection unit 107... Correction content rule storage unit 108... -Correction history storage unit 109... Scenario correction unit 200... Presentation control device 201... Scenario execution unit 202... Execution position notification unit 203... Scenario notification unit 204... Presentation control unit

Claims (8)

An acquisition unit that acquires information representing the state of the audience who is viewing the content,
An extraction unit that extracts the feature amount of the audience from the information representing the state of the audience who views the content;
An estimation unit that estimates the viewing state of the audience as one of a plurality of states including a first state based on the characteristic amount of the audience, and sets the estimated state as a current state;
A determining unit that determines any one of the plurality of states different from the current state as a transition destination state when the current state is not the first state;
A scenario control device, comprising: a selection unit that selects any one of at least one available correction content for the content presentation scenario that is associated with the combination of the current state and the transition destination state. .. The audience feature amount includes a first feature amount indicating an interest level of the audience with respect to the content, and a second feature amount indicating a concentration degree of the audience with respect to the content,
The plurality of states include a second state, a third state and a fourth state in addition to the first state,
When the first feature amount is equal to or more than a first threshold value and the second feature amount is equal to or more than a second threshold value, the estimation unit sets the audience viewing state to the first state. Estimate,
When the first feature amount is equal to or more than the first threshold value and the second feature amount is less than the second threshold value, the estimation unit determines the viewing state of the audience by the second value. Presumed to be a state,
When the first feature amount is less than the first threshold value and the second feature amount is greater than or equal to the second threshold value, the estimation unit determines the viewing state of the audience by the third value. Presumed to be a state,
When the first feature amount is less than the first threshold value and the second feature amount is less than the second threshold value, the estimating unit determines the viewing state of the audience by the fourth value. Presumed to be a state,
The scenario control device according to claim 1. The determination unit determines the second state as the transition destination state when the current state is the fourth state and the progress of the presentation scenario is less than a third threshold,
The determination unit determines the third state as the transition destination state when the current state is the fourth state and the progress of the presentation scenario is equal to or more than the third threshold. ,
The scenario control device according to claim 2. The determining unit determines the second state as the transition destination state when the current state is the fourth state and the total length of the presentation scenario is less than a fourth threshold value. Item 2. The scenario control device according to item 2. The selection unit selects, of the at least one available correction content associated with the combination of the current state and the transition destination state, one having the smallest cumulative selection count. Item 5. The scenario control device according to any one of items 4. The scenario control device according to claim 1, further comprising a correction unit that corrects a presentation scenario of the content based on the selected correction content. A scenario control method executed by a computer, comprising:
Acquiring information that represents the state of the audience watching the content,
Extracting the feature amount of the audience from information representing the state of the audience viewing the content,
Estimating the viewing state of the audience as any one of a plurality of states including a first state based on the characteristic amount of the audience, and setting the estimated state as a current state,
Determining, if the current state is not the first state, any one of the plurality of states different from the current state as a transition destination state;
Selecting any one of at least one available modification to the presentation scenario of the content that is associated with the combination of the current state and the transition destination state. A scenario control program comprising computer-readable instructions for causing a computer to function as the scenario control device according to any one of claims 1 to 6.