JP2023002380A - Program, device and method for associating person speech sound to user comments - Google Patents

Abstract

To provide a program, device and method for automatically generating a spoken dialogue scenario for a dialogue agent that can associate a person speech sound with user comments and that incorporates the person speech sound.SOLUTION: A dialog analyzer 1 comprises: a content acquisition unit 11 for acquiring contents including a person speech sound and a plurality of user comments; a speech analysis unit 12 for converting the person speech sound into a person comment; a corresponding comment selection unit 13 for calculating similarities between the person comment and the user comments posted for a predetermined time before the person comment was uttered and selecting a user comment whose similarity is higher than a predetermined threshold value; a subsequent person comment selection unit 14 that selects a person comment that was subsequently uttered following the person comment for the selected user comment; and a dialogue relationship determination unit 15 that determines the selected user comment and the selected person comment is in a dialogue relation.SELECTED DRAWING: Figure 3

Description

TECHNICAL FIELD The present invention relates to technology of a dialog agent that interacts with a user.

The technology of "dialogue agents" that interact naturally with users using smartphones and tablet terminals is widespread. According to this technique, a dialogue with a user is progressed by voice according to a pre-registered "dialogue scenario". For example, by giving a dialogue agent a profile (attributes such as age, gender, hometown, etc.) and personality, it is possible to give the user a sense of closeness to the user. The user can select a dialogue agent having a desired character according to the situation and tastes and enjoy the dialogue.

2. Description of the Related Art Conventionally, there is a technique for generating a dialogue scenario using posted messages exchanged on a SNS (Social Networking Service) such as twitter (registered trademark) (see Non-Patent Document 1, for example). According to this technology, a posted message having a high degree of similarity to the user's utterance content is retrieved, and a reply to the posted message is used as the response of the dialogue agent.

There is also a technology for speech synthesis of a speaker (reading tone) based on deep learning (see Non-Patent Document 2, for example). According to this technology, it is possible to reproduce the utterance sentence from the dialogue agent with a voice tone corresponding to the characteristics of the character.

Fumihiro Bessho, Tatsuya Harada, Yasuo Kuniyoshi, "Dialogue system using real-time crowdsourcing and Twitter large-scale corpus", Information Processing Society of Japan Research Report, Vol.2012-NL-206, [online], [June 2021 Search on the 10th], Internet <URL: https://ipsj.ixsq.nii.ac.jp/ej/?action=repository_uri&item_id=81894&file_id=1&file_no=1> Kei Hashimoto, Shinji Takagi, "Statistical speech synthesis based on deep learning", Journal of the Acoustical Society of Japan, Vol. 73, No. 1 (2017), pp.55-62, [online], [Retrieved on June 10, 2021] , Internet <URL: https://www.jstage.jst.go.jp/article/jasj/73/1/73_55/_pdf/-char/ja> YouTube > DataAPI, [online], [searched on June 10, 2021], Internet <URL: https://developers.google.com/youtube/v3?hl=ja> Zhang, Aonan, et al. "Fully supervised speaker diarization." ICASSP 2019-2019 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP). IEEE, 2019., [online], [June 10, 2021 Search], Internet <URL: https://www.researchgate.net/publication/332791521_Fully_Supervised_Speaker_Diarization>

Here, it is assumed that the dialogue agent is implemented as a specific character imitating a specific person such as an entertainer. The specific character interacts with the entertainer by imitating the contents of the normal utterances of the entertainer. By interacting with the specific character, the user can feel as if he or she is interacting with the entertainer.

However, even if a normal utterance of a specific character is reproduced by speech synthesis, the intonation and speech speed will differ, and the user will feel uncomfortable with the utterance of the specific character. As a result, the voice response is less likely to be a specific person.
For example, in response to the user's utterance, "Is there anything you eat every day?" There are times when he responds pessimistically, saying, “Nine~”. It is extremely difficult to reproduce such intonation and speech speed by speech synthesis.

In response to this, the inventors of the present application considered whether it would be possible for the dialogue agent to incorporate the uttered voice of a specific person into the dialogue scenario as it is, rather than the text imitating the utterance of the specific person. Specifically, we considered whether it would be possible to acquire the uttered voice of a specific person in a dialogue from broadcast content in which the specific person appears. For this purpose, a technique is required to associate and extract human utterances and user comments from broadcast content.

SUMMARY OF THE INVENTION Accordingly, it is an object of the present invention to provide a program, apparatus, and method for associating human utterances with user comments. Another object of the present invention is to provide a program for automatically generating a voice dialogue scenario of a dialogue agent incorporating human utterances.

According to the present invention, a program that causes a computer to function so as to associate a user comment with a human utterance resulting from the user comment,
content acquisition means for acquiring content including a person's utterance voice and a plurality of user comments;
voice analysis means for converting a person's uttered voice into a person's comment;
Corresponding comment selection means for calculating the degree of similarity between a person's comment and a user comment posted at a predetermined time before the person's comment is uttered, and selecting a user comment whose degree of similarity is higher than a predetermined threshold;
a subsequent human comment selection means for selecting a human comment uttered subsequent to the human comment for the selected user comment;
It is characterized in that the computer functions as dialogue relationship determination means for determining the dialogue relationship between the user comment selected by the corresponding comment selection means and the person's comment selected by the subsequent person comment selection means.

According to another embodiment of the program of the present invention,
the content is broadcast content,
Person-spoken voice is based on speech spoken by a person during the broadcast,
User comments are text-based, posted by listeners during the broadcast,
It is also preferred to have the computer function to include cases where user comments and person comments constitute an interactive form.

According to another embodiment of the program of the present invention,
further comprising voice dialogue scenario generating means for incorporating a person's uttered voice into a voice dialogue scenario of a specific character imitating a specific person;
Dialogue scenario generation means
Supposed sentences are registered in advance in the voice dialogue scenario,
Select the hypothetical sentence with the highest degree of similarity for the user comment that has a dialogue relationship,
Extract the utterance voice of the following person linked to the comment of the following person who has a dialogue relationship,
It is also preferred to have the computer function to associate the selected hypothetical sentence with the extracted subsequent utterance.

According to another embodiment of the program of the present invention,
One or more agreeable words (tags) are associated with the hypothetical sentences registered in advance in the voice dialogue scenario,
It is also preferable that the voice dialogue scenario generation means causes a computer to compare words included in the user comment with words associated with the assumed sentence and calculate the degree of similarity.

According to another embodiment of the program of the present invention,
If the corresponding comment selection means does not select the user comment for the person comment, instead of the subsequent person comment selection means,
a synonym retrieval means for retrieving synonyms from words included in person comments;
Functioning as user comment selection means for comparing words included in each user comment with synonyms of person comments and selecting the user comment with the highest degree of similarity,
It is also preferable that the dialogue relationship determination means causes the computer to function so that the user comment selected by the user comment selection means and the person's comment are in a dialogue relationship.

According to another embodiment of the program of the present invention,
The user comment selection means is
Analyzing the dependent word consisting of nouns and verbs included in each user comment and the dependent word consisting of nouns and verbs included in the person comment,
It is also preferred to have the computer function to compare the dependent terms of each user comment with the dependent terms of the person comment and select the most similar user comment.

According to another embodiment of the program of the present invention,
The content includes voices spoken by multiple persons,
The speech analysis means classifies the uttered voice for each person, converts each uttered voice into a person's comment,
keyword list extracting means for extracting a list of keywords of nouns included in person comments for each person;
keyword counting means for counting keywords included in the user comment in the keyword list for each person;
It is also preferable to cause the computer to function as specific person determination means for determining, among a plurality of persons, the uttered voice of the person whose count number in the keyword list is the highest as the person's uttered voice.

According to another embodiment of the program of the present invention,
The name of a specific person is preset,
It is also preferred that the keyword counting means causes the computer to extract only user comments containing a specific person's name and count the keywords contained in the user comments against the keyword list.

According to the present invention, there is provided a dialogue analysis device that associates a user comment with a human utterance resulting from the user comment,
content acquisition means for acquiring content including a person's utterance voice and a plurality of user comments;
voice analysis means for converting a person's uttered voice into a person's comment;
Corresponding comment selection means for calculating the degree of similarity between a person's comment and a user comment posted at a predetermined time before the person's comment is uttered, and selecting a user comment whose degree of similarity is higher than a predetermined threshold;
a subsequent human comment selection means for selecting a human comment uttered subsequent to the human comment for the selected user comment;
It is characterized by comprising a dialogue relationship determination means for determining a dialogue relationship between the user comment selected by the corresponding comment selection means and the person's comment selected by the succeeding person's comment selection means.

According to the present invention, there is provided a dialogue analysis method for a device that associates a user comment with a human utterance resulting from the user comment, comprising:
The device
a first step of obtaining content including a person speaking voice and a plurality of user comments;
a second step of converting the person's speech into a person's comment;
a third step of calculating a degree of similarity between a person's comment and a user comment posted at a predetermined time before the person's comment is uttered, and selecting a user comment whose degree of similarity is higher than a predetermined threshold;
a fourth step of selecting a person comment subsequently spoken after the person comment for the selected user comment;
A fifth step is executed in which the user comment selected in the third step and the person comment selected in the fourth step are associated with each other.

According to the program, device, and method of the present invention, it is possible to associate a person's uttered voice with a user comment. In addition, it is possible to automatically generate a voice dialogue scenario of a dialogue agent incorporating human utterances.

1 is a system configuration diagram in the present invention; FIG. FIG. 10 is an explanatory diagram showing a correspondence relationship between a person's comment and a user's comment; 1 is a functional configuration diagram of a dialogue analysis device according to the present invention; FIG. FIG. 10 is an explanatory diagram showing the relationship between a user comment, a person's comment, and a subsequent person's comment; FIG. 4 is an explanatory diagram of a voice dialogue scenario creation unit in the present invention; FIG. 10 is a functional configuration diagram of the dialogue analysis device when there is no person's comment to be a correspondence relationship; FIG. 2 is a functional configuration diagram of a dialogue analysis device that determines a specific person among utterances by a plurality of persons; 7 is an explanatory diagram showing processing of each functional configuration unit in FIG. 6; FIG.

BEST MODE FOR CARRYING OUT THE INVENTION Hereinafter, embodiments of the present invention will be described in detail with reference to the drawings.

FIG. 1 is a system configuration diagram in the present invention.

According to FIG. 1, a broadcasting server 2 is arranged as a video sharing platform. The broadcast server 2 delivers broadcast content based on video/audio on-demand services such as YouTube (registered trademark) and radiko (registered trademark). Broadcast content includes video such as that captured by a user and audio such as radio broadcast.
The broadcast server 2 receives broadcast contents from terminals 3 on the broadcaster side, accumulates the broadcast contents, and distributes them to a plurality of terminals 4 on the user side. Broadcast content includes a person's uttered voice uttered by a specific person who is a broadcaster.

The broadcast content is integrally configured so that user comments are displayed sequentially in synchronization with the reproduction of video and audio content.
For example, assuming live-streaming broadcast content, users can post text-based user comments in real time to specific persons (for example, entertainers and celebrities) of the broadcaster during the broadcast. A user comment is a posted sentence such as a question, an opinion, or a self-disclosure of a user as a listener. A list of posted user comments is shared and simultaneously displayed on both the terminal 3 of the broadcaster and the terminal 4 of the user. At this time, the specific person of the broadcaster can also read a part of the user comments and reply while browsing the user comments. For example, a user comment may be a question or impression from a user to a specific person, and a person's utterance voice may be an answer from a specific person to the user. That is, the user's text-based "user comment" and the specific person's uttered voice-based "personal comment" constitute the interactive format.

According to FIG. 1, a “dialogue analysis device 1” is provided that acquires broadcast content from a broadcast server 2 and analyzes the content of the dialogue.
The dialogue analysis device 1 associates human utterances with user comments from broadcast content. The dialogue analysis device 1 also functions as a voice dialogue scenario generation device that automatically generates a voice dialogue scenario of a dialogue agent incorporating human utterances. A dialogue agent based on the voice dialogue scenario can reproduce a person's utterance voice corresponding to the dialogue from a specific character (avatar) imitating the specific person.

FIG. 2 is an explanatory diagram showing the correspondence between personal comments and user comments.

According to FIG. 2(a), personal comments based on uttered voices of a specific person are arranged in chronological order. During the live streaming, the specific person speaks as if he or she is appropriately interacting with the user comments while browsing the user comments.
According to FIG. 2(b), user comments posted by users in live streaming are arranged in chronological order. A specific person of the broadcaster does not speak for all user comments. Therefore, it is unknown which user comment the specific person's utterance is based on.

According to FIG. 2, in response to the user comment "Do you often go to parks?", the specific person reads out "Do you often go to parks?" speaking.
Also, in response to the user comment "What is your favorite food?", the specific person reads "What is your favorite food?"
Further, in response to the user comment "What kind of artists do you usually listen to?", the specific person utters "I wonder if KANA will be heard" without reading out the user comment.

FIG. 3 is a functional configuration diagram of the dialogue analysis device according to the present invention.

The dialogue analysis device 1 of the present invention associates a user comment with a person's uttered voice resulting from the user comment.
According to FIG. 2, the dialogue analysis device 1 of the present invention includes a content acquisition unit 11, a voice analysis unit 12, a corresponding comment selection unit 13, a subsequent person comment selection unit 14, a dialogue relationship determination unit 15, a voice and a dialog scenario creating unit 16 . These functional components are implemented by executing a program that causes a computer installed in the device to function. In addition, the flow of processing of these functional components can also be understood as a dialogue analysis method.

[Content Acquisition Unit 11]
The content acquisition unit 11 acquires broadcast content from the broadcast server 2 .
Broadcast content includes human utterances and user comments. Person-spoken audio is voice-based that is spoken by a person during the broadcast. User comments are text-based posted by listeners during the broadcast. The interaction between the user comment and the person's utterance voice includes an interactive part.

By designating the name of a specific person to the broadcast server 2 as a search key, the content acquisition unit 11 can search and acquire broadcast content spoken by the specific person. As a technology for acquiring live streaming broadcast content, for example, there is YouTube Data API (see Non-Patent Document 3, for example). According to this technology, for example, when a specific person's name is designated, it is possible to acquire past live streaming broadcast contents (.mp4) by that specific person.

The content acquisition unit 11 outputs human speech voice (.mp3) in the broadcast content to the voice analysis unit 12 and the dialogue relationship determination unit 15 . In addition, the content acquisition unit 11 outputs the user comment to the corresponding comment selection unit 13 in synchronization with the person's uttered voice.

[Voice Analysis Unit 12]
The voice analysis unit 12 converts the person's utterance voice (.mp3) into a text-based "person's comment".
Specifically, for speech analysis, for example, Google (registered trademark) Cloud Speech-to-Text (registered trademark) and Microsoft (registered trademark) Speech to Text (registered trademark) technology can be applied. can be done.
The converted personal comment is output to the corresponding comment selection unit 13 .

FIG. 4 is an explanatory diagram showing the relationship between user comments, person comments, and subsequent person comments.

[Corresponding comment selection unit 13]
The corresponding comment selection unit 13 associates the person's comment converted from the person's uttered voice with the user's comment.
Here, only a plurality of "user comments" posted within a predetermined time (for example, one minute) before uttering the person's uttered voice to the "person's uttered voice" of a specific person are targeted. A specific person who is a broadcaster is expected to reply in real time to the latest comment among posted user comments. Therefore, it is considered that user comments posted earlier than a predetermined time period do not affect the specific person's utterances.

In addition, the corresponding comment selection unit 13 calculates the degree of similarity between the person comment and the user comment, and selects the user comment whose degree of similarity is higher than a predetermined threshold.

BERT (registered trademark), for example, may be used to calculate the degree of similarity. BERT (Bidirectional Encoder Representations from Transformers) is an encoded representation of bidirectional learning based on a Seq2seq-based pre-learning model, which is Google's natural language processing model. Applying such a distributed representation generation algorithm (embedding) replaces the text with a high-dimensional vector. It is possible to select a user comment that includes a word group of a vector that exceeds a predetermined threshold and has the highest degree of similarity with respect to the word group vector of the person's comment.

According to FIG. 4, user comments with high similarity to person comments are selected as follows.
Personal comment "Do you often go to the park?"
-> User comment "Do you often go to the park?"
Character comment "What's your favorite food?"
-> User comment "What is your favorite food?"
Here, since the specific person has read out the user comment once, the user comment highly similar to the person comment is selected and associated.

[Subsequent person comment selection unit 14]
The subsequent person's comment selecting unit 14 selects the "next subsequently uttered person's comment (following person's comment)" for the user comment selected by the corresponding comment selecting unit 13 .

According to FIG. 4, for a person comment, a successor person comment is selected as follows.
Personal comment "Do you often go to the park?"
-> Subsequent person's comment "I often go to the park on holidays"
Character comment "What's your favorite food?"
-> Subsequent person's comment "Do you like hamburgers?"
Here, the specific person reads out the user's comment once and then speaks his own answer.

[Dialogue relationship determination unit 15]
The dialogue relationship determination unit 15 determines the user comment selected by the corresponding comment selection unit 13 and the person comment (subsequent person comment) selected by the subsequent person comment selection unit 14 as a dialogue relationship. Then, the dialogue relationship determining unit 15 associates the user comment with the person's comment (subsequent person's comment) and registers them in the memory.

According to FIG. 4, a subsequent person comment is associated with a user comment as follows.
User comment "Do you often go to the park?"
-> Personal comment "I often go to the park on holidays"
User comment "What is your favorite food?"
-> Personal comment "Do you like hamburgers?"

[Voice Dialogue Scenario Creation Unit 16]
A voice dialogue scenario creating unit 16 incorporates a person's utterance voice into a voice dialogue scenario of a specific character imitating a specific person, using the dialogue relation between the user comment and the person's comment registered in the memory. This makes it possible to automatically create a voice dialogue scenario.

A voice dialogue scenario describes a series of scenarios in which a dialogue agent interacts with a user. In the voice interaction scenario, "assumed sentences" that are supposed to be spoken by the user are registered in advance.

FIG. 5 is an explanatory diagram of the voice dialogue scenario creation section in the present invention.

The voice dialogue scenario creation unit 16 processes in the following steps.
(S1) Select a hypothetical sentence with the highest degree of similarity to a user comment that has a dialogue relationship.

In order to calculate the degree of similarity, specifically, each hypothetical sentence may be associated with one or more agreeing words (tags). The voice dialogue scenario creating unit 16 may compare the words included in the user comment and the words linked to the assumed sentences to calculate the similarity between vectors of the words. A hypothetical sentence having the highest word whose inter-vector similarity between words is greater than or equal to a predetermined threshold value is selected. BERT described above may be used to calculate the degree of similarity.

According to FIG. 5, hypothetical sentences having a high degree of similarity are selected for each user comment as follows.
User comment "Do you often go to the park?"
-> word "park" (tag match)
-> Hypothetical sentence "Do you want to go to the park?" (Hypothetical sentence with highest similarity)
User comment "What is your favorite food?"
-> word "food" (tag match)
-> Hypothetical sentence "What is your favorite food?" (Hypothetical sentence with the highest degree of similarity)

(S2) A person's utterance voice (.mp3) associated with the person's comment (subsequent person's comment) that is in dialogue relationship is extracted. A person's uttered voice is included in the broadcast content acquired by the content acquisition unit 11 .

According to FIG. 5, a person's utterance voice for a person's comment is extracted.
User comment "Do you often go to the park?"
-> Personal comment "I often go to the park on holidays."
User comment "What is your favorite food?"
-> Personal comment "Do you like hamburgers?"

(S3) The assumed sentence is associated with the extracted human speech. As a result, a voice response (for example, .mp3) to be responded by the dialogue agent is associated with each assumed sentence in the voice dialogue scenario.

According to FIG. 5, a person's utterance voice for a person's comment is extracted.
User comment "Do you often go to the park?"
-> Voice response [I often go to the park on holidays].mp3
User comment "What is your favorite food?"
-> Voice response [Do you like hamburgers].mp3

The dialog agent can respond with its voice response according to the hypothetical sentence uttered by the user using the voice dialog scenario. As a result, the dialogue sequence can alternately proceed between the user and the dialogue agent.

FIG. 6 is a functional configuration diagram of the dialogue analysis device when there is no person's comment that becomes a correspondence relationship.

According to FIG. 6, it is assumed that the corresponding comment selection unit 13 does not select a user comment for a person's comment. For example, according to FIG. 2 described above, the specific person of the broadcaster utters "Do you listen to KANA?" not. That is, during the broadcast, the specific person of the broadcaster checks the user comments and then replies without reading out the comments. For this reason, no user comment that has a dialogue relationship with the person's comment "I wonder if you'll listen to KANA" has been found.

Therefore, according to FIG. 6, the dialogue analysis apparatus 1 further includes a synonym search unit 141 and a user comment selection unit 142 instead of the subsequent person comment selection unit 14 .

[Synonym search unit 141]
The synonym search unit 141 searches for synonyms from the words included in the person's comment.
Here, for example, Word2vec (word to vector) may be used to derive synonyms. Word2vec is a distributed representation technology that represents words as semantic vectors.
According to FIG. 6, for example, "singer", "artist", and "music" are derived as synonyms of the word "KANA" of the person's comment.

[User comment selection unit 142]
The user comment selection unit 142 compares words included in each user comment with synonyms of person comments, and selects a user comment with the highest degree of similarity.
According to FIG. 6, user comments including "singer", "artist" and "music" are selected as high similarities.

Here, for example, even if user comments including "artist", "music" and "singer" are selected as synonyms of the noun "KANA" of the person comment, there are cases where the following multiple user comments are selected.
Personal comment "I wonder if I'll listen to KANA"
<- User comment "What kind of artists do you usually listen to?"
<- User comment "I want to be an artist"
In this case, it is not possible to recognize which user comment the person's comment "I wonder if you'll listen to KANA" has a dialogue relationship with.

Therefore, as another embodiment of the present invention, the user comment selection unit 142 may have a dependency analysis function. Specifically, the user comment selection unit 142 compares the dependent word (noun and verb) included in each user comment with the dependent word (noun and verb) included in the person comment, and selects the word with the highest degree of similarity. Select high user comments.
Dependency analysis extracts nouns related to predicates (verbs). Japanese dependency parsing methods include, for example, CaboCha (registered trademark) based on SVM (Support Vector Machines).

Here, by performing dependency analysis on personal comments and user comments, the following determination can be made.
(Dependency relationship)
Personal comment "I wonder if I'll listen to KANA" = <KANA, listen>
"KANA"-> synonyms "singer""artist""music"
User comment “What kind of artists do you usually listen to?” = <Artist, listen>
User comment “I want to be an artist” = <Become an artist>
In this case, the user comment "What artist do you usually listen to?" matching the dependency relationship is selected for the person comment "Do you listen to KANA?".

FIG. 7 is a functional configuration diagram of a dialogue analysis device that determines a specific person among a plurality of persons speaking.
FIG. 8 is an explanatory diagram showing processing of each functional configuration unit in FIG.

According to FIGS. 2 to 6 described above, for example, the case is based on the case where there is one broadcaster, and the specific person broadcasts while replying to user comments. However, there are also cases where multiple broadcasters (a particular person and other third parties) converse within the broadcast content. In such a case, broadcast content includes voices uttered by a plurality of people, and it may not be possible to determine which broadcaster the user comment is directed to. Also in order to extract the dialogue relationship between the person's comment and the user's comment, it is necessary to identify the voice utterances of a specific person in the broadcast content.

7 and 8, the dialogue analysis device 1 includes a keyword list extraction unit 121, a keyword count unit 122, and a specific person determination unit 123 between the speech analysis unit 12 and the corresponding comment selection unit 13. have more.

The voice analysis unit 12 further has a speech voice classification function. As a result, the voice analysis unit 12 separates the uttered voices for each person, and converts each uttered voice into a person's comment. Specifically, there is a technique for distinguishing a speaker's voice from voice conversation using machine learning (see, for example, Non-Patent Document 4).
Even if broadcast content contains speeches of multiple people, the speech analysis unit 12 can separate the speeches of speakers A and B according to the progress of time, as shown in FIG. . However, at this stage, even if the speaker can be distinguished, the speaker cannot be identified.

[Keyword list extraction unit 121]
The keyword list extraction unit 121 extracts a list of keywords of nouns included in person comments for each person (speaker).
According to FIG. 8, speaker A has uttered the keywords "food", "badminton" and "baseball" for a certain period of time in the past. In addition, speaker B has uttered the keywords "hamburger", "park" and "KANA" for a certain period of time in the past.
Then, the keyword list extraction unit 121 creates a keyword list for each speaker.

[Keyword count unit 122]
The keyword counting unit 122 counts the keywords included in the user comment in the keyword list for each person (speaker).
According to FIG. 8, "hamburger" and "KANA" appear in the user comments, and the keyword list is checked. Here, speaker B has uttered "hamburger" and "KANA" in the past.

An embodiment in which the name of a specific person is set in advance for the keyword counting unit 122 may be used. The keyword counting unit 122 extracts only user comments containing the name of a specific person, and counts the keywords contained in the user comments against the keyword list.
For example, assume that "X" is set in advance as the name of a specific person. In this case, only user comments that include "X", such as the user comment "I can easily imagine Mr. X eating a hamburger deliciously", are extracted. In this case, a keyword such as "hamburger" is counted against the keyword list.

[Specific person determination unit 123]
The specific person determining unit 123 determines the uttered voice of the person whose keyword list count number is the largest among the plurality of persons as the person's uttered voice.
In particular, it can be said that there is a high possibility that a keyword included in a user comment using a specific person's name, for example, matches a keyword included in a person comment uttered by the specific person.
According to FIG. 8, speaker B has a higher check rate than speaker A (the check rate for speaker A is 0%, and the check rate for speaker B is 67%). Therefore, speaker B is determined as a specific person.

Accordingly, the corresponding comment selection unit 13 can associate the user comment with the personal comment of speaker B. FIG.
It is also preferable to process the personal comment of speaker A as a user comment. Since speaker A is not a specific person, it can be processed in the same way as user comments. For example, when speaker A says, "What is your favorite food?", speaker B may say, "I like hamburgers."

As described in detail above, according to the program, device, and method of the present invention, it is possible to associate human utterances with user comments. In addition, it is possible to automatically generate a voice dialogue scenario of a dialogue agent incorporating human utterances.
As a result, the response voice of the specific character of the dialogue agent becomes the speech voice of the specific person, and the user can feel as if he or she is having a dialogue with the specific person.

Furthermore, as a result, for example, ``it is possible to automatically generate a voice dialogue scenario of a dialogue agent by associating human utterance voices with user comments'', so that the Sustainable Development Goals (SDGs) led by the United Nations Goal 9: “Develop resilient infrastructure, promote sustainable industrialization and foster innovation.”

For the various embodiments of the present invention described above, various changes, modifications and omissions within the spirit and scope of the present invention can be easily made by those skilled in the art. The foregoing description is exemplary only and is not intended to be limiting. The invention is to be limited only as limited by the claims and the equivalents thereof.

1 dialogue analysis device 11 content acquisition unit 12 speech analysis unit 121 keyword list extraction unit 122 keyword count unit 123 specific person determination unit 13 corresponding comment selection unit 14 subsequent person comment selection unit 141 synonym search unit 142 user comment selection unit 15 dialogue relation determination unit Section 16 Voice Dialogue Scenario Creation Section 2 Broadcast Server 3 Broadcaster's Terminal 4 User's Terminal

Claims (10)

A program that causes a computer to function so as to associate a user comment with a person's uttered voice resulting from the user comment,
content acquisition means for acquiring content including a person's utterance voice and a plurality of user comments;
voice analysis means for converting a person's uttered voice into a person's comment;
Corresponding comment selection means for calculating the degree of similarity between a person's comment and a user comment posted at a predetermined time before the person's comment is uttered, and selecting a user comment whose degree of similarity is higher than a predetermined threshold;
a subsequent human comment selection means for selecting a human comment uttered subsequent to the human comment for the selected user comment;
A program for causing a computer to function as dialogue relationship determination means for determining a dialogue relationship between a user comment selected by a corresponding comment selection means and a person comment selected by a succeeding person comment selection means. the content is broadcast content,
Person-spoken voice is based on speech spoken by a person during the broadcast,
User comments are text-based, posted by listeners during the broadcast,
2. A program according to claim 1, causing a computer to function to include cases where user comments and person comments constitute an interactive form. further comprising voice dialogue scenario generating means for incorporating a person's uttered voice into a voice dialogue scenario of a specific character imitating a specific person;
Dialogue scenario generation means
Supposed sentences are registered in advance in the voice dialogue scenario,
Select the hypothetical sentence with the highest degree of similarity for the user comment that has a dialogue relationship,
Extract the utterance voice of the following person linked to the comment of the following person who has a dialogue relationship,
3. The program according to claim 1 or 2, which causes a computer to function so as to associate the selected hypothetical sentence with the extracted subsequent utterance speech. One or more agreeable words (tags) are associated with the hypothetical sentences registered in advance in the voice dialogue scenario,
4. The voice dialogue scenario generation means according to claim 3, wherein the computer is operated to calculate a degree of similarity by comparing words included in the user comment and words linked to the hypothetical sentence. program. If the corresponding comment selection means does not select the user comment for the person comment, instead of the subsequent person comment selection means,
a synonym retrieval means for retrieving synonyms from words included in person comments;
Functioning as user comment selection means for comparing words included in each user comment with synonyms of person comments and selecting the user comment with the highest degree of similarity,
5. The dialogue relation determining means causes the computer to function so that the user comment selected by the user comment selecting means and the person's comment are in a dialogue relation. program. The user comment selection means is
Analyzing the dependent word consisting of nouns and verbs included in each user comment and the dependent word consisting of nouns and verbs included in the person comment,
6. The program according to claim 5, causing the computer to compare the dependent word of each user comment with the dependent word of the person comment and select the user comment with the highest degree of similarity. The content includes voices spoken by multiple persons,
The speech analysis means classifies the uttered voice for each person, converts each uttered voice into a person's comment,
keyword list extracting means for extracting a list of keywords of nouns included in person comments for each person;
keyword counting means for counting keywords included in the user comment in the keyword list for each person;
7. A computer according to any one of claims 1 to 6, wherein the computer functions as a specific person determining means for determining, among a plurality of persons, the uttered voice of a person whose count number in the keyword list is the largest as the person's uttered voice. The program described in Section. The name of a specific person is preset,
The keyword counting means extracts only user comments containing the name of a specific person, and causes the computer to count the keywords contained in the user comments against the keyword list. 8. A program according to claim 7. A dialogue analysis device that associates a user comment with a person's uttered voice resulting from the user comment,
content acquisition means for acquiring content including a person's utterance voice and a plurality of user comments;
voice analysis means for converting a person's uttered voice into a person's comment;
Corresponding comment selection means for calculating the degree of similarity between a person's comment and a user comment posted at a predetermined time before the person's comment is uttered, and selecting a user comment whose degree of similarity is higher than a predetermined threshold;
a subsequent human comment selection means for selecting a human comment uttered subsequent to the human comment for the selected user comment;
A dialogue analysis apparatus, comprising dialogue relation determining means for determining a dialogue relation between a user comment selected by a corresponding comment selecting means and a person's comment selected by a succeeding person's comment selecting means. A dialogue analysis method for a device that associates a user comment with a human utterance resulting from the user comment, comprising:
The device
a first step of obtaining content including a person speaking voice and a plurality of user comments;
a second step of converting the person's speech into a person's comment;
a third step of calculating a degree of similarity between a person's comment and a user comment posted at a predetermined time before the person's comment is uttered, and selecting a user comment whose degree of similarity is higher than a predetermined threshold;
a fourth step of selecting a person comment subsequently spoken after the person comment for the selected user comment;
A dialogue analysis method characterized by executing a fifth step of establishing a dialogue relationship between the user comment selected in the third step and the person comment selected in the fourth step.

Images