JP7507528B1 - Speech information extraction device and program - Google Patents

Abstract

[Problem] To provide an utterance information extraction device and program that improves processing accuracy by clarifying the relationship between the speaker and the microphone when extracting a speaker's utterance from audio data. [Solution] The utterance information extraction device 1 comprises audio data acquisition means 3 for acquiring audio data 2, information acquisition means 4 for acquiring the number of microphones and the number of speakers used in the audio data 2, utterance division means 5 for dividing the audio data 2 into divided utterances by utterance, linking means 6 for linking the divided utterances with the speakers, and display means 7 for displaying the linking results linked with the speakers. When the number of microphones and the number of speakers are the same, a process is performed to link the divided utterances recorded at each microphone in the audio data 2 with a single speaker. [Selected Figure] Figure 1

Description

The present invention relates to a device and a program for recognizing speakers and extracting speech from audio data of online meetings, lectures, etc.

The technology to estimate and extract who spoke and when from audio data of online meetings or lectures is called speaker diarization technology. For example, by recognizing the speaker from the audio data of an online meeting and extracting their speech, it is possible to analyze the effectiveness of the speech in that online meeting after the fact.

Patent document 1 discloses a speaker diarization method for audio files, in which a reference voice of the speaker is used to identify the speaker of the reference voice from the audio file, and the speaker is identified for the remaining speech segments that have not been identified by using clustering.

The method in Patent Document 1 does not describe how to handle the microphone that recorded the audio file, so it simply divides the audio file into individual utterances and identifies which speaker each utterance belongs to.

JP 2022-109867 A

In the method of Patent Document 1, the same processing is performed regardless of whether the audio file was recorded using a single microphone or multiple microphones. However, the inventors of the present application have discovered that when performing speaker diarization, the relationship between the speaker and the microphone affects the accuracy of the processing.

The present invention aims to provide a speech information extraction device and program that clarify the relationship between the speaker and the microphone when extracting the speaker's speech from audio data, thereby improving processing accuracy.

In order to achieve the above object, the speech information extraction device of the present invention is an utterance information extraction device that extracts speaker utterances from audio data, and includes an audio data acquisition means for acquiring the audio data, an information acquisition means for acquiring the number of microphones and the number of speakers used in the audio data, an utterance division means for dividing the audio data into divided utterances for each utterance, a linking means for linking the divided utterances to the speakers, and a display means for displaying the linking results linked to the speakers, and is characterized in that when the number of microphones is the same as the number of speakers, the linking means links the divided utterances for each microphone to a single speaker, and when the number of microphones is less than the number of speakers, it links each divided utterance to the speaker.

The speech information extraction device of the present invention obtains the number of microphones and the number of speakers by an information obtaining means, and when the number of microphones is less than the number of speakers, the audio data recorded by each microphone contains the speech of multiple speakers, so each divided utterance is linked to a speaker. On the other hand, when the number of microphones is the same as the number of speakers, the audio data recorded by each microphone contains only the speech of a single speaker, so once it has been determined which speaker the audio data from each microphone is, there is no need to determine whether it is another speaker or not. Therefore, when the number of microphones is the same as the number of speakers, unnecessary processing is not performed, so erroneous processing is not performed, and linking processing needs to be performed only when the number of microphones is less than the number of speakers, improving processing efficiency and processing accuracy.

In the speech information extraction device of the present invention, the display means may display the speaker associated with each microphone when the number of microphones is the same as the number of speakers, and when there are multiple speakers associated with the microphones, display the main speaker and also display multiple speakers to indicate that there are multiple speakers, and when an instruction to disclose the multiple displays is given, display all of the multiple speakers associated with the microphones.

According to this configuration, when audio data recorded by a microphone includes speech from multiple speakers, the user can immediately identify the main speaker and that there are multiple speakers. Furthermore, if the user wants to know who the other speakers who used the microphone are, the user can find out who those speakers are by issuing a disclosure instruction. On the other hand, if there is no need to know information about the other speakers, or if the speakers have been mistakenly divided, the disclosure instruction can be cancelled. Here, the main speaker may be the speaker who has the longest speaking time in the audio data recorded by that microphone, or the owner of the microphone may be the main speaker.

In the speech information extraction device of the present invention, the linking means further includes a speaker DB learning means for referring to a speaker DB (DB = database; the same applies below) trained using reference voices of a plurality of speakers and learning the speaker DB, and the speaker DB learning means may make it possible to edit , as a parameter via a management screen, information indicating which voice data is to be used for learning the speaker DB as the reference voice linked to the speaker .

According to this configuration, for example, when the data of a certain speaker stored in the speaker DB is viewed, and it is determined that the speech at a meeting on a specific day is inappropriate as a feature of the speaker, for example because the speech was at a meeting attended by a different speaker, the data of the meeting on that specific day can be deleted. This editing makes the data in the speaker DB data for the original speaker, thereby improving the reliability of the data in the speaker DB.

In this configuration, the parameter may be the event at which the reference voice was recorded. Here, events include online meetings, lectures, panel discussions, and the like. In this way, when the parameter is an event, if there is an event in the speaker DB that should not be used as the reference voice, editing such as deletion can be performed for each event. The editing may be performed by a user, or may be performed automatically using predetermined rules, algorithms, etc.

In addition, in this configuration, the speaker DB may contain speech features resulting from an analysis of the reference voice of the speaker. According to this configuration, since the speaker DB data contains the features of the speaker, the speaker's speech can be accurately extracted from the voice data. Here, the reference voice is a voice that should be used as a reference when extracting the speaker's features, and includes not only normal conversation but also conversations with various emotions.

The speech information extraction device of the present invention may include a correction means that allows a user to correct the linking result of the linking means, and the speaker DB learning means may have the speaker DB learn the correction result corrected by the user, and the linking means may perform linking by referring to the learned speaker DB.

According to this configuration, even if an error occurs in the processing by the linking means, the user can correct the error and reflect it in the speaker DB. Therefore, for example, even if the user corrects one linking error, the speaker DB is updated, so that other similar errors are automatically corrected.

The present invention provides a speech information extraction device that clarifies the relationship between the speaker and the microphone when extracting a speaker's speech from audio data, thereby improving processing accuracy.

FIG. 1 is an explanatory diagram showing a functional configuration of an utterance information extraction device according to an embodiment of the present invention. 1A and 1B are explanatory diagrams showing the relationship between microphones and speakers in an online meeting. 4A to 4D are explanatory diagrams showing result display screens of the speech information extraction device of the present embodiment. 4 is a flowchart showing the operation of the speech information extraction device of the present embodiment. FIG. 2 is an explanatory diagram showing a management screen of the utterance information extraction device of the present embodiment. FIG. 13 is an explanatory diagram showing a state of correction on a result display screen of the utterance information extraction device of the present embodiment.

Next, an utterance information extraction device and an utterance information extraction program according to an embodiment of the present invention will be described with reference to Figs. 1 to 6. The utterance information extraction device 1 of this embodiment is a device that extracts and displays audio data 2, which is the audio of a meeting recorded during an online meeting or the like, for each speaker S.

As shown in FIG. 1, the functional configuration of the speech information extraction device 1 of this embodiment includes a voice data acquisition means 3 for acquiring voice data 2, an information acquisition means 4 for acquiring the number of microphones M and the number of speakers S used in the voice data 2, an utterance division means 5 for dividing the voice data 2 into divided utterances 10 by utterance, a linking means 6 for linking the divided utterances 10 with the speakers S, and a display means 7 for displaying the linking results 11 linked with the speakers S.

In addition, in the speech information extraction device 1 of this embodiment, the linking means 6 is connected to the speaker DB 8. The speaker DB 8 is provided with a speaker DB learning means 9 that learns the reference voice of the speaker S. The speaker DB 8 may be connected to the linking means 6 via a network or the like, or may be stored in a storage means inside the speech information extraction device 1.

The audio data 2 is, for example, the audio of a meeting recorded during an online meeting or the like. It also includes data recorded during a panel discussion or the like in which multiple panelists (speakers) participate.

The audio data acquisition means 3 is a functional unit that acquires the audio data 2 to the speech information extraction device 1. The data format of the audio data 2 may be audio-only data or an audio track of video data. The audio data acquisition means 3 can acquire the audio data 2, for example, by uploading the audio data 2 to the computer that serves as the speech information extraction device 1. The audio data 2 may be uploaded via a network or the like, or via a recording medium such as an SD card.

The information acquisition means 4 is a means for acquiring the number of microphones M used in the voice data 2 and the number of speakers. For example, in the case of an online meeting via a network 14 as shown in FIG. 2(B), at site (d), a speaker Sd holds the meeting in front of a meeting device 12d. The speaker Sd wears a headset 13d, and the microphone Md has a one-to-one relationship with the speaker Sd. Meanwhile, at site (e), one microphone Me is installed in front of a meeting device 12e, and multiple speakers Se1 to Se3 speak using the single microphone Me.

When extracting speaker information from audio data 2 of such a meeting, the information acquisition means 4 acquires information on the number of microphones M and the number of speakers S based on data accompanying the audio data 2, such as microphone IDs and speaker IDs. Specifically, a method of acquiring the IDs of users who participated in the meeting using an online conference tool, or a method of having users input information themselves using ACESMeet, an AI-based minutes automation system provided by the applicant, can be used.

The speech division means 5 is a functional unit that divides the voice data 2 into individual utterances to generate divided utterances 10. As a division method, for example, a method based on a power threshold of the voice data 2 can be used. In this method, the voice is divided at regular intervals (for example, 0.25 milliseconds), and if the sum of the power in the divided frames is equal to or greater than the threshold, the frame is regarded as a voice frame, and if it is less than the threshold, it is regarded as a non-voice frame. Then, a section in which voice frames are consecutive is collected as divided utterances 10. Note that the method for dividing the voice data 2 is not limited to this method, and other known methods may be used.

The linking means 6 is a functional unit that links the divided utterances 10 divided by the utterance dividing means 5 with the speaker S. For example, in the case of an online meeting as shown in FIG. 2(A), in locations (a) to (c), each speaker Sa to Sc is provided with one microphone Ma to Mc. In such a case, the divided utterances 10 for the voice data 2 of each microphone Ma to Mc contain only the voices of the speakers Sa to Sc who are the users of the microphones Ma to Mc, so the divided utterances 10 of the microphones Ma to Mc are not linked to speakers other than the speakers Sa to Sc.

On the other hand, when an online meeting like that shown in FIG. 2(B) takes place, multiple speakers Se1 to Se3 are using the microphone Me at location (e), so the divided utterance 10 from the microphone Me is linked to which speaker it belongs to.

The display means 7 is a functional unit that displays the linking result 11, which is the divided utterances 10 linked to the speaker S. As shown in FIG. 3(A), for example, the linking result 11 is displayed on a result display screen 20 for each microphone M, showing the name 21 of the speaker S who is speaking at that microphone M, and the divided utterances 10 showing the utterances made by the speaker at each microphone. FIG. 3(A) shows the linking result 11 for the online meeting of FIG. 2(A).

The linking result 11 during the online meeting in FIG. 2(B) is as shown in FIG. 3(B). In FIG. 3(B), the name 21 of the main speaker of that microphone and the divided speech 10 are displayed for each microphone. Also, in FIG. 3(B), to the left of the display of the name 21 of the microphone in the upper row, a multiple display 22 is displayed to indicate that there are multiple speakers using that microphone.

In this way, in the speech information extraction device 1 of this embodiment, the display means 7 displays the multiple display 22 to inform the user that there are multiple speakers linked to the microphone. In addition, the multiple display 22 can display all of the multiple speakers linked to the microphone when the user issues a disclosure instruction by performing a touch operation or the like.

Speaker DB8 is a database that stores the results of learning using the reference voices of multiple speakers. The data stored in speaker DB8 includes the name of the speaker, multiple reference voices of the speaker, and speech features (vectors) resulting from analysis of the reference voices.

For example, "d-vector" or "i-vector", which are known as text-dependent speaker authentication using a deep neural network, can be used as the speaker DB8. In addition, other databases such as "Pyannote.audio", an open source framework based on Python, can also be used as the speaker DB8.

The speaker DB learning means 9 is a functional unit whose parameters can be edited using the management screen 23 shown in FIG. 5, and which learns the speaker DB 8 by changing the parameters. As shown in FIG. 5, the management screen 23 displays the currently registered conference (event) as a parameter in the item "List of conferences registered in the database for the speaker XX". In this case, the event that is the parameter is a conference.

Next to the conference, a delete button 24 is provided, which is used when deleting the conference. An add button 25 is provided at the bottom of the conference list. A register button 26 for registering edited content and a cancel button 27 for canceling are provided below the management screen 23. In this embodiment, the management screen 23 displays conference data as parameters for learning the speaker DB 8, but this is not limiting, and it is also possible to specify a specific utterance during the conference, for example, only a specific section of one hour of audio data, and an audio file uploaded by the user himself can be used as a parameter.

The linking means 6 determines which speaker a certain split utterance 10 is the utterance of, based on the data stored in this speaker DB 8. As a method of determination, the linking means 6 may use AI (Artificial Intelligence) to make the determination. When making a determination using AI, the input to the AI is the split utterance 10, and the output is the speaker S who is estimated to have spoken the split utterance 10.

As another method of determination, the voice of the divided utterance 10 is analyzed to calculate the features of the utterance, and the features are compared with data on the features stored in the speaker DB 8 to determine the speaker who has made an utterance with similar features. Also, as a method of determination, a publicly known method in the research field of "speaker classification" or "speaker verification" may be adopted.

The speech information extraction device 1 of this embodiment is realized by running the speech information extraction program 1P using a computer (user terminal 1U) used by a user. This speech information extraction program 1P may be installed on a personal computer, or may be installed on a server and used on a client terminal. It may also be stored on a CD-ROM, DVD-ROM, etc., or uploaded to a server and available for download over a network.

The computer on which the speech information extraction program 1P is executed is equipped with a processor such as a CPU (Central Processing Unit), a GPU (Graphics Processing Unit), a hard disk, storage means such as memory, and means for connecting to various networks, a keyboard, a mouse, a display, etc. (not shown).

Next, the operation of the speech information extraction device 1 of this embodiment will be described with reference to Figs. 1 to 6. When a user starts the user terminal 1U and starts the speech information extraction program 1P, an initial screen for recognizing the user is displayed on the screen of the user terminal 1U (not shown). When the user inputs the necessary information on the initial screen, if there is a registered online conference tool, the audio data acquisition means 3 inputs audio data 2 from the video data of that online conference tool.

Next, when the user designates the speaker DB 8 to be referenced and issues an execution instruction, the speech division means 5 first performs a division process to divide the voice data 2 into divided utterances 10 (STEP 1). The division method used is a method based on the power threshold of the voice data 2 described above.

Next, the linking means 6 performs a linking process to link the divided utterances 10 with the speakers S. In this linking process, the linking means 6 first obtains information such as the number of microphones M and the number of speakers S, and the owners of the microphones M, from the data accompanying the voice data 2 (STEP 2).

Here, it is determined whether the number of microphones M is the same as the number of speakers S (STEP 3).
If the number of microphones M and the number of speakers S are the same (Y in STEP 3), a process is performed in which the divided utterances 10 recorded by each microphone M in the voice data 2 are linked to a single speaker (STEP 4).

Specifically, for the divided speech 10 recorded by each microphone M, each microphone M is linked to the speaker S based on the information on the owner of the microphone M obtained in STEP 2. The display means 7 displays the results of this linking on the result display screen 20 (STEP 5).

On the other hand, when the number of microphones M is less than the number of speakers S (N in STEP 3), each divided utterance 10 is linked to a speaker S (STEP 6). In the linking process, the divided utterances 10 generated from the voice data 2 are input to the decision AI, and as an output, information on the divided utterances 10 linked to the speakers is obtained and displayed by the display means 7 (STEP 7).

Here, in the case where the audio data 2 acquired by the audio data acquisition means 3 is not divided into audio data for each microphone M but is in one file, and if the speaker S of each microphone M can be identified from the video data that is the source of the audio data 2, the speaker S is linked to each microphone M. On the other hand, if the speaker S of each microphone M cannot be identified, processing is performed assuming that multiple speakers S are speaking at one microphone M.

In rare cases, the number of microphones M may be greater than the number of speakers S. In this case, during the splitting process by the speech splitting means 5, a microphone M with low power may be identified, and it may be determined that the microphone M is not in use, and subsequent processing may be performed excluding that microphone M. Alternatively, after splitting processing for each microphone M, linking processing may be performed as usual.

The display by the display means 7 is performed by displaying the results of the linking by the linking means 6 on the user terminal 1U (STEPs 5 and 7). The results of the linking can be, for example, as shown in FIG. 3.

When the number of microphones M and the number of speakers S are the same (Y in STEP 3), the display means 7 displays the result display screen 20 shown in FIG. 3(A) (STEP 5). In FIG. 3(A), as a result of the linking, the name (surname) of the main speaker S at each microphone and the divided utterances 10 spoken by the speaker S in the voice data 2 obtained by that microphone are displayed. The divided utterances 10 are displayed in chronological order with the time stored in the voice data 2 on the horizontal axis. In the example of FIG. 3(A), it can be seen that Yamada, the speaker S, spoke first, followed by Suzuki and Sato.

On the other hand, when the number of microphones M is less than the number of speakers S (N in STEP 3), the display becomes the result display screen 20 shown in FIG. 3(B). In FIG. 3(B), the main speaker S of each microphone is displayed as a result of the linking. Specifically, Mr. Yamada's name is displayed on the microphone in the first row, and Mr. Sato's name is displayed on the microphone in the second row.

In addition, on the result display screen 20 in FIG. 3(B), to the left of the microphone display in the first row, a multiple display 22 is displayed, indicating that there are multiple speakers S using that microphone M. Specifically, the multiple display 22 is a triangle with one apex pointing toward speaker S, Mr. Yamada.

The user can issue a disclosure instruction for the multiple displays 22 by clicking the mouse of the user terminal 1U or by touching the screen. When a disclosure instruction from the user is received (Y in STEP 8), the display means 7 displays all of the multiple speakers linked to the microphone M (STEP 9). At this time, the multiple displays 22 are changed so that one apex of the triangle faces downward. In FIG. 3(C), it can be seen that there are two other speakers linked to Mr. Yamada's microphone M.

In FIG. 3(C), the speakers linked to Yamada's microphone M are "Yamada_1" and Suzuki. The display of "Yamada_1" shows a split utterance 10 linked to Yamada's microphone M, which could not be linked by the linking means 6. If the user clicks on the multiple display 22 again in this state, the display of the split utterances 10 other than Yamada's that were displayed will be collapsed, and the state will return to that of FIG. 3(B).

For a speaker S who could not be linked, the user can specify the split utterance 10 of that speaker and instruct it to be played back, thereby playing back and checking the contents of the split utterance 10. In this embodiment, in such a case, the split utterance 10 that is incorrectly linked can be corrected on the result display screen 20 (correction means in the present invention).

If the user wants to modify the speaker S for the segmented utterance 10 (Y in STEP 10), the display of the segmented utterance 10 can be changed to the content modified by the user (STEP 11), as shown in FIG. 3(D). Here, "Yamada_1" is modified to Mr. Saito.

In this way, when the linking of the split utterances 10 is modified by the user, the voice data 2 from which the speaker was extracted this time is registered in the speaker DB 8 as data to be used for linking the speaker (STEP 12). This makes it possible to use the voice data 2 for linking the speaker in the next and subsequent processes.

Next, referring to Figure 5, we will explain the case where a user uses the speaker DB learning means 9 to change parameters on the management screen 23 and learn the speaker DB 8. In Figure 5, a list of conferences registered in the database of the speaker "Mr. XX" is displayed. When the user looks at this list of conferences and decides that, for example, Conference 2 is not suitable for learning, he or she can click the delete button 24 to the right of Conference 2 to delete Conference 2.

Furthermore, if the user decides that he or she wishes to register another conference, he or she clicks the Add button 25. This causes a screen (not shown) to be displayed prompting the user to specify the conference that he or she wishes to register, and the user can then specify the voice data 2 of the conference that he or she believes should be registered, and register it in the speaker DB 8.

Next, correction of errors in the segmented utterances 10 that have been subjected to the linking process will be described with reference to FIG. 6. For example, in the state shown in FIG. 6(A), when the user checked the segmented utterance 10 on the right side of Suzuki out of the three segmented utterances 10 recorded by Yamada's microphone M, it was found to be the speech of Sato, not Suzuki.

In this case, since a correction is necessary (Y in STEP 10), the user can correct the divided utterance 10 on the right side by moving it from Mr. Suzuki to Mr. Sato using a mouse or touch operation (STEP 11). When the user confirms this correction, the correction is registered in the speaker DB 8 (STEP 12).

If the user wishes to perform the linking process again, the process can be executed again, and the linking means 6 can refer to the trained speaker DB 8 after the correction and perform the linking process again. As a result, for example, when it is determined that the second split utterance 10 from the left of Mr. Suzuki in FIG. 6 is also an utterance of Mr. Sato, the display means 7 displays the corrected split utterance 10 as shown in FIG. 6 (B).

As described above, according to the speech information extraction device 1 of this embodiment, when the number of microphones M is the same as the number of speakers S, the audio data 2 recorded by each microphone M contains only the speech of a single speaker, so there is no need to perform processing to determine which speaker S is speaking for the audio data 2 from each microphone M. Therefore, in this case, unnecessary processing is not performed, so erroneous processing is not performed, and linking processing only needs to be performed when the number of microphones M is less than the number of speakers, improving processing efficiency and processing accuracy.

In addition, when the audio data 2 recorded by a certain microphone M includes speech from multiple speakers S, the user can see at a glance on the result display screen 20 which speaker S is the main speaker, and that there are multiple speakers S. In addition, if the user wants to know who the other speakers S are who used that microphone M, they can find out about those speakers S by issuing a disclosure instruction.

On the other hand, when the user checks the other speaker S using the microphone M through the multiple display 22, he or she may discover that the speaker S has been mistakenly recognized as two or more people speaking, when in fact it is only one speaker S. In this case, the user can collapse the display of the multiple split utterances 10 that were displayed by clicking on the multiple display 22 again, returning to the state shown in FIG. 3(B). With this configuration, even if the speaker S has been recognized incorrectly, the incorrect split utterances 10 can be hidden.

In addition, when data on a certain speaker S stored in speaker DB8 is judged to be unsuitable as learning data by looking at the conference records recorded as parameters, the data on the conference on that particular day can be edited, such as by deleting it. This editing can improve the reliability of the data in speaker DB8.

In addition, by making the parameters of the speaker DB8 editable, it becomes possible to link multiple conferences to one speaker S. Even for the same speaker S, the voice characteristics are affected by the acoustic environment, such as the type of microphone used, the speaker's physical condition, speech variation (laughing, whispering, etc.), the distance between the microphone and the speaker, and the size of the room and characteristics related to reverberation. For this reason, if the features of speaker S are estimated only from a specific conference, the accuracy of the determination of speaker S in other acoustic environments will be low, but by linking conferences in multiple acoustic environments, the accuracy of the determination can be improved.

Furthermore, even if there is an error in the split utterance 10 that has been linked, the user can make corrections and the correction results are reflected in the speaker DB 8, so that once one correction is made, other similar corrections can be made automatically.

In the above embodiment, when the voice data 2 recorded by the microphone M includes multiple speakers S, the main speaker S is the speaker S who has the longest speaking time in the voice data 2 recorded by the microphone M on the result display screen 20. However, this is not limited to this, and the speaker S registered as the owner of the microphone M may be the main speaker S.

In addition, in the above embodiment, the voice data 2 is automatically input by the voice data acquisition means 3 from the video data of the registered online conference tool, but this is not limited to the above. An input screen (not shown) may be displayed on the user terminal 1U, and the voice data 2 may be input from the input screen.

In the above embodiment, the multiple displays 22 are triangular displays, but this is not limiting and other display methods may be used as long as it is possible to recognize that there are multiple displays. For example, any display may be used, such as a flashing arrow or other figure, or a flashing figure that stops flashing when expanded.

In the above embodiment, in STEP 4, when the process of linking the divided utterances 10 recorded by each microphone M in the voice data 2 to a single speaker is performed, the divided utterances 10 recorded by each microphone M are linked to each microphone M and speaker S based on the information on the owner of the microphone M obtained in STEP 2, but the information may also be input to a judgment AI that makes a judgment by referring to the speaker DB 8, and the speaker who is estimated to have made the utterance is obtained as the output.

In the above embodiment, the speaker DB learning means 9 is configured to allow parameters to be edited using the management screen 23 shown in FIG. 5, but this is not limiting and learning may be performed automatically. For example, when the split utterances 10 recorded by each microphone M are linked to a single speaker in STEP 4, additional learning may be performed in the speaker DB 8 to update the database. A separate learning management screen (not shown) may also be provided to allow the user to set whether or not to learn the results of the linking process in STEP 4.

In addition, in the above embodiment, the multiple display 22 displays all of the multiple speakers linked to that microphone when the user issues a disclosure instruction by performing a touch operation or the like, but this is not limited to this. A setting screen (not shown) for determining whether or not to display multiple displays may be provided, and it may be possible to set whether or not to display all of the multiple displays according to that setting.

Reference Signs List 1: Utterance information extraction device 1P: Utterance information extraction program 2: Voice data 3: Voice data acquisition means 4: Information acquisition means 5: Utterance division means 6: Linking means 7: Display means 8: Speaker DB
9... Speaker DB learning means 10... Divided utterance 11... Linking result 20... Result display screen 22... Multiple display 23... Management screen M... Microphone S... Speaker

Claims (7)

An utterance information extraction device that extracts a speaker's utterance from audio data, comprising:
a voice data acquisition means for acquiring the voice data;
an information acquiring means for acquiring the number of microphones used in the voice data and the number of speakers;
an utterance division means for dividing the voice data into divided utterances for each utterance;
Linking means for linking the divided utterances with the speakers;
A display means for displaying a result of the association associated with the speaker,
When the number of the microphones is the same as the number of the speakers, the linking means links the divided utterances for each of the microphones to a single speaker;
When the number of microphones is smaller than the number of speakers, the speech information extraction device links each divided utterance with the speaker. The speech information extraction device according to claim 1,
The display means includes:
When the number of the microphones is equal to the number of the speakers, a speaker associated with each of the microphones is displayed;
This speech information extraction device is characterized in that, when there are multiple speakers linked to the microphone, it displays the main speaker and also displays multiple speakers to indicate that there are multiple speakers, and when an instruction to disclose the multiple displays is given, it displays all of the multiple speakers linked to the microphone. The speech information extraction device according to claim 1,
The linking means refers to a speaker DB trained using reference voices of a plurality of speakers,
The speaker DB learning means further includes a speaker DB learning means for learning the speaker DB,
The speaker DB learning means is characterized in that information indicating which voice data is to be used for learning the speaker DB as the reference voice linked to the speaker is editable as a parameter via a management screen. The speech information extraction device according to claim 3,
The speech information extraction device, wherein the parameter is an event at which the reference speech was recorded. The speech information extraction device according to claim 3,
The speech information extraction device, wherein the speaker DB includes speech features resulting from analysis of a reference voice of the speaker. The speech information extraction device according to claim 3 ,
a correction means for allowing a user to correct the result of the linking means,
The speaker DB learning means learns the result of the correction made by the user in the speaker DB,
The utterance information extraction device is characterized in that the linking means performs the linking by referring to the learned speaker DB. An utterance information extraction program for causing a computer to operate as the utterance information extraction device according to any one of claims 1 to 6.