WO2024122304A1 - 声紋照合支援システム、声紋照合支援方法、及びプログラム - Google Patents
声紋照合支援システム、声紋照合支援方法、及びプログラム Download PDFInfo
- Publication number
- WO2024122304A1 WO2024122304A1 PCT/JP2023/041303 JP2023041303W WO2024122304A1 WO 2024122304 A1 WO2024122304 A1 WO 2024122304A1 JP 2023041303 W JP2023041303 W JP 2023041303W WO 2024122304 A1 WO2024122304 A1 WO 2024122304A1
- Authority
- WO
- WIPO (PCT)
- Prior art keywords
- voice data
- voiceprint
- voiceprint matching
- extraction
- time
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Ceased
Links
Images
Classifications
-
- G—PHYSICS
- G10—MUSICAL INSTRUMENTS; ACOUSTICS
- G10L—SPEECH ANALYSIS TECHNIQUES OR SPEECH SYNTHESIS; SPEECH RECOGNITION; SPEECH OR VOICE PROCESSING TECHNIQUES; SPEECH OR AUDIO CODING OR DECODING
- G10L17/00—Speaker identification or verification techniques
- G10L17/06—Decision making techniques; Pattern matching strategies
-
- G—PHYSICS
- G10—MUSICAL INSTRUMENTS; ACOUSTICS
- G10L—SPEECH ANALYSIS TECHNIQUES OR SPEECH SYNTHESIS; SPEECH RECOGNITION; SPEECH OR VOICE PROCESSING TECHNIQUES; SPEECH OR AUDIO CODING OR DECODING
- G10L15/00—Speech recognition
- G10L15/04—Segmentation; Word boundary detection
-
- G—PHYSICS
- G10—MUSICAL INSTRUMENTS; ACOUSTICS
- G10L—SPEECH ANALYSIS TECHNIQUES OR SPEECH SYNTHESIS; SPEECH RECOGNITION; SPEECH OR VOICE PROCESSING TECHNIQUES; SPEECH OR AUDIO CODING OR DECODING
- G10L17/00—Speaker identification or verification techniques
-
- G—PHYSICS
- G10—MUSICAL INSTRUMENTS; ACOUSTICS
- G10L—SPEECH ANALYSIS TECHNIQUES OR SPEECH SYNTHESIS; SPEECH RECOGNITION; SPEECH OR VOICE PROCESSING TECHNIQUES; SPEECH OR AUDIO CODING OR DECODING
- G10L17/00—Speaker identification or verification techniques
- G10L17/02—Preprocessing operations, e.g. segment selection; Pattern representation or modelling, e.g. based on linear discriminant analysis [LDA] or principal components; Feature selection or extraction
-
- G—PHYSICS
- G10—MUSICAL INSTRUMENTS; ACOUSTICS
- G10L—SPEECH ANALYSIS TECHNIQUES OR SPEECH SYNTHESIS; SPEECH RECOGNITION; SPEECH OR VOICE PROCESSING TECHNIQUES; SPEECH OR AUDIO CODING OR DECODING
- G10L17/00—Speaker identification or verification techniques
- G10L17/06—Decision making techniques; Pattern matching strategies
- G10L17/08—Use of distortion metrics or a particular distance between probe pattern and reference templates
Definitions
- This disclosure relates to a voiceprint matching support system, a voiceprint matching support method, and a program.
- Voiceprint matching for voiceprint identification and other purposes involves an operator manually extracting audio or the entire video containing audio by listening to the entire audio to find the section where the speaker being matched is likely to be speaking.
- Patent Document 1 also discloses a speaker search system for solving the problem that in a speaker search system, when the speakers of detected voices are similar, it is difficult to determine whether the detection result is actually spoken by the person to be searched for.
- the speaker search system described in Patent Document 1 includes a voice database that accumulates voice data, an optimal listening interval detection unit, a speaker search unit, and a search result presentation unit.
- the optimal listening interval detection unit detects an optimal listening interval with high speaker specificity from the accumulated voice data.
- the speaker search unit searches the accumulated voice data for voice data spoken by the same speaker as the voice or speaker name input by the user.
- the search result presentation unit presents information about the voice data obtained by the speaker search unit together with information about the optimal listening interval with high speaker specificity of the voice data detected by the optimal listening interval detection unit.
- voiceprint matching for voiceprint identification etc. requires manual work by an operator, but this work requires repeatedly listening to the entire audio, which takes a lot of time, and it is easy to imagine that fatigue can lead to a decrease in the accuracy of voiceprint matching or lead to inconsistencies in accuracy.
- Patent Document 1 describes detecting speech sections of the same speaker by clustering
- the technology described in Patent Document 1 does not take into consideration the fact that speech sections may overlap with speech from speakers other than the specific speaker. Therefore, the technology described in Patent Document 1 cannot present to the user the section in which a specific speaker is speaking while eliminating the overlap of speech sections with speech from speakers other than the specific speaker, and there is room for improvement in the information presented.
- the present disclosure has been made to solve the above problems, and has as its object the provision of a voiceprint matching support system, a voiceprint matching support method, and a program as follows.
- the present disclosure has as its object the provision of a voiceprint matching support system and the like that is capable of suppressing the inclusion of portions of voice data to be compared that are spoken by a person other than the subject when presenting the portions of voice data to be compared that are spoken by a person other than the subject to the voiceprint matching.
- the voiceprint matching support system includes an input unit for inputting sample voice data of a person to be subjected to voiceprint matching, an extraction unit for extracting a time period during which the person is speaking from the voice data to be subjected to voiceprint matching based on the sample voice data, and a display unit for presenting the time period extracted by the extraction unit to a user.
- the voiceprint matching support method also includes an input process for inputting sample voice data of a person to be subjected to voiceprint matching, an extraction process for extracting a time period during which the person is speaking from the voice data to be subjected to voiceprint matching based on the sample voice data, and a display process for displaying the extracted time period on a display device to present to a user.
- the program disclosed herein is a program that causes a computer to execute a voiceprint matching assistance process that includes an input process for inputting sample voice data of a person to be subjected to voiceprint matching, an extraction process for extracting a time period during which the person is speaking from the voice data to be subjected to voiceprint matching based on the sample voice data, and a display process for displaying the extracted time period on a display device to present to a user.
- a voiceprint matching assistance process that includes an input process for inputting sample voice data of a person to be subjected to voiceprint matching, an extraction process for extracting a time period during which the person is speaking from the voice data to be subjected to voiceprint matching based on the sample voice data, and a display process for displaying the extracted time period on a display device to present to a user.
- a voiceprint matching support system that can prevent the inclusion of speech by persons other than the subject when presenting to a user the portion of voice data to be matched that is spoken by the subject of voiceprint matching.
- FIG. 1 is a block diagram showing an example configuration of a voiceprint matching support system according to the present disclosure.
- 2 is a flow diagram for explaining an example of a voiceprint matching support method in the voiceprint matching support system of FIG. 1.
- FIG. FIG. 13 is a block diagram showing another example configuration of the voiceprint matching support system according to the present disclosure.
- 4 is a diagram showing an example of a waveform of voice data to be subjected to voiceprint matching, which is to be processed by the voiceprint matching support system of FIG. 3;
- FIG. 5 is a conceptual diagram conceptually showing an example of voice print matching target voice data to be processed by the voice print matching support system of FIG. 3, and is a diagram typically showing the waveforms of the voice print matching target voice data of FIG. 4 divided for each speaker.
- FIG. 4 is a diagram showing an example of a waveform of sample voice data used in the voiceprint matching support system of FIG. 3.
- FIG. 7 is a conceptual diagram conceptually showing an example of sample voice data used in the voiceprint matching support system of FIG. 3, and is a diagram typically showing a waveform of the sample voice data of FIG. 6.
- 5 is a diagram showing an example of a display in which an extracted time section is displayed for the waveform of the voice data to be voiceprint matched in FIG. 4;
- FIG. 5 is a diagram showing another example of displaying an extracted time section for the waveform of the voice data to be voiceprint matched in FIG. 4 .
- FIG. 5 is a diagram showing another example of displaying an extracted time section for the waveform of the voice data to be voiceprint matched in FIG. 4 .
- FIG. 5 is a diagram showing another example of displaying an extracted time section for the waveform of the voice data to be voiceprint matched in FIG. 4 .
- FIG. FIG. 11 is a flow diagram for explaining the process of creating a voiceprint matching target file in accordance with Comparative Example 1.
- FIG. 11 is a flow diagram for explaining the process of creating a voiceprint matching target file in accordance with Comparative Example 2.
- FIG. 2 is a diagram illustrating an example of a hardware configuration included in the device.
- Fig. 1 is a block diagram showing an example of the configuration of a voiceprint matching support system according to the present disclosure.
- the voiceprint matching support system 1 according to the present disclosure shown in Fig. 1 can be a system that supports a user in performing voiceprint matching by presenting to the user voice data that are candidates for subjects for voiceprint matching.
- the voiceprint matching support system 1 can be used in voiceprint analysis work such as police forensics and various other tasks that require voiceprint matching.
- the voiceprint matching support system 1 shown in FIG. 1 can include an input unit 1a, an extraction unit 1b, and a display unit 1c.
- the input unit 1a inputs sample voice data of a person to be subjected to voiceprint matching.
- the person to be subjected to voiceprint matching is one person.
- the sample voice data can be one-channel or multi-channel voice data recorded so as to include the speech of only the person to be subjected to voiceprint matching.
- the input of the sample voice data is referenced when the extraction unit 1b executes the extraction, so it is sufficient to store it inside the voiceprint matching support system 1 so that it can be referenced before the extraction is executed.
- the input source can be, for example, a voice acquisition device such as a microphone that acquires voice data, or a computer such as a server computer that stores sample voice data acquired in advance. These voice acquisition devices or computers can also be provided in the voiceprint matching support system 1.
- the input unit 1a is a part that inputs sample voice data and passes it to the extraction unit 1b, and can be provided with an input interface such as a communication interface for this purpose.
- the extraction unit 1b extracts time periods in which the subject is speaking from the voice data to be compared to voiceprints, based on the sample voice data. Because this extraction is based on the sample voice data, the extraction unit 1b analyzes both data and extracts time periods that are highly related to the sample voice data.
- the extraction unit 1b can, for example, divide the sample voice data at predetermined time intervals and also divide the voice data to be compared to voiceprints at the same predetermined time intervals, and extract the time periods in which the subject is speaking for each of the predetermined time intervals. Note that no process of recognizing the voice content of the voice data to be compared to voiceprints is required here.
- the extraction unit 1b can also extract time intervals by using a learning model that is trained to input sample voice data and voice data to be compared with voiceprints and output the time interval during which the subject is speaking.
- the algorithm of this learning model is not important, and the input data can be divided into the above-mentioned predetermined time intervals, for example.
- the voice data to be compared to the voiceprint can be recorded voice data of one channel or multiple channels.
- the voice data to be compared to the voiceprint is extracted based on sample voice data of the subject, so that even if the voice data to be compared to the voiceprint is monaural voice data, it is possible to extract the time period during which the subject is speaking.
- the voice data to be compared to the voiceprint can be data associated with video data, in other words, video data with audio.
- the voice data to be subjected to voiceprint matching can be stored in advance in a computer such as a server computer and retrieved from that computer, or can be stored in a storage device provided in the voiceprint matching support system 1 and read out from that storage device.
- This computer can also be provided in the voiceprint matching support system 1.
- the voice data to be compared with the voiceprint can be data of a conversation taking place in real time acquired by a voice acquisition device such as a microphone, or data acquired from the contents of a telephone conversation taking place in real time.
- a voice acquisition device such as a microphone
- data acquired from the contents of a telephone conversation taking place in real time can be data acquired in real time as in these examples.
- the display unit 1c presents the time interval extracted by the extraction unit 1b to the user.
- the display unit 1c may include a display device and a display control unit that controls the display on the display device.
- This display device may be any display capable of displaying images, such as an LCD (Liquid Crystal Display) or an organic EL (Electro-Luminescence) display, or may be a projector.
- the display device may be, for example, a display included in a smartphone or a tablet terminal.
- the voiceprint matching support system 1 can be, for example, a computer such as a server computer, a personal computer, or a smartphone.
- the voiceprint matching support system 1 can be configured to include a computer device including hardware including, for example, one or more processors and one or more memories. At least a portion of the functions of each part in the voiceprint matching support system 1 can be realized by one or more processors operating in accordance with a program read from one or more memories.
- the voiceprint matching support system 1 can be provided with a control unit (not shown) that controls the entire system.
- This control unit can be realized, for example, by a processor, a working memory, and a non-volatile storage device that stores a program.
- This processor can be, for example, a CPU (Central Processing Unit) or a GPU (Graphics Processing Unit).
- This program can be a program for causing the processor to execute the input processing of the input unit 1a, the extraction processing of the extraction unit 1b, and the display control processing of the display unit 1c.
- the voiceprint matching support system 1 can be provided with a storage device that stores the input sample voice data, the voice data to be matched with voiceprints, information indicating the extracted time interval, etc., and this storage device can be, for example, a storage device provided in the above-mentioned control unit.
- the voiceprint matching support system 1 can be configured as a device equipped with dedicated hardware.
- the voiceprint matching support system 1 is not limited to being configured as a single device, but can also be constructed as multiple devices with distributed functions, and the method of distribution is not important.
- One of the distributed devices can, of course, be a display device.
- each device is provided with a control unit, a communication unit, and, as necessary, a memory unit, etc.
- the multiple devices can be connected as necessary via wireless or wired communication and work together to realize the functions described in the voiceprint matching support system 1.
- FIG. 2 is a flow diagram for explaining an example of a voiceprint matching support method in the voiceprint matching support system 1 of FIG. 1.
- This voiceprint matching support method executes the following voiceprint matching support process.
- an input process is executed to input sample voice data of the person to be voiceprint matched (step S1).
- an extraction process is executed to extract a time section in which the person is speaking from the voice data to be matched based on the sample voice data (step S2).
- a display process is executed to display the extracted time section on a display device to present it to the user (step S3), and the process ends.
- the above program can be a program that causes a computer to execute the voiceprint matching support process including such input process, extraction process, and display process.
- the voiceprint matching support system 1 inputs sample voice data of the subject (speaker) of voiceprint matching, analyzes and extracts time intervals from the voice data subject to voiceprint matching that are highly relevant to the sample voice data, and returns time information indicating these time intervals to the user.
- the user who has received the presentation can take into consideration the presented content and select the voice data to be subjected to voiceprint matching from the voice data to be matched, and perform listening work or voiceprint matching processing by a voiceprint matching device.
- the user can execute voiceprint matching processing on the selected voice data, which places a greater burden on the user's listening work and the processing load of the voiceprint matching device.
- this voice data is the data that is actually the subject of voiceprint matching
- the voice data to be matched and processed by extraction unit 1b is the voice data for extracting the subject of voiceprint matching. Therefore, the voice data to be matched can also be referred to as extraction subject data or selection subject data.
- the extraction is based on the sample voice data of the subject, an extraction result with little overlap with the speech of others can be obtained.
- the user can then narrow down the time section for voiceprint matching by listening to the results of the time section in which the subject is speaking automatically extracted by the voiceprint matching support system 1.
- this embodiment it is possible to present to the user the section in which a specific speaker is speaking while suppressing overlapping of speech by speakers other than the specific speaker within the speech section. Therefore, according to this embodiment, in tasks that require voiceprint matching, such as voiceprint analysis, the user who is the worker can listen mainly to the proposed section, thereby making the task more efficient, as opposed to listening (listening) throughout the entire process. Therefore, according to this embodiment, it is possible to reduce the manual work involved in extracting users, prevent a decrease in accuracy, and prevent accuracy variations.
- this embodiment has been described on the assumption that there is one person to be the subject of voiceprint matching, there may be multiple people.
- a time period in which the multiple people are speaking simultaneously can be extracted from the voice data to be compared with the voice, and presented to the user.
- FIG. 2 is a block diagram showing another configuration example of the voiceprint matching support system according to the present disclosure.
- the voiceprint matching support system 100 can include a server computer (hereinafter, server) 10, a user terminal 20, a processing target voice acquisition device 30, and a sample voice acquisition device 40.
- server a server computer
- user terminal a user terminal
- processing target voice acquisition device a processing target voice acquisition device
- sample voice acquisition device 40 a sample voice acquisition device
- the server 10 may include a control unit 11, a storage unit 12, and a communication unit 13.
- the control unit 11 is a part that controls the entire server 10, and may be realized by, for example, a processor such as a CPU or GPU, a working memory, and a non-volatile storage device that stores a program.
- This program may be a program for causing the processor to execute processing including part of the input processing of the input unit 1a, the extraction processing of the extraction unit 1b, and the display control processing of the display unit 1c.
- part of the display control processing may refer to processing for instructing the user terminal 20 via the communication unit 13 to display the extracted time interval on the user terminal 20.
- the control unit 11 reads out and executes the above program, and the server 10 may realize functions including part of the input function of the input unit 1a, the extraction function of the extraction unit 1b, and the display control function of the display unit 1c.
- the storage unit 12 can be a storage device that stores sample voice data input from the sample voice acquisition device 40, voice data to be processed (hereinafter, voice data to be processed) input from the processing target voice acquisition device 30, and information indicating the extracted time interval.
- the storage unit 12 can also store thresholds and time intervals to be used in the extraction process.
- the communication unit 13 can be a communication interface that communicates with the user terminal 20, the processing target voice acquisition device 30, and the sample voice acquisition device 40 via a wired or wireless network.
- the user terminal 20 can be an information processing device such as a personal computer or a smartphone, and can include a control unit 21, a memory unit 22, a communication unit 23, and a display unit 24.
- the control unit 21 is a part that controls the entire user terminal 20, and can be realized by, for example, a processor such as a CPU or GPU, a working memory, and a non-volatile storage device that stores a program.
- This program can be a program for causing the processor to execute processes including a process for accessing the server 10 via the communication unit 23, and a part of the display control process of the display unit 1c.
- the above program is read and executed by the control unit 21, and the user terminal 20 can realize the function of performing these processes.
- the part of the display control function can refer to a process of displaying an extracted time period on the display unit 24 in accordance with an instruction received from the server 10 via the communication unit 23.
- the memory unit 22 can be a storage device that stores various settings related to the display of time intervals. These settings can include, for example, display settings in a user interface for displaying time intervals.
- the communication unit 23 can be a communication interface that communicates with the server 10 via a wired or wireless network.
- the processing target voice acquisition device 30 is a device that acquires processing target voice data to be processed by the server 10 and transmits it to the server 10.
- the processing target voice acquisition device 30 can be a telephone system, a network system capable of voice calls, a recording device equipped with a microphone, etc.
- the sample voice acquisition device 40 is a device that acquires sample voice data to be used in the extraction process in the server 10 and transmits it to the server 10, and can be a recording device equipped with a microphone, etc.
- the voiceprint matching support system 100 can store the target voice data acquired by the target voice acquisition device 30 in the server 10, and can also store the sample voice data acquired by the sample voice acquisition device 40 in the server 10.
- the order of storage is not important.
- control unit 11 of the server 10 executes an extraction process to extract a time period during which the subject is speaking from the stored processing target voice data based on the stored sample voice data, as described as a function of the extraction unit 1b.
- the extraction process in embodiment 1 is performed based on sample voice data of the subject, so the extracted voice data extracted from the voice data to be processed includes voice data for the section in which the subject is speaking, and this section is presented to the user.
- voice data for the section in which the subject is conversing with another person may also be extracted.
- the extraction process in this embodiment extracts time periods during which the subject is speaking alone, that is, extracts time periods excluding periods during which others are speaking.
- the control unit 11 also transmits an instruction to the user terminal 20 via the communication unit 13 to display the extracted time interval on the user terminal 20 in order to present it to the user.
- this instruction is received from the server 10 via the communication unit 23, and the control unit 21 controls the display unit 24 to display the time interval, and the time interval is displayed on the display unit 24.
- the user can focus on the time interval suggested for display, making the task of listening (watching) from start to finish more efficient.
- the information indicating the time interval to be displayed is merely a suggestion, and voiceprint matching for this time interval is not performed automatically.
- the user can select the time interval for which voiceprint matching is required from the voiceprint matching for this time interval on the user terminal 20, and have the voiceprint matching performed by a voiceprint matching device (not shown).
- This voiceprint matching device can also be installed in the server 10.
- voice data in sections in which the subject is conversing with others and in which the speech of others is mixed in can be excluded from extraction, and compared to embodiment 1, there is no overlap of other people's voices.
- the user who will be the worker can listen to the proposed section in particular, making the task of listening (watching) the entire process even more efficient than in embodiment 1.
- the control unit 11 can extract a time interval based on a similarity score between the features of the sample voice data and the features of the voice data to be processed.
- Existing methods may be used for the type of feature, the calculation of the feature, and the calculation method of the similarity score.
- the feature is calculated, for example, for each predetermined time interval, the similarity score is also calculated for each predetermined time interval, and the time interval can be extracted in units of the predetermined time interval. With this method, the similarity score will be lowered in sections of the predetermined time interval in the voice data to be processed that contain the voice of another person other than the subject indicated by the features of the sample voice data.
- the control unit 11 may extract, as a time section, a section in the voice data to be processed in which the similarity score is equal to or greater than a predetermined threshold.
- a time section By employing such an extraction process, it is possible to present the user with a time section while excluding sections in which other people's voices are mixed in. This makes it possible to present the user with only information that is highly relevant to the subject (high similarity score), in other words, only information indicating the section necessary for listening, in an easily viewable manner.
- Fig. 4 is a diagram showing an example of the waveform of target voice data to be processed by voiceprint matching support system 100.
- Fig. 5 is a conceptual diagram conceptually showing an example of target voice data to be processed by voiceprint matching support system 100, and is a diagram showing the waveform of the target voice data in Fig. 4 divided for each speaker.
- Fig. 6 is a diagram showing an example of the waveform of sample voice data used in voiceprint matching support system 100.
- Fig. 7 is a conceptual diagram conceptually showing an example of sample voice data to be used in voiceprint matching support system 100, and is a diagram showing the waveform of the sample voice data in Fig. 6.
- the speech data to be processed that is acquired by the speech acquisition device 30 and processed by the server 10 has a waveform such as that shown in FIG. 4.
- the speech data to be processed, whose waveform is shown in FIG. 4, can be divided into one or more speakers and one or more noises according to the features, for example, by frequency analysis or clustering.
- FIG. 5 shows a schematic example of the features for speaker A, speaker B, speaker C, and noise.
- the features are values that indicate various types of features, including volume and sound pressure.
- the sample voice data acquired by the sample voice acquisition device 40 and used in the extraction process in the server 10 has a waveform, for example, as shown in FIG. 6.
- FIG. 6 shows an example of sample voice data of speaker C.
- FIG. 7 shows an example of the feature quantities of the sample voice data of the waveform exemplified in FIG. 6.
- FIG. 5 is merely an illustration to explain an example of the overlapping state of multiple speakers and noise (disturbances), and is merely an illustration to explain that the feature quantities of speaker C shown in FIG. 7 can be extracted.
- Fig. 8 is a diagram showing a display example that displays an extracted time interval for the waveform of the audio data to be processed in Fig. 4.
- Figs. 9 to 11 are diagrams showing other display examples that display an extracted time interval for the waveform of the audio data to be processed in Fig. 4.
- the server 10 can instruct the display unit 24 to display the extracted time interval in a display format different from other intervals while displaying the waveform of the speech data to be processed, including a time axis corresponding to the speech data to be processed, as exemplified in FIG. 8.
- the display unit 24 can then execute such a display.
- rectangular areas cs1 and cs2 are shown on the waveform, indicating the calculation target time when the similarity score is equal to or greater than a predetermined threshold.
- the similarity score is a value in the range of 0 to 100, and the predetermined threshold is set to 70.
- the rectangular areas cs1 and cs2 indicating the calculation target time for the score value correspond to the features c1 and c2, respectively, among the features of speaker C in FIG. 5.
- the display form is not limited to this, and the extracted time interval can be displayed in a highlighted manner, or the higher the similarity score, the darker the gradation or color can be displayed.
- the similarity score is divided into a number of threshold ranges, and each range is displayed in a different display form. This makes it possible to display highly relevant time intervals in a ranked form based on the similarity score, making it easier for the user to visually recognize the level of relevance. The ranked number can also be displayed together.
- the feature quantities exemplified in Figure 5 can be displayed instead of the waveforms in Figure 4.
- a waveform including a time axis is displayed
- rectangular areas cs1 and cs2 indicating the calculation target time for which the similarity score is equal to or greater than a predetermined threshold can be displayed on the time axis.
- the magnitude of the similarity score (height indicating the value) and the rectangular areas cs1 and cs2 indicating the calculation target time for the similarity score are areas corresponding to the features c1 and c2 of speaker C's features in FIG. 5, respectively.
- the display example of FIG. 8 also shows the value of the similarity score, and corresponds to the following processing. That is, the server 10 can also instruct the display unit 24 to visualize and display the similarity score for at least the extracted time section, while displaying the waveform of the audio data to be processed, including the time axis corresponding to the audio data to be processed. The display unit 24 can then execute such a display.
- visualization of the similarity score may mean visualizing the value of the similarity score itself, as in the example of FIG. 8, or visualizing the magnitude of the similarity score value, as in the example of FIG. 9.
- the server 10 can also cause the display unit 24 to display the density of similarity scores for at least the extracted time section while displaying only the waveform or time axis of the audio data to be processed, including the time axis corresponding to the audio data to be processed, as exemplified in FIG. 10.
- the server 10 can instruct the display unit 24 to visualize and display the density of sections with high similarity scores.
- a section with a high similarity score can refer to, for example, a section that is higher than another predetermined threshold value.
- the other predetermined threshold value can also be set lower than the above predetermined threshold value.
- the display unit 24 can then execute such a display.
- the sparse/dense state is displayed with no hatching when sparse and with hatching when dense, with the density increasing as the density increases.
- the example of Figure 10 shows an example in which the above-mentioned specified time interval serving as the extraction unit is shorter than the example of Figure 8.
- the display of the sparse/dense state is not limited to this example, and it is sufficient if it is possible to visually see where areas with high similarity scores are concentrated, and the sparse/dense state may be displayed in a so-called one-dimensional heat map or a similar display format.
- the server 10 can also be equipped with an interval setting unit that sets the time interval (the above-mentioned specified time interval) that is the extraction unit in the extraction process.
- This interval setting unit can be installed, for example, by storing an interval setting program that executes such settings in the storage unit 12 in a state that is executable by the control unit 11. This allows the user to select the section to listen to while changing the time interval and changing the proposal content.
- the setting of such a time interval can be accepted by a user interface (UI).
- UI user interface
- the server 10 either spontaneously or upon request from the user terminal 20, can present a UI 50 as exemplified in FIG. 11 to the user terminal 20 and display it on the display unit 24.
- the user can input an interval in an input field 55 for an extracted time interval, and a graph 58 including information indicating the time interval displayed in the area below will be changed and displayed based on the input time interval.
- the graph 58 shows the example in Figure 8, but it is also possible to display other graphs, such as the graphs shown in Figures 9 and 10, and it is also possible to select the graph type.
- the UI 50 includes an input area 51, which can include an input field 55.
- the input area 51 can also include, for example, a delete low score area button 52, a delete but leave selected area button 53, a delete selected area button 54, an extraction threshold input field 56, and a create voiceprint matching target file button 57.
- the server 10 may also be provided with a threshold setting unit that sets the above-mentioned predetermined threshold.
- This threshold setting unit may be installed, for example, by storing a threshold setting program that executes such settings in the storage unit 12 in a state that is executable by the control unit 11.
- Such settings of predetermined thresholds may also be accepted by the UI.
- the server 10 may present a UI 50 including an extraction threshold input field 56 as exemplified in FIG. 11 to the user terminal 20, either on its own initiative or upon request from the user terminal 20, and display it on the display unit 24.
- the graph 58 displayed in the area below it is changed and displayed based on the input extraction threshold.
- the server 10 can also include a deletion unit that deletes data for sections other than the extracted time section from the audio data to be processed.
- This deletion unit can be implemented as one function of the control unit 11, and can be implemented, for example, by storing a deletion program that executes such deletion in a state that the control unit 11 can execute. Such deletions can also be accepted by the UI.
- the server 10 can present a UI 50 including at least one of buttons 52-54 as exemplified in FIG. 11 to the user terminal 20 and display it on the display unit 24.
- buttons 52-54 By selecting one of buttons 52-54 on the UI 50 displayed on the display unit 24, the user can delete the area indicated by the button from the speech data to be processed.
- button 52 when button 52 is selected, data corresponding to a similarity score below a predetermined threshold is deleted from the speech data to be processed.
- Buttons 53 and 54 can be made selectable by selecting a time region on graph 58 in advance. It is also preferable to configure the system so that it is possible to select a time interval such as regions cs1, cs2, etc. when making this selection.
- the voiceprint matching file creation button 57 can be selected to store the data resulting from removing such areas from the voice data to be processed as a file to be compared against voiceprints.
- the storage destination can be at least one of the memory unit 12 of the server 10 and the memory unit 22 of the user terminal 20, but it can also be another storage destination, and the storage destination can also be selectable on the UI 50.
- Figure 12 is a flow diagram for explaining the work of creating a file to be compared with voiceprints in Comparative Example 1
- Figure 13 is a flow diagram for explaining the work of creating a file to be compared with voiceprints in Comparative Example 2.
- FIG. 12 shows the steps for creating a file for voiceprint matching from the audio data to be processed.
- the user plays the audio data to be processed (step S11) and listens to it.
- the audio data to be processed is transcribed into text for some or all of it and then saved.
- the user determines whether the audio data to be processed contains reference (exists) to transcription data that clearly identifies the target person (step S12), and if not, discards the audio data (step S18) and ends the process. If the answer is YES in step S12, the user listens to only the section where the target person is speaking and divides the waveform (step S13).
- step S14 determines whether the subject's voice overlaps with other people's voices or noise in the recorded data after processing in step S13 (step S14), and if so, discards the overlapping sections in the recorded data (step S15). This results in the recorded data after processing in step S13 being partially deleted.
- step S16 it is determined whether the duration of the subject's voice is equal to or greater than a specified value, such as 5 seconds (step S16). If the answer is NO in step S14, the process proceeds to step S16 without passing through step S15. If the answer is YES in step S16, the remaining recorded data up to that point is registered as a voiceprint matching target file (step S17), and the process ends. On the other hand, if the answer is NO in step S16, the remaining recorded data up to that point is registered as a part file so that it is equal to or greater than the specified value (step S19), and the process ends.
- a specified value such as 5 seconds
- Comparative Example 2 shown in Figure 13 shows the steps for creating a file for voiceprint matching from video data that includes audio recording data.
- the user plays the video data to be processed (step S21) and watches it.
- video is present, so the user checks the video and determines whether it is clear that the subject is in the video data to be processed (step S22). If it is not clear, the video data is excluded from processing (step S28) and the work ends. If the answer is YES in step S22, the user listens only to the section where the subject is speaking and performs the work of dividing the waveform (step S23).
- step S24 judges whether the voice of the subject overlaps with other voices or noise in the video data after the processing of step S23 (step S24), and if so, discards the overlapping section in the video data (step S25). This results in the video data after the processing of step S23 being partially deleted.
- step S26 it is judged whether the duration of the subject's voice is equal to or longer than a specified value, such as 5 seconds (step S26). If the answer is NO in step S24, the process proceeds to step S26 without passing through step S25. If the answer is YES in step S26, the voice data of the video data remaining up to that point is registered as a voiceprint matching target file (step S27), and the process ends.
- step S26 the video data remaining up to that point is treated as non-processed data (step S28), and the process ends. If the answer is NO in step S26, the video data remaining up to that point may be registered as a part file so that it is equal to or longer than the specified value, as in step S19, and then the process ends.
- FIG. 14 is a diagram showing an example of the hardware configuration included in the device.
- the device 1000 shown in FIG. 14 includes a processor 1001, a memory 1002, and a communication interface 1003.
- the functions of each device can be realized by the processor 1001 reading a program stored in the memory 1002 and executing it in cooperation with the communication interface 1003.
- the above-mentioned program includes instructions (or software code) that, when loaded into a computer, cause the computer to perform one or more functions described in the embodiments.
- the program may be stored on a non-transitory computer-readable medium or a tangible storage medium.
- computer-readable medium or tangible storage medium includes random-access memory (RAM), read-only memory (ROM), flash memory, solid-state drive (SSD) or other memory technology, CD-ROM, digital versatile disk (DVD), Blu-ray® disk or other optical disk storage, magnetic cassette, magnetic tape, magnetic disk storage or other magnetic storage device.
- the program may be transmitted on a transitory computer-readable medium or a communication medium.
- the transitory computer-readable medium or communication medium includes electrical, optical, acoustic, or other forms of propagated signals.
- (Appendix 1) an input unit for inputting sample voice data of a person to be subjected to voiceprint matching; an extraction unit that extracts a time period during which the subject is speaking from the voice data to be compared with the voiceprint based on the sample voice data; a display unit that presents the time interval extracted by the extraction unit to a user;
- a voiceprint matching support system (Appendix 2) The extraction unit extracts, as the time period, a time period during which the subject is speaking alone. 2.
- a voiceprint matching support system as described in claim 1. (Appendix 3) a deletion unit that deletes data of a section other than the time section extracted by the extraction unit from the voice data to be voiceprint matched, 3.
- the extraction unit extracts the time interval based on a similarity score between a feature amount of the sample voice data and a feature amount of the voice print matching target voice data; 4.
- a voiceprint matching support system according to any one of claims 1 to 3.
- the extraction unit extracts, as the time section, a section in which the similarity score in the voice data to be voiceprint matched is equal to or greater than a predetermined threshold value; 5.
- a voiceprint matching support system as described in claim 4. (Appendix 6) A threshold setting unit that sets the predetermined threshold, 6.
- the display unit visualizes and displays the similarity score for at least the time period extracted by the extraction unit while displaying a time axis corresponding to the voice data to be compared with the voice print comparison or a waveform of the voice data to be compared with the voice print comparison including the time axis.
- a voiceprint matching support system according to any one of claims 4 to 6.
- the display unit while displaying a time axis corresponding to the voice data to be compared with the voice print, or a waveform of the voice data to be compared with the time axis, visualizes and displays a sparse/dense state of a section having a high similarity score for at least the time section extracted by the extraction unit.
- a voiceprint matching support system according to any one of appendix 4 to 7.
- said display unit displays the time interval extracted by said extraction unit in a display form different from other intervals while displaying a time axis corresponding to said voice data to be compared with the voice print comparison target voice data or a waveform of said voice data to be compared with the time axis.
- a voiceprint matching support system according to any one of appendix 1 to 8.
- An interval setting unit that sets a time interval that is an extraction unit in the extraction unit, 10.
- a voiceprint matching support system according to any one of claims 1 to 9.
- a voiceprint matching assistance method comprising: (Appendix 12) The extraction process extracts, as the time period, a time period during which the subject is speaking alone. 12.
- (Appendix 13) a process of deleting data of a section other than the time section extracted in the extraction process from the voice data to be voiceprint matched, 13.
- the extraction process is performed based on a similarity score between the feature amount of the sample voice data and the feature amount of the voice data to be compared with the voiceprint.
- a voiceprint matching support method according to any one of claims 11 to 13.
- the extraction process extracts, as the time section, a section in which the similarity score in the voice data to be compared is equal to or greater than a predetermined threshold value; 15.
- a method for assisting voiceprint matching as described in claim 14. (Appendix 16) setting the predetermined threshold value; 16.
- the display process includes a process of visualizing and displaying the similarity score for at least the time period extracted in the extraction process, in a state in which a time axis corresponding to the voice data to be compared with the voice print comparison or a waveform of the voice data to be compared with the voice print comparison including the time axis is displayed on the display device, 17.
- a voiceprint matching support method according to any one of claims 14 to 16.
- the display process includes a process of visualizing and displaying, on the display device, a sparse/dense state of a section having a high similarity score for at least the time section extracted in the extraction process, in a state in which a time axis corresponding to the voice data to be compared with the voice print comparison or a waveform of the voice data to be compared with the voice print comparison including the time axis.
- a voiceprint matching support method according to any one of appendices 14 to 17.
- the display process includes a process of displaying, on the display device, the time interval extracted in the extraction process in a display form different from other intervals, while displaying a time axis corresponding to the voice data to be compared with the voice data to be compared with the time axis or a waveform of the voice data to be compared with the voice data to be compared with the time axis.
- a voiceprint matching support method according to any one of appendices 11 to 18.
- Appendix 20 A process of setting a time interval that is an extraction unit in the extraction process.
- a voiceprint matching support method according to any one of appendices 11 to 19.
- an input process for inputting sample voice data of a person to be subjected to voiceprint matching On the computer, an input process for inputting sample voice data of a person to be subjected to voiceprint matching; an extraction process for extracting a time period during which the subject is speaking from the voice data to be subjected to voiceprint matching based on the sample voice data; a display process for displaying the extracted time interval on a display device for presentation to a user;
- a program for executing a voiceprint matching support process including: (Appendix 22) The extraction process extracts, as the time period, a time period during which the subject is speaking alone. 22.
- the voiceprint matching support process includes a process of deleting data of a section other than the time section extracted in the extraction process from the voiceprint matching target voice data, 23.
- the program according to claim 21 or 22. the extraction process is performed based on a similarity score between the feature amount of the sample voice data and the feature amount of the voice data to be compared with the voiceprint. 24.
- the program according to any one of claims 21 to 23. the extraction process extracts, as the time section, a section in which the similarity score in the voice data to be compared is equal to or greater than a predetermined threshold value; 25.
- the voiceprint matching support process includes a process of setting the predetermined threshold value, 26. The program according to claim 25.
- the display process includes a process of visualizing and displaying the similarity score for at least the time period extracted in the extraction process, in a state in which a time axis corresponding to the voice data to be compared with the voice print comparison or a waveform of the voice data to be compared with the voice print comparison including the time axis is displayed on the display device, 27.
- the program according to any one of claims 24 to 26.
- the display process includes a process of visualizing and displaying, on the display device, a sparse/dense state of a section having a high similarity score for at least the time section extracted in the extraction process, in a state in which a time axis corresponding to the voice data to be compared with the voice print comparison or a waveform of the voice data to be compared with the voice print comparison including the time axis.
- the display process includes a process of displaying, on the display device, the time interval extracted in the extraction process in a display form different from other intervals, while displaying a time axis corresponding to the voice data to be compared with the voice data to be compared with the time axis or a waveform of the voice data to be compared with the voice data to be compared with the time axis. 29.
- the voiceprint matching support process includes a process of setting a time interval that is an extraction unit in the extraction process, 30.
- Reference Signs List 1 100 Voiceprint matching support system 1a Input unit 1b Extraction unit 1c Display unit 10 Server 11 Control unit 12 Storage unit 13 Communication unit 20 User terminal 21 Control unit 22 Storage unit 23 Communication unit 24 Display unit 1000 Device 1001 Processor 1002 Memory 1003 Communication interface
Landscapes
- Engineering & Computer Science (AREA)
- Multimedia (AREA)
- Audiology, Speech & Language Pathology (AREA)
- Human Computer Interaction (AREA)
- Physics & Mathematics (AREA)
- Acoustics & Sound (AREA)
- Health & Medical Sciences (AREA)
- Computer Vision & Pattern Recognition (AREA)
- Business, Economics & Management (AREA)
- Game Theory and Decision Science (AREA)
- Computational Linguistics (AREA)
- Telephonic Communication Services (AREA)
- User Interface Of Digital Computer (AREA)
Priority Applications (2)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| US19/132,454 US20260196224A1 (en) | 2022-12-05 | 2023-11-16 | Voiceprint matching support system, voiceprint matching support method, and program |
| JP2024562653A JPWO2024122304A1 (cg-RX-API-DMAC7.html) | 2022-12-05 | 2023-11-16 |
Applications Claiming Priority (2)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| JP2022194509 | 2022-12-05 | ||
| JP2022-194509 | 2022-12-05 |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| WO2024122304A1 true WO2024122304A1 (ja) | 2024-06-13 |
Family
ID=91379107
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| PCT/JP2023/041303 Ceased WO2024122304A1 (ja) | 2022-12-05 | 2023-11-16 | 声紋照合支援システム、声紋照合支援方法、及びプログラム |
Country Status (3)
| Country | Link |
|---|---|
| US (1) | US20260196224A1 (cg-RX-API-DMAC7.html) |
| JP (1) | JPWO2024122304A1 (cg-RX-API-DMAC7.html) |
| WO (1) | WO2024122304A1 (cg-RX-API-DMAC7.html) |
Citations (6)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| JP2005321530A (ja) * | 2004-05-07 | 2005-11-17 | Sony Corp | 発話識別装置および発話識別方法 |
| JP2005326576A (ja) * | 2004-05-13 | 2005-11-24 | Sony Corp | 音声処理装置 |
| JP2008032825A (ja) * | 2006-07-26 | 2008-02-14 | Fujitsu Fsas Inc | 発言者表示システム、発言者表示方法および発言者表示プログラム |
| JP2010113167A (ja) * | 2008-11-06 | 2010-05-20 | Neikusu:Kk | 有害顧客検知システム、その方法及び有害顧客検知プログラム |
| WO2020003413A1 (ja) * | 2018-06-27 | 2020-01-02 | 日本電気株式会社 | 情報処理装置、制御方法、及びプログラム |
| US20210326421A1 (en) * | 2020-04-15 | 2021-10-21 | Pindrop Security, Inc. | Passive and continuous multi-speaker voice biometrics |
-
2023
- 2023-11-16 JP JP2024562653A patent/JPWO2024122304A1/ja active Pending
- 2023-11-16 US US19/132,454 patent/US20260196224A1/en active Pending
- 2023-11-16 WO PCT/JP2023/041303 patent/WO2024122304A1/ja not_active Ceased
Patent Citations (6)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| JP2005321530A (ja) * | 2004-05-07 | 2005-11-17 | Sony Corp | 発話識別装置および発話識別方法 |
| JP2005326576A (ja) * | 2004-05-13 | 2005-11-24 | Sony Corp | 音声処理装置 |
| JP2008032825A (ja) * | 2006-07-26 | 2008-02-14 | Fujitsu Fsas Inc | 発言者表示システム、発言者表示方法および発言者表示プログラム |
| JP2010113167A (ja) * | 2008-11-06 | 2010-05-20 | Neikusu:Kk | 有害顧客検知システム、その方法及び有害顧客検知プログラム |
| WO2020003413A1 (ja) * | 2018-06-27 | 2020-01-02 | 日本電気株式会社 | 情報処理装置、制御方法、及びプログラム |
| US20210326421A1 (en) * | 2020-04-15 | 2021-10-21 | Pindrop Security, Inc. | Passive and continuous multi-speaker voice biometrics |
Also Published As
| Publication number | Publication date |
|---|---|
| US20260196224A1 (en) | 2026-07-09 |
| JPWO2024122304A1 (cg-RX-API-DMAC7.html) | 2024-06-13 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN111541904B (zh) | 直播过程中的信息提示方法、装置、设备及存储介质 | |
| US10657945B2 (en) | Noise control method and device | |
| JP6969576B2 (ja) | 情報処理装置、および情報処理方法 | |
| US10733970B2 (en) | Noise control method and device | |
| US20160247533A1 (en) | Audio recording apparatus, audio recording method, and non-transitory recording medium | |
| US9196253B2 (en) | Information processing apparatus for associating speaker identification information to speech data | |
| JP7208564B2 (ja) | 応対履歴作成支援装置、応対履歴作成支援方法、プログラム、及び表示装置 | |
| US12496004B2 (en) | Control method, conference system, and non-transitory recording medium | |
| JP2016102920A (ja) | 文書記録システム及び文書記録プログラム | |
| KR102797767B1 (ko) | 장면 설명의 재생 제어 | |
| US10216732B2 (en) | Information presentation method, non-transitory recording medium storing thereon computer program, and information presentation system | |
| US20170068512A1 (en) | Electronic apparatus and information processing method thereof | |
| JP6739811B2 (ja) | 発言禁止用語に対し注意を喚起するためのプレゼンテーション支援装置 | |
| JPWO2015019662A1 (ja) | 分析対象決定装置及び分析対象決定方法 | |
| US9412380B2 (en) | Method for processing data and electronic device thereof | |
| US20220084505A1 (en) | Communication between devices in close proximity to improve voice control of the devices | |
| JP2022114906A (ja) | 心理状態管理装置 | |
| KR101933822B1 (ko) | 얼굴인식 기반 지능형 스피커, 이를 이용한 능동적인 대화 제공 방법 및 이를 수행하기 위한 기록매체 | |
| JP2001272990A (ja) | 対話記録編集装置 | |
| US20240282312A1 (en) | Voice processing device, method, and recording medium | |
| US11755652B2 (en) | Information-processing device and information-processing method | |
| JP2008032825A (ja) | 発言者表示システム、発言者表示方法および発言者表示プログラム | |
| JP2011151497A (ja) | 電話応答結果予測装置、方法、およびそのプログラム | |
| WO2020116001A1 (ja) | 情報処理装置および情報処理方法 | |
| JP6838739B2 (ja) | 近時記憶支援装置 |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| 121 | Ep: the epo has been informed by wipo that ep was designated in this application |
Ref document number: 23900425 Country of ref document: EP Kind code of ref document: A1 |
|
| WWE | Wipo information: entry into national phase |
Ref document number: 2024562653 Country of ref document: JP |
|
| NENP | Non-entry into the national phase |
Ref country code: DE |
|
| 122 | Ep: pct application non-entry in european phase |
Ref document number: 23900425 Country of ref document: EP Kind code of ref document: A1 |