KR102299269B1 - Method and apparatus for building voice database by aligning voice and script - Google Patents

Abstract

The present invention relates to a voice database building method by aligning voice and script and an apparatus thereof. According to an embodiment, a voice signal and the script are aligned in a sentence unit based on first similarity between a first voice recognition sentence corresponding to part of the voice signal and a first input sentence corresponding to part of the script, second similarity between a second voice recognition sentence connecting a next sentence of the first voice recognition sentence to the first voice recognition sentence and the first input sentence, and third similarity between a second input sentence connecting a next sentence of the first input sentence to the first input sentence and the first voice recognition sentence.

Description

Method and apparatus for building voice database by aligning voice and script

The embodiments below relate to a method and apparatus for building a voice database by sorting voices and scripts.

Speech processing technology is a technology in which a computer automatically understands and processes or generates a human natural language utterance, and includes a speech recognition technology and a speech synthesis technology. Speech recognition technology refers to a technology that converts an acoustic signal, such as a voice spoken by a human, into text data through a computer, and is also referred to as speech-to-text (STT). A speech recognition technology that converts an audio signal, which is an analog signal, into a digital signal that can be processed by a computer can be used in various fields. For example, a voice recognition assistant function that controls a device by recognizing a person's voice command is installed and utilized in various devices such as a mobile phone, a speaker, and a home appliance. In general, the speech recognition model may include an acoustic model (AM) and a language model (LM). The acoustic model is a model for predicting the probability that a specific word text corresponds to a specific voice, and the language model is a model for predicting the probability that a specific word text matches the current position of a sentence regardless of the voice. Using probabilities according to the two models, when specific voice data is input, voice recognition for recognizing a corresponding word text may be performed. In this case, a speech recognition model may be built using a large-capacity text database.

Speech synthesis technology is a technology in which a machine automatically generates sound waves of speech sounds corresponding to text input, and is also called text-to-speech (TTS). Speech synthesis technology can be used in various fields such as artificial intelligence assistants, chatbots, public transportation guidance voices, software that reads text for people who have difficulty reading text (screen readers), and language learning.

In order to obtain good performance in various voice processing models such as voice preprocessing, voice recognition, and voice synthesis, it is necessary to build a database containing a large volume of voices and sentences uttered by various speakers and environments, A technique for securing a large amount of raw speech and sentence data as a database for speech processing modeling is required.

Embodiments may provide a technique for constructing a voice database used for a voice processing model by segmenting and arranging a raw script and a voice signal uttering the script in sentence units.

In addition, the embodiments may provide a technique for detecting an error in which the script and the voice signal do not match in the process of arranging the script and the voice signal uttering the script, and building a database by correcting the data detected as an error.

A method of constructing a voice database according to one aspect includes: obtaining a first similarity between a first voice recognition sentence corresponding to a part of a voice signal and a first input sentence corresponding to a part of a script of the voice signal; obtaining a second similarity between a second speech recognition sentence in which a next sentence of the first speech recognition sentence is connected to the first speech recognition sentence and the first input sentence; obtaining a third similarity between a second input sentence in which a next sentence of the first input sentence is connected to the first input sentence and the first speech recognition sentence; and arranging the voice signal and the script in units of sentences based on the first degree of similarity, the second degree of similarity, and the third degree of similarity.

The arranging may include, when the first similarity is the highest, storing a first voice signal corresponding to the first voice recognition sentence and the first input sentence in a voice database; when the second similarity is the highest, updating the first speech recognition sentence to the second speech recognition sentence and transferring it to a next iteration; and when the third similarity is the highest, updating the first input sentence to the second input sentence and transferring it to a next iteration.

The aligning may include: determining whether to update the first speech recognition sentence or the first input sentence based on the first degree of similarity, the second degree of similarity, and the third degree of similarity; When it is determined to update the first speech recognition sentence or the first input sentence, the first speech recognition sentence or the updated first input sentence is based on the first similarity, the second similarity, and the third similarity. repeatedly performing the step of determining whether to update; and storing, in the database, a first voice signal corresponding to the first voice recognition sentence and the first input sentence in response to determining not to update the first voice recognition sentence and the first input sentence. can

Repeatedly performing the step of determining whether to update may include, when it is determined to update the first speech recognition sentence, based on the updated first speech recognition sentence and the first input sentence, the first similarity, the second obtaining a second degree of similarity and a third degree of similarity; and when it is determined to update the first input sentence, acquiring a first degree of similarity, a second degree of similarity, and a third degree of similarity based on the updated first input sentence and the first speech recognition sentence. have.

A voice database construction method according to one side comprises the steps of segmenting the voice signal into sentence units; converting the speech signal into text based on speech recognition; and segmenting the script into sentence units.

A voice database construction method according to one side comprises the steps of: receiving the voice signal and the script; pre-processing the received voice signal; and pre-processing the received script.

The first degree of similarity, the second degree of similarity, and the third degree of similarity may be obtained based on an editing distance.

The aligning may include modifying at least one of the first voice signal, the first voice recognition sentence, and the first input sentence based on the first similarity according to a predetermined criterion.

The arranging may include modifying at least one of the first voice signal, the first voice recognition sentence, and the first input sentence based on the number of connections corresponding to the number of repetitions of the iteration. .

The method of constructing a voice database according to an aspect may further include displaying a result of obtaining at least one of the first similarity, the second similarity, and the third similarity through an interface.

The step of storing in the database may include: forced speech sorting of the first speech signal and the first input sentence; and segmenting and storing the first voice signal and the first input sentence based on the forced voice alignment result.

The method for constructing a voice database may further include, after acquiring the first similarity, determining whether to acquire a second degree of similarity and whether to acquire a third degree of similarity based on the first similarity. In this case, the step of obtaining the second degree of similarity includes obtaining the second degree of similarity based on the determination, and the step of obtaining the third degree of similarity includes obtaining the third degree of similarity based on the determination. may include steps.

The determining of whether to obtain the second degree of similarity and whether to obtain the third degree of similarity includes determining that the second degree of similarity and the third degree of similarity are not obtained when the first degree of similarity corresponds to a predetermined criterion; ; and when the first degree of similarity does not correspond to a predetermined criterion, determining to obtain the second degree of similarity and the third degree of similarity.

The acquiring of the first similarity includes acquiring the number of characters to be deleted, the number of characters to be inserted, and the number of characters to be replaced in order to change the first speech recognition sentence into the first input sentence. ; and obtaining an edit distance between the first speech recognition sentence and the first input sentence based on the number of characters to be deleted, the number of characters to be inserted, and the number of characters to be replaced. .

The determining of whether to obtain the second degree of similarity and whether to obtain the third degree of similarity may include any one of the second degree of similarity and the third degree of similarity based on the number of characters to be deleted and the number of characters to be inserted. determining to obtain one.

The first similarity includes a degree of similarity between the first speech recognition sentence converted into a phoneme sequence and the first input sentence converted into a phoneme sequence, and the second similarity includes the second speech recognition sentence converted into a phoneme sequence and a phoneme sequence. may include a degree of similarity between the converted first input sentences, and the third similarity may include a degree of similarity between the first speech recognition sentence converted into a phoneme sequence and the second input sentence converted into a phoneme sequence.

The apparatus for constructing a speech database according to one side obtains a first similarity between a first speech recognition sentence corresponding to a part of a voice signal and a first input sentence corresponding to a part of a script of the voice signal, and the first voice recognition A second speech recognition sentence in which a next sentence of a sentence is connected to the first speech recognition sentence and a second degree of similarity between the first input sentence are obtained, and the next sentence of the first input sentence is connected to the first input sentence obtaining a third degree of similarity between a second input sentence and the first speech recognition sentence, and arranging the voice signal and the script in sentence units based on the first similarity, the second similarity, and the third similarity at least one processor; and a voice database for storing the voice signal and the script arranged in units of the sentences.

When the processor aligns the voice signal and the script in sentence units, when the first similarity is the highest, the processor stores the first voice signal corresponding to the first voice recognition sentence and the first input sentence in the voice database. stored, and when the second similarity is the highest, the first speech recognition sentence is updated to the second speech recognition sentence and transmitted to the next iteration, and when the third similarity is the highest, the first input sentence is The second input sentence may be updated and transmitted to the next iteration.

When the processor aligns the voice signal and the script in sentence units, based on the first similarity, the second similarity, and the third similarity, whether the first voice recognition sentence or the first input sentence is updated and determining to update the first speech recognition sentence or the first input sentence, the first similarity, the second similarity, and the third similarity based on the updated first speech recognition sentence or the updated first input sentence repeatedly performing the step of determining whether to update the first speech recognition sentence based on The first input sentence may be stored in the database.

When the processor repeatedly performs the step of determining whether to update the first voice recognition sentence, when it is determined to update the first voice recognition sentence, based on the updated first voice recognition sentence and the first input sentence, the first Acquire similarity, second similarity, and third similarity, and when it is determined to update the first input sentence, based on the updated first input sentence and the first speech recognition sentence, the first similarity and second similarity and a third degree of similarity may be obtained.

The processor may segment the voice signal into sentence units, convert the voice signal into text based on voice recognition, and segment the script into sentence units.

The processor may receive the voice signal and the script, pre-process the received voice signal, and pre-process the received script.

When the processor aligns the voice signal and the script in sentence units, based on the first similarity according to a predetermined criterion, at least one of the first voice signal, the first voice recognition sentence, and the first input sentence You can edit one.

The processor may display a result of obtaining at least one of the first degree of similarity, the second degree of similarity, and the third degree of similarity through an interface.

When the processor stores the first voice signal and the first input sentence corresponding to the first voice recognition sentence in the database, the first voice signal and the first input sentence are forcibly voice-aligned, and the forced voice Based on the alignment result, the first voice signal and the first input sentence may be segmented and stored.

After obtaining the first degree of similarity, the processor determines whether to obtain a second degree of similarity and whether to obtain a third degree of similarity based on the first degree of similarity, and based on the determination, the second degree of similarity and the third degree of similarity can be obtained.

In determining whether to obtain the second degree of similarity and whether to obtain the third degree of similarity, the processor determines that the second degree of similarity and the third degree of similarity are not obtained when the first degree of similarity corresponds to a predetermined criterion. is determined, and when the first degree of similarity does not correspond to a predetermined criterion, it may be determined that the second degree of similarity and the third degree of similarity are acquired.

By segmenting and arranging unprocessed speech signals and script data in units of paragraphs or more in units of sentences to construct a voice database, it is possible to increase the yield of the constructed database.

Based on the similarity between the speech recognition sentence of the voice signal and the sentence included in the script, an inconsistency between the voice signal and the script is detected. You can separate the data with

By building a voice database with the input script data without building a large-capacity database in advance, it is possible to reduce the recognition error of the input script data.

1 is a diagram illustrating a voice database building system according to an embodiment.
FIG. 2 is a view for explaining various methods of acquiring input voice signals and scripts for constructing a voice database according to an embodiment; FIG.
3A and 3B are diagrams illustrating an operation flowchart of an alignment algorithm according to an embodiment;
4 to 6 are diagrams illustrating examples for specifically explaining an alignment algorithm according to an embodiment.
7 is a view for explaining an operation of detecting and correcting an error in the voice database building system according to an embodiment;

Hereinafter, embodiments will be described in detail with reference to the accompanying drawings. However, since various changes may be made to the embodiments, the scope of the patent application is not limited or limited by these embodiments. It should be understood that all modifications, equivalents and substitutes for the embodiments are included in the scope of the rights.

The terms used in the examples are used for description purposes only, and should not be construed as limiting. The singular expression includes the plural expression unless the context clearly dictates otherwise. In the present specification, terms such as “comprise” or “have” are intended to designate that a feature, number, step, operation, component, part, or combination thereof described in the specification exists, but one or more other features It should be understood that this does not preclude the existence or addition of numbers, steps, operations, components, parts, or combinations thereof.

Unless defined otherwise, all terms used herein, including technical or scientific terms, have the same meaning as commonly understood by one of ordinary skill in the art to which the embodiment belongs. Terms such as those defined in commonly used dictionaries should be interpreted as having a meaning consistent with the meaning in the context of the related art, and should not be interpreted in an ideal or excessively formal meaning unless explicitly defined in the present application. does not

In addition, in the description with reference to the accompanying drawings, the same components are given the same reference numerals regardless of the reference numerals, and the overlapping description thereof will be omitted. In the description of the embodiment, if it is determined that a detailed description of a related known technology may unnecessarily obscure the gist of the embodiment, the detailed description thereof will be omitted.

In addition, in describing the components of the embodiment, terms such as first, second, A, B, (a), (b), etc. may be used. These terms are only for distinguishing the component from other components, and the essence, order, or order of the component is not limited by the term. When it is described that a component is “connected”, “coupled” or “connected” to another component, the component may be directly connected or connected to the other component, but between each component another component It will be understood that may also be "connected", "coupled" or "connected".

Components included in one embodiment and components having a common function will be described using the same names in other embodiments. Unless otherwise stated, a description described in one embodiment may be applied to another embodiment, and a detailed description in the overlapping range will be omitted.

1 is a diagram illustrating a voice database building system according to an embodiment.

Referring to FIG. 1 , the voice database building system according to an embodiment includes a segmented voice list 103 of a received voice signal 101, and a segmented recognition sentence list 104 obtained by recognizing the segmented voice signal. , and based on the segmented input sentence list 105 of the received script 102 , a module 150 for arranging the voice signal and the script in sentence units. In addition, the voice signal and the script according to an embodiment may be sorted by the sentence unit 150 and stored in the voice database 106 . A voice database constructed in the system according to an embodiment may be used for a voice processing model. Hereinafter, a voice database used for voice processing technology may be briefly referred to as a DB.

The voice signal 101 corresponds to a sound signal through which various speakers utter natural language, and the script 102 corresponds to text data corresponding to the voice signal. In order to use the voice signal 101 and the script 102 as a database in the voice processing model, it is necessary to go through a processing process such as segmentation and sorting in sentence units.

Referring to FIG. 2 , a voice signal 101 and a script 102 input for constructing a voice database may be acquired online and/or offline through various routes and various methods. For example, voice signals and scripts may be obtained through web crawling, multimedia content capture, and the like.

Since the speech signal and script data obtained in this way can be obtained in units of paragraphs or more, it is necessary to segment the speech signal and script into specific units in order to be used as a DB in the speech processing model. For example, a voice signal and a script may be segmented into sentence units and used as a voice database for a voice processing model. Hereinafter, a unit obtained by segmenting a voice signal and a script will be described as a sentence unit, but a sentence here refers to a segmented unit of a voice signal or a script and does not necessarily mean a sentence according to a linguistic definition.

According to an embodiment, in order to align the voice signal and the script in sentence units, in the case of the voice signal, a process of converting the voice signal into text through voice recognition is required along with segmentation processing of the voice signal. Hereinafter, the speech recognition sentence or the recognition sentence means text corresponding to the speech signal segmented in sentence units.

In other words, the voice signal 101 input to the voice database building system according to an embodiment may be segmented into a specific unit by the voice segmentation and voice recognition module 130, and may be converted into text through voice recognition. . The system according to an embodiment may include a module 110 for pre-processing a voice signal input to the voice recognition and voice segmentation module 130 . The pre-processing module 110 of the voice signal may include, for example, an operation of removing elements not necessary for the DB, such as music, background sound, and noise, and performing an operation such as normalization of a sound source standard. Pre-processing, segmentation, and speech recognition of a speech signal may be performed according to any technique. A speech signal according to an embodiment is segmented and speech recognized, and a segmented speech list 103 including parts of the speech signal and a segmented recognition sentence list 104 including texts as a result of speech recognition of parts of the speech signal can be formed. Hereinafter, a case in which segmented speech recognition is performed after speech segmentation will be described. However, embodiments may be substantially the same even when speech recognition is performed simultaneously with speech segmentation or a sentence recognized after speech recognition is segmented. .

The script 102 input to the voice database building system according to an embodiment may be segmented into a specific unit by the script segmentation module 140 , and the system according to an embodiment is a script inputted to the script segmentation module 140 . may include a module 120 for pre-processing. The script pre-processing module 120 may perform, for example, unification of symbols, which are different types included in the script, an operation for improving the performance of a segment, such as removal of unnecessary symbols, and an operation such as sentence encoding normalization. Pre-processing and segmentation of the script may be performed according to any technique. A script according to an embodiment may be segmented to form a segmented input sentence list 105 including parts of the script.

Voice signals and scripts may be segmented by different methodologies due to different data characteristics. For example, a voice signal that is a sound signal may be segmented into a specific unit according to the strength of the sound signal, and a script that is text data may be segmented into a specific unit according to a type of symbol such as a period included in the script. That is, when a voice signal and a script are each segmented according to an automatic segmentation algorithm, segmentation results may be different from each other due to differences in the types of algorithms used and performance of the algorithms. In other words, when a voice signal and a script corresponding thereto are segmented in sentence units according to each automatic segmentation algorithm, the segmentation result of the voice signal and the segmentation result of the script may not be the same. For example, when the speech signal uttering the sentences A, B, and C and the script composed of the sentences A, B, and C are segmented into sentence units, sentences A and B are recognized as one sentence as a result of the segmentation of the voice signal, It may be segmented into A+B and sentence C, and on the other hand, it may be segmented into sentence A, sentence B, and sentence C as a result of segmentation of the script. Alternatively, as a result of segmentation of the voice signal, the voice signal uttering the sentence A may be recognized as the sentence A', and may be segmented into the sentence A', the sentence B, and the sentence C.

The module 150 for arranging the segmentation result of the voice signal and the script may correct the segmentation result so that the segmented voice signal and the segmented script correspond. In other words, the results of the segmented speech signal and the segmented script may be different from each other, and this may be corrected by performing an algorithm for aligning the segmented speech signal and the segmented script. A more specific operation of the alignment module 150 according to an embodiment will be described in detail with reference to FIGS. 3A to 6 below.

According to one embodiment, the segmented speech signal and the segmented script may be sorted and then stored in the database 106 . In addition, related metadata such as a speaker, source, and date of a list corresponding to the segmented voice signal and the segmented script may be stored in the database.

At this time, if the speaker does not read the script exactly as the script or there is an error in the voice signal or the script, the alignment module 150 may detect an error and store the error as a separate list. An error detection operation according to an embodiment will be described in detail with reference to FIG. 7 below.

3A and 3B are diagrams illustrating an operation flowchart of an alignment algorithm according to an exemplary embodiment.

Hereinafter, the first speech recognition sentence is data corresponding to any one of the segmented speech signals, and means text data converted as a result of recognizing any one of the segmented speech signals. That is, it means any one of the sentences included in the segmented recognition sentence list 104 . Also, hereinafter, the first input sentence refers to any one of the segmented scripts, and refers to any one of the input sentences included in the segmented input sentence list 105 .

Referring to FIG. 3A , an alignment algorithm according to an exemplary embodiment includes aligning a voice signal and a script in units of sentences based on a first degree of similarity, a second degree of similarity, and a third degree of similarity. The first similarity is a degree of similarity between the first speech recognition sentence and the first input sentence, and the second similarity is the second speech recognition sentence and the first input sentence in which the next sentence of the first speech recognition sentence is connected to the first speech recognition sentence. and the third similarity is a degree of similarity between the second input sentence and the first speech recognition sentence in which the next sentence of the first input sentence is connected to the first input sentence.

Here, the next sentence of the first voice recognition sentence according to an embodiment means a sentence located next to the first voice recognition sentence in the result of segmenting the voice signal, and the segmented recognition sentence list 104 according to the embodiment. ) may mean the next voice recognition sentence of the first voice recognition sentence. According to the meaning of the next sentence, the second voice recognition sentence may mean a sentence in which the first voice recognition sentence and the next sentence are connected.

The next sentence of the first input sentence means a sentence located next to the first input sentence in the result of the script segmentation, and the next input sentence of the first input sentence on the segmented input sentence list 105 according to an embodiment. can mean According to the meaning of the next sentence, the second input sentence may mean a sentence in which the first input sentence and the next sentence are connected.

According to an embodiment, a Levenshtein distance may be calculated and used to align a speech recognition sentence corresponding to a segmented voice signal and an input sentence that is a segmented script. In other words, the similarity between the speech recognition sentence corresponding to the segmented speech signal and the input sentence as the segmented script is obtained based on the editing distance between the speech recognition sentence corresponding to the segmented speech signal and the input sentence as the segmented script. can be

The edit distance represents the minimum value calculated by how many times a character in the character string A must be inserted, deleted, or replaced to be changed to the character string B, and can be used as an index to judge the similarity between two strings. For example, if you calculate the edit distance between the string A "it's over" and the string B "that's it", replace "end" with "that" in string A, and replace "this" with "chi" Since it can be changed to character string B, the editing distance between character string A and character string B is 2. For another example, when calculating the editing distance between the string A "Geachi" and the string B "Geechiya", you must insert "Ya" in the string A to change it to the string B, so the editing of the string A and the string B The distance will be 1.

The similarity between two sentences is calculated by calculating the editing distance between the two sentences, and when the editing distance is small, the similarity between the two sentences is interpreted as high, and when the editing distance is large, the similarity between the two sentences can be interpreted as low. In addition to the editing distance, various scales indicating the similarity between two sentences or the distance between the two sentences may be applied to the embodiments described below.

According to an embodiment, the similarity between the speech recognition sentence and the input sentence may correspond to the similarity between the voice recognition sentence converted into the phoneme sequence and the input sentence converted into the phoneme sequence. The phoneme sequence conversion is converting a phoneme sequence into a phoneme sequence corresponding to the phoneme sequence, and may mean converting a text into a code indicating pronunciation corresponding to the text according to a phonological rule or the like. For example, the sentences of “two thousand six hundred seventy five” can be converted into “the two thousand six hundred seventy five words” through phoneme column conversion. When a number is included in the sentence, the number in the sentence may also be converted into a phoneme sequence corresponding to the pronunciation of the number. For example, the sentences of “2675 chae” can be converted into “this choeotbek chil ssi bochae” through phoneme column conversion.

When obtaining the similarity between the voice recognition sentence and the input sentence by calculating the editing distance between the voice recognition sentence and the input sentence according to an embodiment, after converting each of the voice recognition sentence and the input sentence into a phoneme string, the converted voice recognition By calculating the editing distance between the sentence and the converted input sentence, the similarity between the speech recognition sentence and the input sentence may be obtained.

For example, when the number of input sentences is “2675” and the number of speech recognition sentences is “two thousand six hundred and seventy-five”, the editing distance is calculated as 7 even though the input sentence and the voice recognition sentence are pronounced the same. On the other hand, when the editing distance is calculated after each of the input sentences and the speech recognition sentences are converted into phoneme columns, the input sentences converted into phoneme columns and the speech recognition sentences converted into phoneme columns are the same as, The distance is counted as 0.

Referring to FIG. 3A , in the alignment algorithm according to an embodiment, when the first speech recognition sentence is updated to the second speech recognition sentence or the first input sentence is updated to the second input sentence, the first step of the algorithm is to obtain similarities again. Returning to step 300 , subsequent steps may be repeatedly performed by recognizing the second speech recognition sentence as a new first speech recognition sentence or the second input sentence as a new first input sentence. That is, the steps of obtaining similarities based on the first voice recognition sentence and the first input sentence and determining whether to update the first voice recognition sentence or the first input sentence based on the similarities and updating the first voice recognition sentence or the first input sentence include the first The iteration may be performed according to the update of the speech recognition sentence or the update of the first input sentence.

More specifically, when it is determined that the second similarity is the highest in steps 340 and 350, the first speech recognition sentence is updated (360) with the second speech recognition sentence and transferred to the next iteration, step When it is determined in steps 340 and 350 that the third similarity is the highest, the first input sentence may be updated as the second input sentence (370) and transmitted to the next iteration. According to an embodiment, when the first similarity is the highest, the iteration of the sorting algorithm is terminated, and the method includes storing a first voice signal and a first input sentence corresponding to the first voice recognition sentence in a voice database. can do.

An equal sign is not included in comparing the first degree of similarity, the second degree of similarity, and the third degree of similarity in step 340 of FIG. 3A , but an equal sign may be included according to an embodiment. That is, according to an embodiment, when the first degree of similarity is equal to the second degree of similarity and greater than the third degree of similarity, when the first degree of similarity is equal to the third degree of similarity and greater than the second degree of similarity, and when the first degree of similarity, the second degree of similarity and Even when all the third similarities are the same, the iteration of the sorting algorithm may be terminated.

Although the equal sign is not included in comparing the second degree of similarity and the third degree of similarity in step 350 of FIG. 3A , the equal sign may be included in some embodiments. That is, when the second degree of similarity and the third degree of similarity are the same, the first speech recognition sentence may be updated to the second speech recognition sentence (360), and the first input sentence may be updated to the second input sentence (370). ) may be However, when the second degree of similarity and the third degree of similarity are the same, either one of steps 360 and 370 is performed, but not both steps 360 and 370 are performed. That is, when the second degree of similarity and the third degree of similarity are the same, any one of an operation of updating the first speech recognition sentence to the second speech recognition sentence and an operation of updating the first input sentence to the second input sentence is performed. According to an embodiment, when the second degree of similarity and the third degree of similarity are the same, a different criterion may be introduced in determining which operation is to be performed. For example, when the second degree of similarity and the third degree of similarity obtained by the editing distance are the same, the step of re-acquiring the second degree of similarity and the third degree of similarity by using another criterion for determining the degree of similarity between sentences may be performed. Alternatively, if the second degree of similarity obtained by the editing distance and the third degree of similarity are the same, in order to make the two strings the same when the editing distance is obtained, a step ( Any one of 360) and step 370 may be selected.

Referring to FIG. 3B , the sorting algorithm according to an embodiment includes first obtaining a first degree of similarity ( 310 ) and determining whether to obtain a second degree of similarity and whether to obtain a third degree of similarity based on the first degree of similarity ( 310 ). may include. For example, if the first similarity corresponds to a predetermined criterion, it is determined not to acquire the second similarity and the third similarity, and if the first similarity does not correspond to the predetermined criterion, the second similarity and the second similarity are not obtained. and determining to obtain 3 similarities.

Referring to FIG. 3B , the sorting algorithm according to an embodiment may include determining whether to obtain a second degree of similarity and a third degree of similarity by comparing (320) the first degree of similarity with a first predetermined threshold. As a result of the comparison, if the first degree of similarity is less than or equal to a predetermined first threshold, acquiring ( 330 ) a second degree of similarity and a third degree of similarity may be included. On the other hand, when the first similarity is greater than a predetermined first threshold, the sorting algorithm is terminated without acquiring the second and third similarities, and the first voice signal corresponding to the first voice recognition sentence and the first input sentence may be stored in this voice database. If the first input sentence and the first speech recognition sentence are substantially the same, the sorting algorithm may be terminated without acquiring the second similarity and the third similarity. Here, when the editing distance between the first input sentence and the first speech recognition sentence is less than a predetermined threshold distance, or when the first similarity between the first input sentence and the first voice recognition sentence is greater than the predetermined threshold similarity, the first It may be determined that the input sentence and the first voice recognition sentence are substantially the same. When the first input sentence and the first voice recognition sentence are substantially the same, it is determined that the first voice signal and the first input sentence are aligned, and the first voice recognition sentence is added to the first voice recognition sentence without the need to acquire the second similarity and the third similarity. By storing the corresponding first voice signal and the first input sentence in the voice database, the efficiency of the sorting operation may be increased.

Although FIG. 3B illustrates a case in which it is determined 320 whether the first similarity exceeds the first threshold, it may be determined whether the first similarity is greater than or equal to the first threshold according to an exemplary embodiment. In step 320 according to an embodiment, the first threshold serving as a criterion for determination may correspond to a predetermined arbitrary value.

Referring to FIG. 3B , in the alignment algorithm according to an embodiment, when the first similarity is the largest, the first voice signal (or the first voice signal) corresponding to the first voice recognition sentence based on the first similarity or the number of connections. and storing ( 390 ) the first speech recognition sentence) and the first input sentence in a mismatch suspicious list. The number of connections according to an embodiment is the sum of the number of times the first speech recognition sentence is updated to the second speech recognition sentence and the number of times the first input sentence is updated to the second input sentence, and when the iteration of the sorting algorithm is repeated It can be obtained by increasing (380) the number of connections every time. That is, the number of connections according to an embodiment may correspond to the number of iterations of the sorting algorithm.

According to an embodiment, when the first similarity is highest and the number of connections exceeds a second predetermined threshold, or when the first similarity is the highest and the first similarity is less than a third predetermined threshold, the first voice signal (Or, the first voice signal and the first voice recognition sentence) and the first input sentence may be stored in the mismatch suspicious list. For example, when the first similarity is greater than the first threshold or greater than the second and third similarities, an additional linking operation (updating the speech recognition sentence or updating the input sentence) may not be performed. In this case, if the number of connections is greater than the second threshold, there is a possibility that two or more sentences are aligned together, and thus, the list may be stored in the suspicious inconsistency list. Alternatively, irrespective of the number of connections, even when the first similarity is less than the third threshold, it may be stored in the mismatch suspicious list. The third threshold may be set to be the same as or set to be different from the first threshold.

On the other hand, when the first similarity is the highest, the number of connections is less than or equal to the second threshold, and the first similarity is greater than or equal to the third threshold, the sorting algorithm may be terminated without being stored in the mismatch suspicious list. The second threshold and the third threshold, which are the criteria for determining whether to store the inconsistency suspicious list, according to an embodiment, may correspond to predetermined arbitrary values.

Although FIG. 3B illustrates a case in which it is determined whether the number of connections exceeds the second threshold and whether the first similarity is less than the third threshold, it may be determined whether the number of connections is equal to or greater than the second threshold according to an embodiment. Also, it may be determined whether the first similarity is equal to or less than a third threshold. In addition, in determining whether to store in the suspicious inconsistency list, the determination criterion shown in FIG. 3B is only an example, and the criteria for determining whether to store in the inconsistent suspicious list based on the first similarity or the number of connections may vary. can

According to an embodiment, the step of arranging the voice signal and the script in units of sentences may include determining whether to update the first voice recognition sentence or the first input sentence based on the first similarity, the second similarity, and the third similarity. , whether the first speech recognition sentence or the first input sentence is updated based on the first similarity, the second similarity, and the third similarity based on the updated first speech recognition sentence or the updated first input sentence By repeatedly performing the step of determining, and determining not to update the first voice recognition sentence and the first input sentence, the first voice signal corresponding to the first voice recognition sentence and the first input sentence are stored in the database It may include the step of storing it in .

When the first speech recognition sentence or the first input sentence is updated, the sorting algorithm according to an embodiment may include a first similarity degree, a second similarity degree, and a second similarity degree based on the updated first speech recognition sentence or the updated first input sentence. The steps of obtaining 3 similarities and determining whether to update based on the obtained similarities may be repeatedly performed, and when neither the first speech recognition sentence nor the first input sentence is updated, the sorting algorithm may be terminated. .

Although not shown in the drawings, according to an embodiment, the voice signal and the script aligned according to the alignment algorithm may be segmented into smaller units. In this case, the step of storing the first voice signal and the first input sentence corresponding to the voice recognition sentence in the database is based on the forced voice alignment of the first voice signal and the first input sentence and the result of the forced voice sorting, It may include segmenting and storing the first voice signal and the first input sentence. For example, when the first voice signal according to an embodiment corresponds to the voice signal corresponding to the sentence A+B and the first input sentence corresponds to the sentence A+B, the first voice signal and the first input sentence is forced speech alignment according to the forced speech alignment algorithm, and based on the result, the first speech signal is segmented into a speech signal corresponding to the sentence A and the speech signal corresponding to the sentence B, and the first input sentence is divided into the sentence A and the sentence It can be segmented into B and stored. Whether the first voice signal and the first input sentence are segmented according to the forced voice sorting and stored in the database may be determined according to a predetermined criterion.

4 to 6 are diagrams illustrating examples for specifically explaining an alignment algorithm according to an embodiment.

Referring to FIG. 4 , the voice signals 410 and 420 segmented into sentences of “what the hell are you doing now” and “what the hell are you doing” and the script are “what the hell are you doing now?” and input sentences 430 and 440 segmented in sentence units of “I have to communicate with listeners” are shown. The first similarity 450 , the second similarity 460 , and the third similarity 470 shown in FIG. 4 are calculated results based on the voice signal, the voice recognition sentence, and the input sentence shown in FIG. 4 .

Referring to FIG. 4 , the first similarity 450 is obtained based on the editing distance between the first speech recognition sentence 411 and the first input sentence 430 . Since the first speech recognition sentence 411, “what the hell are you doing” and “what the hell are you doing now,” the first input sentence 430 is a difference of 7 characters from “what are you doing”, the editing distance is calculated as 7. The second similarity 460 is the editing distance between the second speech recognition sentence linking the next sentence 421 of the first voice recognition sentence 411 to the first voice recognition sentence 411 and the first input sentence 430 . is obtained based on Since the second speech recognition sentence "what the hell are you doing now" and the first input sentence 430 "what the hell are you doing now" are the same, the editing distance is 0. The third similarity 470 is based on the editing distance between the second input sentence connecting the next sentence 440 of the first input sentence 430 to the first input sentence 430 and the first voice recognition sentence 411 . is obtained by The editing distance is the 18-character difference between the first speech recognition sentence "what the hell are you doing" and the second input sentence, "what the hell are you doing right now?" 18. As described above, since the smaller the editing distance between two sentences means that the two sentences are similar, in contrast to the magnitude relationship of the editing distance, the magnitude relationship of the first degree of similarity, the second degree of similarity, and the third degree of similarity is the second similarity > first degree of similarity. The degree of similarity > the third degree of similarity.

The sorting algorithm according to an embodiment includes updating (360) the first speech recognition sentence to the second speech recognition sentence when the second similarity is the greatest. That is, since the first speech recognition sentence 411 is updated to the second speech recognition sentence according to the sorting algorithm, the first speech recognition sentence is updated to "what the hell are you doing now". Acquiring and updating the first degree of similarity, the second degree of similarity, and the third degree of similarity based on the updated first speech recognition sentence and the first input sentence may be repeated.

Referring to FIG. 5 , the voice signals 510 and 520 of “I’m so nervous” and “My career is”, which are some of the voice signals segmented in sentence units, and input sentences in which the script is segmented sentence-by-sentence units. Some of them say, "It's over." and input sentences 530 and 540 of "I'm so nervous." The first similarity 550 , the second similarity 560 , and the third similarity 570 shown in FIG. 5 are calculated results based on the voice signal, the voice recognition sentence, and the input sentence shown in FIG. 5 .

Referring to FIG. 5 , the first similarity 550 is obtained based on the editing distance between the first speech recognition sentence 511 and the first input sentence 530 . The difference between the first speech recognition sentence 511, “I’m so nervous,” and the first input sentence 530, “It’s over,” is two letters of “It’s over” and “It’s over” and four letters of “I’m so nervous” Therefore, the edit distance is calculated as 6. The second similarity 560 is the editing distance between the second speech recognition sentence linking the next sentence 521 of the first voice recognition sentence 511 to the first voice recognition sentence 511 and the first input sentence 530 . is obtained based on The second speech recognition sentence "Hey, I'm so nervous, my career" and the first input sentence 530, "It's over." Since there is a difference of 9 characters, the edit distance is calculated as 11. The third similarity 570 is based on the editing distance between the second input sentence connecting the next sentence 540 of the first input sentence 530 to the first input sentence 530 and the first voice recognition sentence 511 . is obtained by Since the first speech recognition sentence "I'm so nervous" and the second input sentence, "It's over, I'm so nervous" is a two-letter difference between "It's the end" and "I'm so nervous," the editing distance is 2. As described above, since the smaller the editing distance between two sentences means that the two sentences are similar, in contrast to the magnitude relationship of the editing distance, the magnitude relationship of the first similarity, the second similarity, and the third similarity is the third similarity > first The degree of similarity > second degree of similarity.

The sorting algorithm according to an embodiment includes updating the first input sentence to the second input sentence when the third similarity is the largest ( 370 ). That is, since the first input sentence 511 is updated to the second input sentence according to the sorting algorithm, the first input sentence is updated to "It's over. I'm too nervous."

Referring to FIG. 6 , the first input sentence 530 of FIG. 5 is updated, the first input sentence 630 of “It’s over. I’m so nervous.” and the first input sentence 630 of “my career is ruined.” The next sentence 640 of ) is shown. The first similarity 650 , the second similarity 660 , and the third similarity 670 shown in FIG. 6 are calculated results based on the voice signal, the voice recognition sentence, and the input sentence shown in FIG. 6 .

Referring to FIG. 6 , the first similarity 650 is obtained based on the editing distance between the first speech recognition sentence 511 and the first input sentence 630 . Since the first speech recognition sentence 511, "It's so terrifying" and the first input sentence 630, "It's over. I'm so nervous," is a two-letter difference between "the end" and "it's so", so the editing distance is 2 Calculated. The second similarity 660 is the editing distance between the second speech recognition sentence linking the next sentence 521 of the first voice recognition sentence 511 to the first voice recognition sentence 511 and the first input sentence 630 . is obtained based on The second speech recognition sentence “Hey, I’m so nervous, my career” and the first input sentence 630, “It’s over. I’m so nervous.” are two letters of “the end” and “I’m so nervous,” and “my career is” Since it is a difference of 5 characters, the editing distance is calculated as 7. The third similarity 770 is based on the editing distance between the second input sentence linking the next sentence 640 of the first input sentence 630 to the first input sentence 630 and the first voice recognition sentence 511 is obtained by The first speech recognition sentence "I'm so nervous" and the second input sentence, "It's over. I'm so nervous. My career is shattered" is the two letters of "It's over" and "It's over" and "My career is shattered." " Since there is a difference of 9 characters, the edit distance is 11. As described above, since the smaller the editing distance between two sentences means that the two sentences are similar, in contrast to the magnitude relationship of the editing distance, the magnitude relationship of the first similarity, the second degree of similarity, and the third degree of similarity is 1st similarity > 2nd degree of similarity. The degree of similarity > the third degree of similarity.

The sorting algorithm according to an embodiment may be terminated when the first similarity is the greatest, and the first voice signal corresponding to the first voice recognition sentence and the first input sentence may be stored in a voice database. In other words, when the first similarity is the greatest, the system according to an embodiment determines not to update the first speech recognition sentence and the first input sentence, and determines not to update the first speech recognition sentence corresponding to the first speech recognition sentence according to the decision not to update. and storing the first voice signal and the first input sentence in a database. That is, the voice signal 510 and the first input sentence 630 corresponding to the first voice recognition sentence 511, “I’m so nervous,” may be stored in the database.

According to an embodiment, when the first similarity is obtained based on the editing distance between the first speech recognition sentence and the first input sentence, changing the first voice recognition sentence to the first input sentence in the editing distance calculation process To this end, based on the number of characters to be inserted, the number of characters to be deleted, and/or the number of characters to be replaced, either one of the step of obtaining the second degree of similarity and the step of obtaining the third degree of similarity may be selected. can In other words, the acquiring of the first similarity includes acquiring the number of characters to be deleted, the number of characters to be inserted, and the number of characters to be replaced in order to change the first speech recognition sentence into the first input sentence. and obtaining an edit distance between the first speech recognition sentence and the first input sentence based on the number of characters to be deleted, the number of characters to be inserted, and the number of characters to be replaced. In this case, it may be determined to obtain any one of the second degree of similarity and the third degree of similarity based on the obtained number of characters to be deleted and the number of characters to be inserted.

For example, when the number of characters to be inserted in order to change the first speech recognition sentence into the first input sentence occupies a certain ratio or more in the editing distance, the next sentence of the first voice recognition sentence is recognized by the first voice recognition It may be determined to obtain a second degree of similarity between the second speech recognition sentence connected to the sentence and the first input sentence. On the other hand, in order to change the first speech recognition sentence into the first input sentence, when the number of characters to be deleted occupies a certain ratio or more in the editing distance, the first sentence connecting the next sentence of the first input sentence to the first input sentence It may be determined to obtain a third degree of similarity between the second input sentence and the first speech recognition sentence.

In this way, when it is determined to obtain any one of the second and third similarities, the first input sentence or the first voice is obtained by obtaining any one of the second and third similarities according to the determination and comparing it with the first similarity. The iteration of the sorting algorithm may be repeated by updating the recognition sentence, or the first voice signal and the first input sentence corresponding to the first voice recognition sentence may be stored in the database, and the sorting algorithm may be terminated.

7 is a diagram for explaining an operation of detecting and correcting an error in the voice database building system according to an embodiment.

Referring to FIG. 7 , the system for constructing a voice database according to an embodiment provides a first voice signal, a first voice recognition sentence, and a first The module may further include a module 160 for correcting at least one of the input sentences.

More specifically, in the process of storing the first voice signal and the first input sentence corresponding to the first voice recognition sentence in the database according to the decision not to update the first voice recognition sentence and the first input sentence, the first voice signal and Based on the first similarity between the first input sentences, the first voice signal and the first input sentence may be separately classified and stored as data requiring correction according to a predetermined criterion. For example, when the first similarity between the first speech recognition sentence and the first input sentence for which it is determined not to be updated is less than a predetermined value, the first voice signal (or the first voice signal and the first voice recognition sentence) and the second One input sentence may be classified as the corrected necessary data and stored in the inconsistency suspicious list 108. When the first similarity is greater than a predetermined value, the first voice signal (or the first voice signal and the first voice recognition sentence) ) and the first input sentence may be stored in the match list 107 . As described above, even when the first similarity is greater than a predetermined value, the first voice signal (or the first voice signal and the first voice recognition sentence) and the first input sentence are inconsistent suspicious list 108 based on the number of connections. ) can be stored in

In other words, the system according to an embodiment of the present invention provides an error (eg, the speaker incorrectly reads the script, or there is an error in voice recognition) in the script and the corresponding voice signal based on the first similarity according to a predetermined criterion. , there is a typo in the script, etc.). The first voice signal (or the first voice signal and the first voice recognition sentence) and the first input sentence determined to have errors in the system according to an embodiment may be classified as data requiring correction and stored separately.

For example, when the system according to an embodiment determines that there is an error when the editing distance of two sentences is not 0, between the first speech recognition sentence 511 and the first input sentence 630 of FIG. 6 . Since the editing distance of is 2, the first voice signal 510, the first voice recognition sentence 511, and the first input sentence 630 are determined to have errors and may be stored as a mismatch suspicious list.

In the correction module 160 according to an embodiment, the data stored in the suspicious discrepancy list may be corrected based on various methods. For example, when a voice recognition sentence is corrected based on an input sentence, the first voice signal of FIG. 6 may be stored in the database as being recognized as "I'm too nervous at the end". Alternatively, a user interface may be provided so that the corresponding data can be easily corrected using the inconsistency suspicious list.

According to an embodiment, the result of obtaining at least one of the first degree of similarity, the second degree of similarity, and the third degree of similarity may be displayed through the interface. For example, sentences to be compared in the first degree of similarity, the second degree of similarity, and the third degree of similarity may be displayed through the interfaces shown in FIGS. 4, 5, and 6 , and numerical values of the calculated editing distances. In addition, a portion different from the first input sentence 630 in the first voice recognition sentence 511 through the interface as shown in FIG. 6 may be displayed with an underline.

The method according to the embodiment may be implemented in the form of program instructions that can be executed through various computer means and recorded in a computer-readable medium. The computer-readable medium may include program instructions, data files, data structures, etc. alone or in combination. The program instructions recorded on the medium may be specially designed and configured for the embodiment, or may be known and available to those skilled in the art of computer software. Examples of the computer readable recording medium include magnetic media such as hard disks, floppy disks and magnetic tapes, optical media such as CD-ROMs and DVDs, and magnetic media such as floppy disks. - includes magneto-optical media, and hardware devices specially configured to store and execute program instructions, such as ROM, RAM, flash memory, and the like. Examples of program instructions include not only machine language codes such as those generated by a compiler, but also high-level language codes that can be executed by a computer using an interpreter or the like. The hardware devices described above may be configured to operate as one or more software modules to perform the operations of the embodiments, and vice versa.

The software may comprise a computer program, code, instructions, or a combination of one or more thereof, which configures a processing device to operate as desired or is independently or collectively processed You can command the device. The software and/or data may be any kind of machine, component, physical device, virtual equipment, computer storage medium or device, to be interpreted by or to provide instructions or data to the processing device. , or may be permanently or temporarily embody in a transmitted signal wave. The software may be distributed over networked computer systems, and stored or executed in a distributed manner. Software and data may be stored in one or more computer-readable recording media.

As described above, although the embodiments have been described with reference to the limited drawings, those skilled in the art may apply various technical modifications and variations based on the above. For example, the described techniques are performed in a different order than the described method, and/or the described components of the system, structure, apparatus, circuit, etc. are combined or combined in a different form than the described method, or other components Or substituted or substituted by equivalents may achieve an appropriate result.

Therefore, other implementations, other embodiments, and equivalents to the claims are also within the scope of the following claims.

Claims (25)

obtaining a first similarity between a first speech recognition sentence corresponding to a part of the voice signal and a first input sentence corresponding to a part of the script of the voice signal;
obtaining a second similarity between a second speech recognition sentence in which a next sentence of the first speech recognition sentence is connected to the first speech recognition sentence and the first input sentence;
obtaining a third similarity between a second input sentence in which a next sentence of the first input sentence is connected to the first input sentence and the first speech recognition sentence; and
arranging the voice signal and the script in units of sentences based on the first degree of similarity, the second degree of similarity, and the third degree of similarity;
containing
How to build a voice database.
According to claim 1,
The sorting step is
storing a first voice signal corresponding to the first voice recognition sentence and the first input sentence in a voice database when the first similarity is the highest;
when the second similarity is the highest, updating the first speech recognition sentence to the second speech recognition sentence and transferring it to a next iteration; and
When the third similarity is the highest, updating the first input sentence to the second input sentence and transferring the second input sentence to the next iteration;
containing
How to build a voice database.
According to claim 1,
The sorting step is
determining whether to update the first speech recognition sentence or the first input sentence based on the first degree of similarity, the second degree of similarity, and the third degree of similarity;
When it is determined to update the first speech recognition sentence or the first input sentence, the first speech recognition sentence or the updated first input sentence is based on the first similarity, the second similarity, and the third similarity. repeatedly performing the step of determining whether to update; and
Storing a first voice signal corresponding to the first voice recognition sentence and the first input sentence in the database in response to determining not to update the first voice recognition sentence and the first input sentence
containing
How to build a voice database.
4. The method of claim 3,
Repeatedly performing the step of determining whether to update is
obtaining a first degree of similarity, a second degree of similarity, and a third degree of similarity based on the updated first speech recognition sentence and the first input sentence when it is determined to update the first speech recognition sentence; and
obtaining a first degree of similarity, a second degree of similarity, and a third degree of similarity based on the updated first input sentence and the first speech recognition sentence when it is determined to update the first input sentence;
containing
How to build a voice database.
According to claim 1,
segmenting the speech signal into sentence units;
converting the speech signal into text based on speech recognition; and
Segmenting the script into sentence units
further comprising
How to build a voice database.
According to claim 1,
receiving the voice signal and the script;
pre-processing the received voice signal; and
pre-processing the received script
further comprising
How to build a voice database.
According to claim 1,
wherein the first degree of similarity, the second degree of similarity, and the third degree of similarity are obtained based on an editing distance.
How to build a voice database.
3. The method of claim 2,
The sorting step is
modifying at least one of a first voice signal, the first voice recognition sentence, and the first input sentence based on the first similarity according to a predetermined criterion;
containing
How to build a voice database.
3. The method of claim 2,
The sorting step is
modifying at least one of the first voice signal, the first voice recognition sentence, and the first input sentence based on the number of connections corresponding to the number of repetitions of the iteration
containing
How to build a voice database.
According to claim 1,
displaying a result of obtaining at least one of the first degree of similarity, the second degree of similarity, and the third degree of similarity through an interface;
further comprising
How to build a voice database.
4. The method of claim 3,
The step of storing in the database is
forcibly aligning the first speech signal and the first input sentence; and
Segmenting and storing the first voice signal and the first input sentence based on the forced voice alignment result
containing
How to build a voice database.
According to claim 1,
After obtaining the first degree of similarity,
determining whether to obtain the second degree of similarity and whether to obtain the third degree of similarity based on the first degree of similarity;
further comprising,
The step of obtaining the second degree of similarity includes:
based on the determination, obtaining the second degree of similarity;
The step of obtaining the third degree of similarity includes:
based on the determination, obtaining the third degree of similarity;
How to build a voice database.
13. The method of claim 12,
The step of determining whether to obtain the second degree of similarity and whether to obtain the third degree of similarity includes:
determining not to acquire the second similarity and the third similarity when the first similarity corresponds to a predetermined criterion; and
determining that the second degree of similarity and the third degree of similarity are obtained when the first degree of similarity does not correspond to a predetermined criterion;
containing,
How to build a voice database.
13. The method of claim 12,
The step of obtaining the first similarity is
obtaining the number of characters to be deleted, the number of characters to be inserted, and the number of characters to be replaced in order to change the first speech recognition sentence into the first input sentence; and
obtaining an edit distance between the first speech recognition sentence and the first input sentence based on the number of characters to be deleted, the number of characters to be inserted, and the number of characters to be replaced
including,
The step of determining whether to obtain the second degree of similarity and whether to obtain the third degree of similarity includes:
determining, based on the number of characters to be deleted and the number of characters to be inserted, to obtain any one of the second degree of similarity and the third degree of similarity;
containing,
How to build a voice database.
According to claim 1,
The first similarity includes a degree of similarity between the first speech recognition sentence converted into a phoneme sequence and the first input sentence converted into a phoneme sequence,
The second degree of similarity includes a degree of similarity between the second speech recognition sentence converted into a phoneme sequence and the first input sentence converted into a phoneme sequence,
The third degree of similarity includes a degree of similarity between the first speech recognition sentence converted into a phoneme sequence and the second input sentence converted into a phoneme sequence,
How to build a voice database.
16. A computer program stored on a medium in combination with hardware to execute the method of any one of claims 1 to 15.
Obtaining a first similarity between a first speech recognition sentence corresponding to a part of the voice signal and a first input sentence corresponding to a part of the script of the voice signal, and converting the next sentence of the first voice recognition sentence to the first voice A second input sentence and the first sentence obtained by obtaining a second similarity between a second speech recognition sentence connected to a recognition sentence and the first input sentence, and a next sentence of the first input sentence connected to the first input sentence at least one processor for obtaining a third degree of similarity between speech recognition sentences, and arranging the speech signal and the script in sentence units based on the first degree of similarity, the second degree of similarity, and the third degree of similarity; and
A voice database for storing the voice signal and the script sorted by the sentence unit
containing
Voice database building device.
18. The method of claim 17,
the processor
In aligning the voice signal and the script in sentence units,
When the first similarity is the highest, the first voice signal corresponding to the first voice recognition sentence and the first input sentence are stored in the voice database, and when the second similarity is the highest, the first voice recognition The sentence is updated to the second speech recognition sentence and transferred to the next iteration, and when the third similarity is the highest, the first input sentence is updated to the second input sentence and transferred to the next iteration.
Voice database building device.
18. The method of claim 17,
the processor
In aligning the voice signal and the script in sentence units,
It is determined whether to update the first speech recognition sentence or the first input sentence based on the first similarity, the second similarity, and the third similarity, and the first speech recognition sentence or the first input sentence is Upon determining the update, repeatedly performing the step of determining whether to update based on the first similarity, the second similarity, and the third similarity based on the updated first speech recognition sentence or the updated first input sentence, Storing the first voice signal and the first input sentence corresponding to the first voice recognition sentence in the database in response to determining not to update the first voice recognition sentence and the first input sentence
Voice database building device.
20. The method of claim 19,
the processor
In repeatedly performing the step of determining whether to update,
When it is determined to update the first speech recognition sentence, a first degree of similarity, a second degree of similarity, and a third degree of similarity are obtained based on the updated first speech recognition sentence and the first input sentence, and the first input when it is determined to update the sentence, obtaining a first degree of similarity, a second degree of similarity, and a third degree of similarity based on the updated first input sentence and the first speech recognition sentence;
Voice database building device.
18. The method of claim 17,
the processor
segmenting the voice signal into sentence units, converting the voice signal into text based on voice recognition, and segmenting the script into sentence units
Voice database building device.
19. The method of claim 18,
the processor
In aligning the voice signal and the script in sentence units,
modifying at least one of the first voice signal, the first voice recognition sentence, and the first input sentence based on the first similarity according to a predetermined criterion
Voice database building device.
18. The method of claim 17,
the processor
In storing the first voice signal corresponding to the first voice recognition sentence and the first input sentence in the database,
performing forced voice alignment of the first voice signal and the first input sentence, and segmenting and storing the first voice signal and the first input sentence based on a result of the forced voice alignment
Voice database building device.
18. The method of claim 17,
the processor
After obtaining the first similarity,
determining whether to obtain the second degree of similarity and whether to obtain the third degree of similarity based on the first degree of similarity, and obtaining the second degree of similarity and the third degree of similarity based on the determination;
Voice database building device.
25. The method of claim 24,
the processor
In determining whether to obtain the second degree of similarity and whether to obtain the third degree of similarity,
If the first degree of similarity meets a predetermined criterion, it is determined not to acquire the second degree of similarity and the third degree of similarity; determining to obtain a third degree of similarity;
Voice database building device.

Images