KR102094935B1 - System and method for recognizing speech - Google Patents

Abstract

The speech recognition method capable of generating automatic phonemes according to the present invention includes unsupervised learning a feature vector of speech data; Generating a phoneme set by clustering selected acoustic characteristics based on the unsupervised learning result; And assigning a phoneme string to the voice data based on the generated phoneme set, and generating an acoustic model based on the phoneme data to which the phoneme string is assigned and the phoneme string.

Description

Speech recognition system and method {SYSTEM AND METHOD FOR RECOGNIZING SPEECH}

The present invention relates to a speech recognition system and method, and more particularly, to a speech recognition system and method capable of generating automatic phonemes.

As a phoneme clustering method according to the prior art, there is a method using Bhattacharyya distance measurement proposed by B.Mak (B. Mak, E. Barnard, “Phone clustering using the Bhattacharyya distance”, in Proc. ICSLP, 1996). This is to use the Bhattacharyya distance measurement method as a means for classifying the acoustic characteristics of each phoneme and tying the acoustic distance. However, this method has a problem in that it uses a predetermined phoneme set based on expertise.

That is, in the case of the prior art, acoustic modeling is performed using a phoneme set predetermined by the knowledge of phonetic experts, and a pronunciation dictionary is generated by using linguistic pronunciation rules.

However, this method has a problem that does not correspond to natural continuous speech, and in particular, it is difficult to deal with diversity such as pronunciation reduction and omission and variable pronunciation. In addition, since the acoustic model is generated based on the pronunciation dictionary in the environment in which the transcription is given, the acoustic model is affected by an inaccurate pronunciation dictionary, and there is a limitation that the model can be constructed only when there is the transferred data. exist.

In addition, various top-down and bottom-up methods for clustering phonemes have been proposed, but these methods also have a problem of using a predetermined phoneme set.

Meanwhile, R.Singh proposed a method for automatically determining phoneme sets and pronunciation dictionaries (R. Singh, B. Ray and R.Stern, “Automatic generation of phone sets and lexical transcriptions”, in Proc. ICASSP, 2000 ). This method uses the maximum a posteriori (MAP) method to create an optimal pronunciation dictionary, and optimizes the acoustic model of a phoneme set based on likelihood. That is, the goal is to gradually generate an optimized phoneme and pronunciation dictionary by applying the MAP method cyclically.

In addition, in the case of the prior art, a method of selecting an optimal candidate by generating a graph for a pronunciation candidate is used to generate a word pronunciation dictionary. This method can gradually select the best candidate based on the initial manually entered phoneme and pronunciation dictionary.

However, this method is not easy to optimize and requires more appropriate data collection because it is an extremely general approach that requires a large number of phonemes to be generated manually at the beginning, and has no restrictions on how to generate a complete pronunciation dictionary. there is a problem.

In addition, since the objective function used based on the applied likelihood does not fit well in solving the problem, cyclic processing to solve the optimization problem is not an optimal solution.

In addition, since the conventional speech recognition process goes through a pronunciation rule conversion module, there is a problem that there is an artificial pronunciation distortion that differs from the actual acoustic speech.

Unlike an existing method of determining and classifying a phone, which is an acoustic basic unit for performing speech recognition, by an expert analysis, an embodiment of the present invention is a speech recognition system capable of automatically generating a phoneme based on speech data And methods.

However, the technical problem to be achieved by the present embodiment is not limited to the technical problem as described above, and other technical problems may exist.

As a technical means for achieving the above technical problem, a speech recognition method capable of generating automatic phonemes according to a first aspect of the present invention includes unsupervised learning a feature vector of speech data; Generating a phoneme set by clustering selected acoustic characteristics based on the unsupervised learning result; And assigning a phoneme string to the voice data based on the generated phoneme set, and generating an acoustic model based on the phoneme data to which the phoneme string is assigned and the phoneme string.

In addition, the speech recognition system capable of generating automatic phonemes according to the second aspect of the present invention includes a memory in which a program for speech recognition is stored and a processor for executing a program stored in the memory, wherein the processor executes the program. Accordingly, unsupervised learning is performed by extracting a feature vector from non-transcribed speech data, clustering selected acoustic characteristics based on the unsupervised learning result, and generating a phoneme set, based on the generated phoneme set, A phoneme string is assigned to voice data, and an acoustic model is generated based on the voice data to which the phoneme string is assigned and the phoneme string.

According to any one of the above-described problem solving means of the present invention, since the pronunciation dictionary is generated based on the data uttered outside the limitations of the existing G2P generating a pronunciation dictionary based on a rule and a dictionary, the observed variable pronunciation is dictionary As it can be reflected in, the difference in pronunciation between the pronunciation conversion rule and the actual spoken voice can be reduced, thereby overcoming the limitation of free speech recognition.

In addition, since a phoneme is determined from voice data and an acoustic model is generated using information about the phoneme, there is an advantage that acoustic modeling is possible only with voice data, since it is difficult to collect a large amount of transferred voice data.

In addition, since a pronunciation dictionary is generated by distinguishing the uttered transcripts, unlike a conventional pseudo morpheme or word used as a speech recognition unit, speech recognition is based on a semantic element corresponding to a high-frequency word or phrase that is a bundle of speech. The unit may be reconstructed to provide speech recognition performance in a form more similar to human recognition.

In addition, by using the transcribed voice data, it can be adapted to a local dialect or expanded to a pronunciation dictionary adapted to an individual user's speech style.

In addition, in the case of a foreign language pronunciation conversion rule, it is an inaccessible factor for non-native speakers to become multi-lingual, but according to an embodiment of the present invention, multi-language expansion is easy.

In addition, if a new vocabulary is to be added to the speech recognition system, it is possible to easily add it by inputting a word and speaking it.

1 is a block diagram of a speech recognition system according to an embodiment of the present invention.
2 is a block diagram illustrating the function of a speech recognition system according to an embodiment of the present invention.
3 is a flowchart of a speech recognition method according to an embodiment of the present invention.
4 is a diagram for explaining a method of extracting a feature vector from speech data.
5 is a diagram for explaining a process of learning a feature vector using a stacked autoencoder.
6 is a diagram for explaining a process of learning a feature vector using a composite product autoencoder.
7 is a diagram for explaining a process of learning a feature vector using an auto-encoder cross-learning method.
8 is a flowchart of a method for generating a pronunciation dictionary and a language network.

Hereinafter, embodiments of the present invention will be described in detail with reference to the accompanying drawings so that those skilled in the art to which the present invention pertains can easily practice. However, the present invention can be implemented in many different forms and is not limited to the embodiments described herein. In addition, in order to clearly describe the present invention, parts not related to the description are omitted.

When a part of the specification "includes" a certain component, it means that other components may be further included instead of excluding other components, unless otherwise specified.

The present invention relates to a speech recognition system (1) and method capable of automatic phoneme generation.

According to an embodiment of the present invention, unlike a conventional method of determining and classifying a phone, which is an acoustic basic unit for performing speech recognition, by analyzing an expert, it is possible to automatically generate a phoneme.

In particular, an embodiment of the present invention defines a pattern-based phoneme set as unsupervised deep learning, and rules-based to assign phoneme strings to text using G2P using existing expert knowledge Deviating from the method, a phoneme string can be allocated by analyzing the voice signal itself.

Accordingly, an embodiment of the present invention solves the problem that the transferred voice data is relatively insufficient, so that an acoustic model can be generated even when there is no transcription, and an effective response to the variation of word pronunciation centered on a space-unit This is possible, it is possible to enable the speech recognition of the dialogue is severe distortion.

Hereinafter, with reference to the accompanying drawings will be described in detail.

1 is a block diagram of a speech recognition system 1 according to an embodiment of the present invention. 2 is a block diagram for explaining the function of the speech recognition system 1 according to an embodiment of the present invention.

The speech recognition system 1 according to an embodiment of the present invention includes a memory 10 and a processor 20.

A program for speech recognition is stored in the memory 10. At this time, the memory 10 refers to a non-volatile storage device and a volatile storage device that keep stored information even when power is not supplied.

For example, the memory 10 is a compact flash (CF) card, a secure digital (SD) card, a memory stick, a solid-state drive (SSD), and a micro SD NAND flash memory such as cards, magnetic computer storage devices such as hard disk drives (HDDs), and optical disc drives such as CD-ROMs, DVD-ROMs, and the like. You can.

The processor 20 executes a program stored in the memory 10, extracts a feature vector from non-transcription speech data as the program is executed, performs unsupervised learning, and based on the unsupervised learning result. The selected acoustic characteristics are clustered to generate a phoneme set. Then, a phoneme string is assigned to the voice data based on the generated voice set, and an acoustic model is generated based on the voice data and phoneme string to which the phoneme string is assigned.

In addition, the voice recognition system 1 may include a microphone for receiving voices of a user or the like, or an interface having the same.

Meanwhile, the voice recognition system 1 according to an embodiment of the present invention operated by the components of FIG. 1 may be represented by a functional block as shown in FIG. 2.

The speech recognition system 1 according to an embodiment of the present invention includes an unsupervised feature vector learning unit 100, a clustering unit 200, an acoustic model generator 300, a pronunciation dictionary generator 400, and a language network. It may include a portion 500.

In addition, as a configuration for generating a speech recognition result based on the generated acoustic model and language network, a feature vector extraction unit 600 and a speech recognition decoder 700 may be included.

For reference, the components shown in FIGS. 1 and 2 according to an embodiment of the present invention may be implemented in software or in a hardware form such as a field programmable gate array (FPGA) or an application specific integrated circuit (ASIC). Roles can be played.

However, 'components' are not limited to software or hardware, and each component may be configured to be in an addressable storage medium or may be configured to reproduce one or more processors.

Thus, as an example, a component is a component, such as software components, object-oriented software components, class components, and task components, processes, functions, attributes, procedures, subs It includes routines, segments of program code, drivers, firmware, microcode, circuitry, data, database, data structures, tables, arrays and variables.

Components and functions provided within those components may be combined into a smaller number of components or further separated into additional components.

Hereinafter, a voice recognition method performed in a voice recognition system according to an embodiment of the present invention will be described in detail with reference to FIGS. 3 to 8.

3 is a flowchart of a speech recognition method according to an embodiment of the present invention.

1. Unsupervised Feature Vector Learning Stage

In a voice recognition method according to an embodiment of the present invention, first, the unsupervised feature vector learning unit 100 unsupervises and learns a feature vector of speech data.

Specifically, the unsupervised feature vector learning unit 100 extracts a feature vector from non-transcribed voice data collected without transcription to extract the acoustic characteristics of the spoken voice data as a pattern without a preset symbol (S110).

4 is a diagram for explaining a method of extracting a feature vector from speech data.

As illustrated in FIG. 4, the unsupervised feature vector learning unit 100 converts voice data into a spectrogram (P1), and converts voice data converted into a spectrogram (P1) into preset time frame units (for example, 10ms) to convert to a mel-scale filterbank (mel-scale filterbank) to generate a first feature vector (S111). Then, the second feature vector is generated by splicing a window corresponding to a predetermined number of frames from the left and right of the first feature vector, and the generated second feature vector is used as a feature vector applied to unsupervised learning. It can be (S112).

In addition, the first feature vector obtained by the mel-scale filterbank (Discrete Cosine Transform) by the discrete cosine transform (Mel-Frequency Cepstral Coefficients) generated by the feature vector It can also be used as a first feature vector.

In another method, the unsupervised feature vector learning unit 100 first converts speech data into a spectrogram (P1). Then, in order to use the acoustic pattern of the spectrogram itself, the speech data converted to the spectrogram is grouped into two-dimensional units based on x frames (S115), and the generated feature matrix is applied to unsupervised learning. It can be used as a feature vector (S116).

The method of generating a feature vector using the filter bank (S111, S112) is a feature vector used for general in-depth learning, and can use information of every frame as input. And the method of generating and using the feature matrix (S115, S116) is a two-dimensional data that can be applied to deep learning of convolution, and extracts features having a shift-frame of less than x frames. Thus, the extracted feature matrix information can be used as input.

After the feature vector is extracted in this way, the unsupervised feature vector learning unit 100 performs unsupervised learning on the extracted feature vector (S120). At this time, the unsupervised feature vector learning unit 100 may learn the feature vector using the autoencoder illustrated in FIGS. 5 to 7.

5 is a diagram for explaining a process of learning a feature vector using a stacked autoencoder. 6 is a diagram for explaining a process of learning a feature vector using a composite product autoencoder. 7 is a diagram for explaining a process of learning a feature vector using an auto-encoder cross-learning method.

The unsupervised feature vector learning unit 100 places the extracted feature vectors in the input node I1 and the output node O1 of the stacked autoencoder as shown in Fig. 5A to unmap the feature vector. Can learn. Such an accreditation autoencoder can learn feature vectors by increasing the number of intermediate nodes by one layer.

That is, the unsupervised feature vector learning unit 100 symmetrically sets the feature vectors generated using the mel-scale filter bank at the inputs I1 and O1 of the multi-layer auto-encoder, and the intermediate nodes (hidden nodes). Feature vectors can be unsupervised by learning all connected weight matrices W1, W2, W3, and W4. Where W 'is the transpose matrix of W.

In addition, the unsupervised feature vector learning unit 100 places the extracted feature vectors in the input node I2 and output node O2 of the convolutional autoencoder as shown in FIG. You can learn unsupervised.

Such a convolutional autoencoder is a method used in image pattern analysis, and is composed of a convolutional node sharing a weighting matrix according to a weighting direction, and intermediate layers having a weighting matrix for fully connected nodes.

The unsupervised feature vector learning unit 100 symmetrically sets the feature matrix generated by grouping the spectrogram in two-dimensional units at the input (I2) and output (O2) of the composite auto-encoder, and the weight matrix connected to the intermediate node. Can learn ( W1 , W2 , W3 , W4 ). Where W ' is the transpose matrix of W.

In addition, the unsupervised feature vector learning unit 100 crosses the feature vector extracted using a cross auto-encoder combining the above-described stacked auto-encoder and convolutional auto-encoder, as shown in FIG. 7 (a). By placing the input node (I3) and the output node (O3) of the non-visible can be learned. When such a cross autoencoder is used, the combined intermediate node can aggregate characteristics for input data of the two characteristics described above.

Referring to FIG. 2 again, the unsupervised feature vector learning unit 100 generates an artificial neural network including an acoustic pattern corresponding to the feature vector based on the results of the unsupervised learning (S130).

That is, the network including the weight matrix trained by the unsupervised feature vector learning unit 100 may be generated by artificial neural networks as shown in FIGS. 5 (b), 6 (b), and 7 (b). have. Such an artificial neural network may output an output value in a form in which a corresponding acoustic pattern is distinguished with respect to a feature vector input to input nodes I1, I2, and I3 to the final nodes P1, P2, and P3.

2. Automatic phoneme clustering steps

After the unsupervised learning step is performed by the unsupervised feature vector learning unit 100, the clustering unit 200 generates a phoneme set by clustering selected acoustic characteristics based on the unsupervised learning results (S210).

Specifically, when the artificial neural network generated as a result of the unsupervised learning is transmitted from the unsupervised feature vector learning unit 100, the clustering unit 200 may generate a phoneme set by listing output values for every input data of the artificial neural network.

At this time, the clustering unit 200 lists and outputs the output values for each input data as one vector, and the distance between vectors is less than or equal to a specific boundary among the vectors listed based on top-down or bottom-up vector clustering. Extract them. Then, a group vector is generated by averaging the extracted vectors, and then a phoneme set can be generated based on the listed vectors and the generated group vector. According to this method, a final phoneme set may be generated using a method of distinguishing a boundary between a vector value representing a phoneme and a neighboring phoneme.

As another method, the clustering unit 200 sets the output node itself of the artificial neural network as one phoneme candidate, and places an active function such as softmax at each node, corresponding to every input data. You can create a phoneme set by listing the index of an output node as an output value.

That is, the clustering unit 200 lists the index of the output node as an output value for each input data, and performs clustering based on an index among output indexes having a predetermined number of times or more. This is a method of combining an index having a low frequency of occurrence with an adjacent index having a high frequency of occurrence in an index column generated on a time axis, and thus a phoneme set can be generated according to the above method.

After generating the phoneme set by the above two methods, the clustering unit 200 allocates a phoneme string to the voice data based on the generated phoneme set (S220). At this time, the clustering unit 200 lists candidate phoneme sequences using an artificial neural network, extracts a final phoneme sequence divided into clustering boundaries based on the generated phoneme set and candidate phoneme sequences, and allocates the final phoneme sequence to the voice data can do.

3. Crystal phoneme-based acoustic model creation step

After the automatic phoneme clustering step is performed by the clustering unit 200, the acoustic model generator 300 generates an acoustic model based on the phoneme string and phoneme string to which the phoneme string is assigned. The acoustic model generation step uses a GMM-HMM (Gaussian mixture model-hidden Markov model) model, a DNN-HMM (Deep neural net-hidden Markov model) model alone, or a CNN (Convolutional neural network), Recurrent neural networks (RNNs) or mixed models of these are also available.

Specifically, the acoustic model generator 300 generates a context independent phoneme string model by using the phoneme string and the phoneme string to which the phoneme string is reassigned (S310). This has the same meaning as the conventional monophone learning, and the acoustic model generator 300 models the distribution or pattern of the phoneme set determined by the clustering unit 200.

Next, the acoustic model generator 300 generates a context-dependent tree based on a combination of a context-independent phoneme string model and a phoneme string context (S320). That is, the acoustic model generator 300 may generate a context-dependent tree in consideration of the context with other adjacent phonemes, which clusters in a direction to reduce the entropy of learning data for all context-dependent phonemes that can be generated. It means to do.

Next, the acoustic model generator 300 defines a context-dependent phone state for the context-dependent phoneme based on the generated context-dependent tree (S330), and defines the voice data by using a phoneme string. The allocated context-dependent state is assigned (S340).

Then, the acoustic model generator 300 trains the context-dependent state models by using a deep learning method based on the assigned context-dependent state information and voice data, which is learning data (S350). At this time, the acoustic model generator 300 reassigns the phoneme string to the voice data used as the learning data by using the learned context-dependent state models, and the context-dependent state information and voice data derived from the reassigned phoneme string. Based on this, context-dependent states can be retrained. As such, one embodiment of the present invention can gradually and accurately generate a context-dependent state model by performing repetitive learning.

4. Pronunciation dictionary generation stage

8 is a flowchart of a method for generating a pronunciation dictionary and a language network.

In general, a speech recognition system processes a speech signal using a language network and an acoustic model to which a language model is applied. The existing language network is generated using a language model made of a training corpus and a pronunciation dictionary generated by converting a training corpus into a G2P (Grapheme to Phoneme) transformation.

On the other hand, the speech recognition system 1 according to an embodiment of the present invention may generate a space-unit pronunciation dictionary based on the speech data in which the transcription is present, as shown in FIG. The pronunciation dictionary can be expanded by replacing syllable-unit connections.

In addition, a language network can be generated by linking the generated pronunciation dictionary with a language model made of a learning corpus. In this case, since the pronunciation dictionary is generated in units of word units, in the case of words, speech is a unit that continues at a time, so it can be expressed by including various pronunciation variation phenomena occurring in continuous speech in natural language.

Specifically, as illustrated in FIG. 8, the pronunciation dictionary generating unit 400 divides the transcription speech data into sections of a word unit through measurement of a signal pitch and an energy section (S410).

In this case, the word unit means a group of one or more words that are a space or a reading unit. For example, Korean words can be grouped into nouns and investigations, or they can consist of single nouns. In the alphabetic language, words are mainly words, but short words or short phrases that are spoken continuously can be treated as words. In addition, in Japanese, Chinese, etc. where there are no spaces, it is possible to read spaces that are meaningful bundles.

On the other hand, in general, when the spacing of the transcription statement coincides with the spacing reading, the transcription statement can be easily assigned to a speech signal using a pitch and an energy section in units of words. However, for languages in which spaces are different from space reads or there are no spaces, other voice recognizers may be used to force-align the transcription to the time axis of the voice signal.

Next, the pronunciation dictionary generating unit 400 allocates a phoneme string to the transcription voice data divided by word units (S420), and arranges a phoneme sequence corresponding to the word of the transcription voice data to which the phoneme string is assigned (S430). . At this time, the phonological phoneme string corresponding to the word is slightly different for each utterance, and can be uttered differently for each utterance, so multiple pronunciations are acquired in a number of cases when measuring the phoneme fever.

Next, the pronunciation dictionary generating unit 400 refines the sorted phoneme string based on the time axis and the number of frames (S440). At this time, the phonetic dictionary generating unit 400 may refine the phoneme string by merging phonemes that are repeated on the time axis from phoneme strings in units of frames, and removing phonemes generated in too few frames.

After the phoneme string is sorted and refined as described above, the pronunciation dictionary generating unit 400 generates a phoneme string pronunciation dictionary based on the refined phoneme string (S450).

At this time, the pronunciation dictionary generating unit 400 may generate a pronunciation dictionary of partial words or syllable units generated by dividing words of transcription speech data for interworking with words that do not match the word unit. The pronunciation dictionary of the partial word or syllable unit is collected by collecting the pronunciation units of the word unit including the letters corresponding to the target subword or syllable, and comparing and extracting phoneme strings of the common character, and then generating the average occurrence phoneme or multiple phonemes. Can be created by reassignment.

5. Language network generation step

When the pronunciation dictionary is generated by the pronunciation dictionary generation unit 400, the language network generation unit 500 may generate a language network by linking the word-based language model generated by the learning corpus with the pronunciation dictionary (S510). (S520).

In this case, the word-based language model can be generated by changing from a semantic morpheme unit based on an existing morpheme analysis to a word-based unit bundle. This is to estimate the probability of vocalization using word-based statistics, which means that the word unit will be processed as the basic recognition unit. This method means attempting speech recognition centered on a word, which is a bundle of speech, similar to a meaning that is a bundle that a person learns and recognizes.

On the other hand, the language network generator 500 may expand the pronunciation dictionary of the word unit by interworking with the pronunciation dictionary of the partial word or syllable unit among words in the language model that are not included in the word unit pronunciation dictionary.

As such, the language network generation unit may generate a language network by linking the language model with the generated or expanded pronunciation dictionary. At this time, according to an embodiment of the present invention, since the word unit is one or more words, a language model composed of one word may be combined.

After the language network is generated in this way, the speech recognition decoder 700 may generate a speech recognition result using the acoustic model and the language network.

Specifically, when the feature vector extracting unit 600 extracts the feature vector from the voice data input by the user, the voice recognition decoder 700 receives the feature vector as an input by applying an acoustic model and a language network, and inputs the result. On the basis of this, a word sequence of speech data may be extracted to generate a speech recognition result.

In this case, the feature vector extracting unit 600 may extract the feature vector as described in FIG. 4, and of course, the feature vector may be extracted by a method different from FIG. 4.

Meanwhile, in the above description, steps S301 to S521 may be further divided into additional steps or combined into fewer steps according to an embodiment of the present invention. In addition, some steps may be omitted if necessary, and the order between the steps may be changed. In addition, even if omitted, the contents already described in FIGS. 1 to 2 may be applied to the management platform registration method of FIGS. 3 to 5.

According to any one of the exemplary embodiments of the present invention, the pronunciation dictionary is generated based on the uttered data out of the limitations of the existing G2P generating a pronunciation dictionary based on a rule and a dictionary, so that the observed variable pronunciation is dictionary. As it can be reflected in, the difference in pronunciation between the pronunciation conversion rule and the actual spoken voice can be reduced, thereby overcoming the limitation of free speech recognition.

In addition, since a phoneme is determined from voice data and an acoustic model is generated using information about the phoneme, there is an advantage in that acoustic modeling is possible only with voice data, since it is difficult to collect a large amount of transferred voice data.

In addition, since a pronunciation dictionary is generated by distinguishing the uttered transcripts, unlike a conventional pseudo morpheme or word used as a speech recognition unit, speech recognition is based on a semantic element corresponding to a high-frequency word or phrase that is a bundle of speech. The unit may be reconstructed to provide speech recognition performance in a form more similar to human recognition.

In addition, by using the transcribed voice data, it can be adapted to a local dialect or expanded to a pronunciation dictionary adapted to an individual user's speech style.

In addition, in the case of a foreign language pronunciation conversion rule, it is an inaccessible factor for non-native speakers to become multi-lingual expansion, but according to an embodiment of the present invention, multi-language expansion is easy.

In addition, if a new vocabulary is to be added to the speech recognition system, it is possible to easily add it by inputting a word and speaking it.

On the other hand, the speech recognition method in the speech recognition system 1 according to an embodiment of the present invention may also be implemented in the form of a computer program stored in a medium executed by a computer or a recording medium including instructions executable by a computer. Can. Computer readable media can be any available media that can be accessed by a computer and includes both volatile and nonvolatile media, removable and non-removable media. In addition, computer readable media may include both computer storage media and communication media. Computer storage media includes both volatile and nonvolatile, removable and non-removable media implemented in any method or technology for storage of information such as computer readable instructions, data structures, program modules or other data. Communication media typically include computer readable instructions, data structures, program modules, or other data in a modulated data signal such as a carrier wave, or other transport mechanism, and includes any information delivery media.

Although the methods and systems of the present invention have been described in connection with specific embodiments, some or all of their components or operations may be implemented using a computer system having a general purpose hardware architecture.

The above description of the present invention is for illustration only, and those of ordinary skill in the art to which the present invention pertains can understand that it can be easily modified into other specific forms without changing the technical spirit or essential features of the present invention. will be. Therefore, it should be understood that the embodiments described above are illustrative in all respects and not restrictive. For example, each component described as a single type may be implemented in a distributed manner, and similarly, components described as distributed may be implemented in a combined form.

The scope of the present invention is indicated by the following claims rather than the above detailed description, and it should be interpreted that all changes or modified forms derived from the meaning and scope of the claims and equivalent concepts thereof are included in the scope of the present invention. do.

1: Speech recognition system
10: memory
20: processor
100: unsupervised feature vector learning unit
200: clustering unit
300: acoustic model generator
400: pronunciation dictionary generator
500: language network generator
600: feature vector extraction unit
700: speech recognition decoder

Claims (19)

In the speech recognition method capable of automatic phoneme generation,
Unsupervised learning a feature vector of speech data;
Generating a phoneme set by clustering selected acoustic characteristics based on the unsupervised learning result;
Assigning a phoneme string to the voice data based on the generated phoneme set;
Generating an acoustic model based on the phoneme string to which the phoneme string is assigned and the phoneme string; And
Generating a result of speech recognition through a speech recognition decoder to which the acoustic model and language network are applied
Speech recognition method comprising a. According to claim 1,
The voice data is non-transcription voice data. According to claim 2,
Unsupervised learning the feature vector of the speech data,
Extracting the feature vector from the speech data;
Unsupervised learning the extracted feature vector, and
And generating an artificial neural network including an acoustic pattern corresponding to the feature vector based on the unsupervised learning result. The method of claim 3,
Extracting the feature vector from the speech data,
Converting the voice data into a spectrogram;
Generating a first feature vector by converting the speech data converted into the spectrogram into a mel-scale filterbank in a predetermined time frame unit, and
And splicing a window corresponding to a predetermined number of frames to the left and right of the first feature vector to generate a second feature vector.
The speech recognition method of extracting the generated second feature vector as the feature vector. The method of claim 4,
Non-supervised learning the extracted feature vector,
A speech recognition method in which the feature vector is unsupervised learning by placing the extracted feature vector at an input node and an output node of a stacked autoencoder. The method of claim 3,
Extracting the feature vector from the speech data,
Converting the voice data into a spectrogram, and
The step of generating a feature matrix by grouping the speech data converted into the spectrogram into two-dimensional units based on x frames,
The speech recognition method of extracting the generated feature matrix into the feature vector. The method of claim 6,
Non-supervised learning the extracted feature vector,
A voice recognition method for unsupervised learning of the feature vectors by placing the extracted feature vectors at input and output nodes of a convolutional autoencoder. The method of claim 3,
The step of generating a phoneme set by clustering selected acoustic characteristics based on the unsupervised learning result,
The speech recognition method of generating the phoneme set by listing output values for every input data of the artificial neural network. The method of claim 8,
Generating the phoneme set,
Expressing and listing output values for each input data as vectors;
Extracting vectors in which the distance between vectors is less than or equal to a specific boundary based on vector clustering among the listed vectors;
Generating a group vector by averaging the extracted vectors, and
And generating the phoneme set based on the listed vector and the generated group vector. The method of claim 8,
Generating the phoneme set,
Listing the index of the node as an output value for each input data, and
And generating the phoneme set by performing the clustering around an index whose output frequency is greater than or equal to a preset number of indexes. The method of claim 8,
The step of assigning a phoneme string to the voice data,
Listing candidate phoneme strings based on the artificial neural network, and
And extracting a final phoneme sequence based on the generated phoneme set and the candidate phoneme sequence and assigning the final phoneme sequence to the voice data. The method of claim 11,
Generating the acoustic model,
Generating a context-independent phoneme string model using the phoneme string re-assigned and the phoneme string;
Generating a context-dependent tree based on the context independent phoneme string model and a combination according to the context of the phoneme string;
Defining a context dependent state for a context dependent phoneme based on the context dependent tree;
Assigning the defined context-dependent state to the speech data using the phoneme string, and
And learning the context-dependent state based on the assigned context-dependent state information and the speech data. The method of claim 12,
The step of learning the context-dependent state,
Reassigning the learned context dependent models to the speech data, and
And learning the reassigned context dependent state based on the information of the reassigned context dependent state and the speech data. According to claim 1,
Further comprising the step of generating a pronunciation dictionary of the word unit based on the transcriptional speech data,
The step of generating the pronunciation dictionary,
Dividing the transcription speech data into sections of word units;
Allocating the phoneme string to the transcription speech data divided by the word unit;
Sorting a phoneme string corresponding to a word of the transcription voice data to which the phoneme string is assigned;
Purifying the sorted phoneme string based on the time axis and the number of frames; and
And generating a pronunciation dictionary of the word unit based on the refined phoneme string. The method of claim 14,
The step of generating a pronunciation dictionary of the word unit may include:
A speech recognition method for generating a partial dictionary or a pronunciation dictionary in units of syllables generated by dividing the words of the transcriptional speech data. The method of claim 15,
Linking the word-based language model generated by the learning corpus with the generated pronunciation dictionary, and
And generating the language network based on the interworking result. The method of claim 16,
The step of linking with the pronunciation dictionary,
The words of the language model, the words that are not included in the pronunciation dictionary of the word unit include the step of expanding the pronunciation dictionary of the word unit in conjunction with the pronunciation dictionary of the partial word or syllable unit. The method of claim 16,
Extracting a feature vector from speech data input by the user;
Inputting the feature vector into the speech recognition decoder to which the generated acoustic model and the language network are applied, and
And extracting a word sequence of the input speech data based on the input result to generate the speech recognition result. In the speech recognition system capable of automatic phoneme generation,
Memory for storing programs for speech recognition and
It includes a processor for executing a program stored in the memory,
As the processor executes the program, the feature vector is extracted from the non-transcriptional speech data to perform unsupervised learning, and clusters selected acoustic characteristics based on the unsupervised learning results to generate a phoneme set, A phoneme string is assigned to the voice data based on the generated phoneme set, an audio model is generated based on the phoneme data to which the phoneme string is assigned, and the phoneme string, and a speech recognition decoder to which the sound model and language network is applied is generated. A voice recognition system that generates sounding recognition results through.

Images