RU2466468C1 - System and method of speech recognition - Google Patents

Abstract

FIELD: information technologies. ^ SUBSTANCE: system and method are used for speech recognition, which receive a speech signal at the inlet of a reception unit; process the speech signal with an information processing unit, including its processing with an analogue-to-digital converter with preset digitisation frequency and separation into segments, spectral analysis of speech signal segments and normalisation of spectrum at high frequencies; identify pauses, noise and speech signals in the normalised spectrum. Then on the basis of the initial speech signal and the normalised spectrum availability/absence of acoustic criteria of the speech signal is identified in each segment, combinatory sets of which are compared with preset parameters of phoneme groups in the memory unit, and based on comparison results a sequence of symbols is generated to indicate groups of phonemes corresponding to combinatory sets of acoustic criteria of each segment, conversion of which into a cohesive text is carried out with serial decoding of a combinatory combination of symbols of sequence phoneme groups on the basis of a dictionary marked by symbols of phoneme groups. ^ EFFECT: reduced duration and high accuracy of speech recognition. ^ 19 cl, 5 dwg, 3 tbl

Description

The invention relates to speech recognition technologies, i.e. systems and methods for translating an audio signal containing speech into text consisting of words included in the lexical and pronunciation dictionaries of a speech recognition system.

Field of application of the invention: voice control of electronic devices, automatic shorthand, special programs for visually impaired, voice control of moving means at a distance of audible voice.

The claimed invention allows to recognize continuous continuous speech, regardless of the individual characteristics of the speaker, based on the definition of phoneme groups according to their characteristics and the method of sequentially decoding sequences of characters denoting phoneme groups into a chain of words that make up a statement (text).

A technical solution is known that characterizes the method of speaker-independent recognition of speech sounds, including preliminary segmentation of the speech signal to determine the time duration of the sound segments, determining the periodicity of each segment of the acoustic components of the speech signal to correlate the sound segment by the method of its formation to voice, noisy or noisy voice form speech sounds, determining the amplitude and frequency of each of the first three formants in the spectrum of the sound segment as informative features of speech sounds, integration of the mentioned informative features for each sound segment, phonemic recognition of each sound segment by comparing the integral values of its informative features with the existing data bank separately for each type of speech sounds, making a decision on the recognizable speech sound and presenting it in letter or transcriptional notation. The main segmentation of the speech signal is performed according to three main modes depending on the previously found type of sound segment, with the mentioned phonemic recognition, the integrated values of the informative features of each sound segment are compared both for each mentioned type of speech sounds and for each type depending on the number of formants in the sound segment, then set the temporal boundaries of speech sounds depending on changes in the phonemic affiliation of the audio segment, after which they take the decision statement regarding the recognizable sound of speech (patent for the invention of the Russian Federation 2234746, G10L 19/02, 10.30.2002). The disadvantages of this solution include the low discriminating ability and speed of speech recognition by formants, the progressive nature of recognition, which determine the consistent recognition of each formant, as well as the need to use dictionaries and reference samples in the recognition process.

There is also a technical solution to the method for recognizing words in continuous speech, consisting in the fact that with the utterance of the speech utterance, samples of the acoustic signal of this utterance digitized with a given quantization frequency are periodically taken at fixed time intervals and, based on the totality of these samples, the functional determining the current acoustic state is calculated while the resulting sequence of current acoustic states is used to restore the sequence of words (working hypothesis), p spoken in the original speech utterance, for which a lexical decoding network is used, which sets the patterns for following reference acoustic states in a language. In this case, a search for a working hypothesis is carried out, which is optimal in the sense of the maximum degree of its coincidence with the original speech signal, which is ensured by the use of the moving marker algorithm, and the working hypothesis is restored from the marker, which at this point in time is at the final vertex of the lexical decoding network (patent for the invention RF 2297676, G10L 15/02, 03/30/2005) Despite the fact that in this method the distinguishing ability is higher than in the previous method, however, it is similar to the previous known method of detection Speech skills are among the disadvantages of this method should also include the duration of the recognition process, due to the need to refer to the reference samples, as well as the alternate recognition of each formant in the word.

There is also known a method and a speech recognition system constructed using phonemic analysis methods (US patent No. 5315689, IPC G10L 5/06, 1995), in which two-level processing of the speech signal is used. The first level block recognizes a word (command) as a sound (auditory) image as a whole. An alternative block of the second level produces phonemic recognition of the audio signal. The disadvantage of this method is the reduction in the likelihood of correct recognition of words (phrases) with an increase in the volume of a speech fragment and recognition of continuous speech.

There are also known solutions to a speech recognition system and method (application for US invention US 2010332231 A1, G10L 15/04, 06/01/2010), consisting in the fact that from a single speech at the first stage a sequence of phonemes to be recognized is determined, which are then compared with the stored in the device’s memory with a list of words corresponding to the selected phonemes, a probabilistic assessment is then carried out, according to previously established criteria, based on which the most probable words are selected from the previously generated word, and the unfamiliar word is entered into the dictionary determine the criteria for the subsequent probability estimate. The disadvantages of this method include its excessive complexity and high requirements on the memory resources of a device that performs speech recognition in accordance with this method, in addition, the solution does not allow recognition of continuous speech, since recognition is too slow and with a sufficient degree of accuracy, it is possible only the definition of individual speech commands, and not continuous speech.

There is also known a solution to a speech recognition system (US patent US 4,624,011 A, G10L 5/00 01/28/1983), in which speech recognition is carried out first by determining the frequency spectra of speech, and then by determining the sequence of phonemes and their acoustic characteristics emitted by the auxiliary module , which are then compared with the reference characteristics stored in the memory for the above sequences, and the degree of similarity is calculated, and if the predefined threshold is exceeded, a conclusion is made about the correspondence phonemes to the standard, and if the threshold value is not reached, they introduce new words into the dictionary. The disadvantages of this method of speech recognition and the system for its implementation are similar to the previous solution.

A known method and system for speech recognition in which (US patent US 4696042, G10L 5/00, 11/03/1983) speech recognition is carried out by linguistic phonemes pre-distributed into groups corresponding to the characteristic features of the pronunciation of sounds. Moreover, the sound is determined, among other things, by the characteristic harmony with the following sound, which allows to increase the distinguishing ability and quality of recognition, however, it requires a rather long time for sequential processing of both a single sound and a diphthong.

The closest analogue in terms of essential features recognized as a prototype is the solution of a speech recognition system and method known from US Pat. No. 4,852,170 (G10L 5/04, 12/18/1986), characterized by a real-time digital voice recognition system including a spectrum analyzer that determines the particular characteristic of each speech segment of a given duration, while each speech segment is analyzed logically for the presence of phonemes and their belonging to a particular class, part of which they are, and then the frequency spectrum of the segment is analyzed for features that allow the recognition of specific phonemes within the type. The sequence of phonemes can be stored in compact groups and then transformed to synchronize with the voice and logically translated from one language to another. The disadvantages of this method and system of speech recognition should include the phonon recognition of sounds within the allocated speech segment, which requires significant time and a significant increase in hardware and memory resources to ensure the required system speed, the inability to implement a system based on compact devices. The need to focus on speech patterns by different age categories, taking into account the peculiarities of their spectral characteristics, distinguished by various criteria, as in previous cases, either leads to a loss of quality or to a significant increase in recognition time.

The technical task of the claimed invention is to propose a method that provides speaker-independent recognition of continuous speech in real time in combination with a compact system for its implementation.

The technical result achieved by the claimed invention is to reduce the duration of speech recognition in combination with high accuracy, provided regardless of the specific features of different language systems.

The claimed technical result is achieved by using a speech recognition method, which includes sequentially executed steps for receiving a speech signal at the input of a receiving unit; processing the speech signal by the information processing unit, including its processing by an analog-to-digital converter with a pre-set sampling frequency and segmentation, spectral analysis of the speech signal segments and normalization of the spectrum at high frequencies; highlighting in the normalized spectrum of pauses, noise and sound signals with its subsequent recognition and conversion into text using a predefined dictionary. Moreover, it differs from the prototype in that at the recognition stage, based on the initial speech signal and the normalized spectrum in each segment, the presence / absence of acoustic features of the speech signal is determined, combinatorial sets of which characterize groups of phonemes whose parameters are predefined in the memory unit, and they compare certain combinatorial ones sets of acoustic features of a segment with predefined parameters of phoneme groups, with the simultaneous formation of a sequence of characters, I mean groups of phonemes corresponding combinatorial sets of acoustic characteristics of each segment, wherein the conversion is performed in a coherent text sequential combinatorial decoding combining groups of symbols phoneme sequence based on the vocabulary of the markup symbols phoneme groups.

In a preferred embodiment of the invention, the determination of the combinatorial set of acoustic features of a segment of a speech signal is carried out simultaneously and simultaneously.

In one embodiment of the invention, at the stage of dividing the speech signal into segments, the speech signal is processed by short windows of the same length with an offset of half the length, with the possibility of registering short-term characteristic phenomena within speech sounds and smooth transitions from one sound in the speech stream to another.

When classifying phoneme groups, a combinatorial set of acoustic features is used, including determining the presence or absence in the speech signal of at least the fundamental tone, broadband noise, the difference in the intensity of the speech signal, high-frequency noise, sonority, vowel, vowel, a combinatorial combination of which forms 11 groups phonemes with a unique set of acoustic features: voiced deaf noisy consonants, voiced voiced noisy consonants, deaf noisy slotted consonants, voiced noisy slotted accords waistband, voiceless sibilants, voiced sibilants, nasal and slotted sonants, trembling sonants, front vowels, vowels mixed-row and back vowels. And the conversion of the sequence of characters denoting phoneme groups into text is carried out on the basis of the sequential decoding method, which consists in sequentially reducing options in the selection of words and word forms from the dictionary marked up by the characters of phonemes, formed by the combinatorial sequence of characters of phoneme groups from the first character in the segment to a fragment of a sequence including a combination of symbols of phoneme groups defining a single word from the dictionary, after which recognition The next input sequence begins, starting with the first character of the phoneme group after the word in the previously defined word.

At the same time, in one embodiment of the claimed invention, a dictionary is formed based on words written in transcription with pronunciation options consisting of phonememes marked out by symbols.

The dictionary can be executed pre-installed in the memory block, with the possibility of its initialization at the recognition stage, and additionally loaded into the memory block as a file with a list of words with their corresponding transcriptions marked up by phoneme group symbols. At the stage of decoding a sequence of symbols denoting phoneme groups, one or a group of predefined thematic dictionaries marked up by the symbols of phoneme groups and different language systems can be used in the text. The thematic dictionary or a group of dictionaries marked up by the symbols of phoneme groups can be selected from the list of predefined thematic dictionaries at the initial stage of the speech recognition method.

The claimed technical result is also achieved by the fact that for the implementation of the method, a speech recognition system is used, including a speech signal receiving unit connected in series with an analog-to-digital converter, the output of which is connected to the analysis and normalization unit of the frequency spectrum of the speech signal connected to the control unit and the memory unit made with the possibility of recording and storing the recognized speech signal, storing and calling the dictionary. Moreover, the speech recognition system according to the invention differs from the prototype in that it further comprises a series-connected unit for determining the presence of acoustic features and the formation of a sequence of characters representing groups of phonemes, made possible by the current combinatorial combinations of acoustic features present in the speech signal corresponding to them character sequences denoting phoneme groups, a character sequence decoding unit, about signifying groups of phonemes, into the text, at least one of the inputs of which is connected to the input unit of a predefined dictionary marked up by the symbols of phoneme groups and called from the memory block, made possible to define words based on the dictionary by combinatorial combination of phoneme group symbols of the current sequence , as well as a device for outputting and storing text, at least one input / output of which is connected to a control unit configured to control the operating modes b shackles, data exchange therebetween and interacting with the memory unit. In this case, the input of the unit for determining the presence of acoustic signs and the formation of a sequence of characters denoting phoneme groups is connected to the output of the analog-to-digital converter, providing the possibility of inputting a speech signal, and the output of the unit for analyzing and normalizing the private spectrum of the speech signal, making it possible to input a normalized speech signal , and a block of acoustic signs of phoneme groups, with the possibility of comparing the current combinatorial set of acoustic signs of a speech signal with the parameters of a set of acoustic signs of predefined phoneme groups, and connected to a memory unit. The text output and storage device is further provided with a display for outputting and displaying the recognized text on the screen.

Moreover, in one embodiment of the invention, the block of acoustic signs of phoneme groups contains a combinatorial set of acoustic signs of 11 groups of phonemes: voiced deaf noisy consonants, voiced voiced noisy consonants, deaf noisy slotted consonants, voiced noisy slotted consonants, deaf sibilants, voiced sibilants, nasal and slotted sonants, trembling sonants, front row vowels, mixed row vowels and back row vowels, including for each of the phoneme groups at least a window presence / absence characteristic pitch broadband noise, the differential intensity of a speech signal of high frequency noise, sonority, vowel, vowel series.

The claimed invention is illustrated by the drawings shown in figures 1-3, where figure 1 is a structural diagram of a speech recognition system, figure 2 is a functional diagram of speech recognition, including the steps of determining acoustic features, forming a sequence of characters representing groups of phonemes, and its decoding into the text, FIG. 3 — an example of the formation of a sequence of characters defining phoneme groups: 3a) —the oscillogram with the marking of the speech signal into allophones; 3b) - spectrogram with markup for allophones; 3c) is an illustration of the definition of acoustic characteristics based on the waveform of the normalized speech signal and the formation of a sequence of symbols defining phoneme groups. This example describes a particular implementation of the system and method in accordance with the invention and is not an exhaustive description of their possible implementations.

In the presented implementation example, the speech recognition system according to the invention is implemented on a digital device representing the architecture of a computer under the control of a central microprocessor with operational (RAM) and permanent (ROM) storage devices. As follows from the diagram shown in figure 1, the speech recognition system contains a block 1 for receiving a speech signal (sound receiver), which can be used as a microphone, either built-in or external, of any type and model known from the prior art, in series connected to analog-to-digital Converter 2 with a sampling frequency of at least 22 kHz, converting the speech signal into digital form. The control unit 4, made on the basis of a multi-channel central microprocessor, is connected via standard interface, multiplex communication channels with a unit 3 for analyzing and normalizing the frequency spectrum of the speech signal, unit 5 for determining acoustic characteristics and generating a sequence of characters representing phoneme groups, unit 6 decoding a sequence of characters denoting phoneme groups into text and a text output and storage device 7, as well as with a memory unit 11. Block 5 for determining the presence of acoustic signs and the formation of a sequence of characters denoting phoneme groups is additionally connected to the output of the analog-to-digital converter 2, providing the possibility of transmitting a speech signal side 5 to the input, as well as with block 9 of acoustic signs of phoneme groups, allowing comparison current combinations of acoustic features of a speech signal with predefined combinatorial sets of acoustic features of phoneme groups. At the same time, the unit 6 for detecting the sequence of symbols denoting phoneme groups is connected to the text via interface communication channels with the output of the vocabulary module 10, the input of which is connected to the memory unit 11 with the possibility of recalling from the memory unit any predefined thematic dictionary or group of dictionaries marked by symbols phoneme groups, and selecting words corresponding to combinations of phoneme group symbols in the current sequence. The memory unit 11 through the control unit 4 is connected to a text output and storage device 7, one output of which is connected to a recognized text output device made, for example, in the form of a display device 8. As the text output device 7, a printing device and / or an external storage device may be used. As the display device 8, any device known in the art used in computer or telecommunication devices can be used. The system blocks and functional connections between them are made in a hardware-software manner, with the possibility of configuring and adapting them to the used computer and multimedia tools.

The system and method of speech recognition according to the invention work as follows.

The speech signal in the form of an audio data stream is fed to the input of the speech signal receiving unit of the speech recognition system and is then transmitted to the input of an analog-to-digital converter, where the signal is converted to digital form. The resulting digital speech signal is transmitted to the input of the speech signal analysis and normalization block, where it is segmented when processing with short windows of the same length and with an offset of half the length, which allows to detect both smooth transitions from one sound in the speech stream to another, and short-term characteristic phenomena inside speech sounds, for example, explosions of consonant consonants. Processing windows are selected in length so as to obtain the most optimal and smoothed features of phoneme groups by the time they sound in the speech stream. It is empirically established that a window length of 25 ms gives an optimal result. The acoustic features of a speech signal characteristic of phoneme groups used as basic elements for recognition are determined simultaneously and simultaneously within each window (segment of the speech signal). In this case, part of the acoustic characteristics is determined directly from the waveform of the speech signal received from the analog-to-digital converter, and part is determined from the spectrum of the speech signal obtained by processing the speech signal in the analysis and normalization unit using the fast Fourier transform. The resulting spectrum is normalized at high frequencies in accordance with the nonlinear perception of different frequencies by the human auditory system, which allows you to compensate for the lower intensity of high frequencies compared to low frequencies in the speech signal.

As indicated above, to determine the acoustic characteristics of a speech signal in each window, both the original speech signal and the normalized spectrum are used. Based on combinations of values of acoustic features, a group of phonemes is determined to which the speech signal belongs within the current processing window.

In this case, the procedure for extracting acoustic features used to determine phoneme groups from a signal according to the invention is carried out within each window (segment of a speech signal) in parallel and simultaneously, in a hardware-software manner. Since the classification of phoneme groups uses the following set of acoustic features that make it possible to unambiguously identify each of the 11 phoneme groups considered in the framework of the invention: the presence / absence of the fundamental tone, the presence / absence of broadband noise, the presence / absence of a difference in the intensity of the speech signal, the presence / absence of high-frequency noise , presence / absence of sonority, an acoustic sign of the presence / absence of a vowel, an acoustic sign of a number of vowels, then the analysis of the speech signal in each window is carried out The presence / absence of acoustic features inherent in phoneme groups in a speech signal or normalized spectrum is considered.

One of the most important acoustic characteristics is the presence of the fundamental tone in the speech signal. The absence of the fundamental tone in the signal indicates that at the given moment either a dull consonant is pronounced or there is a break in speech (pause). The presence of the fundamental tone is determined by the high intensity of the frequency components in the low frequency region in the range of possible values of the frequency of the fundamental tone. The intensity of the frequency components in the current window determines their relative maximum intensity in the speech signal over a relatively long length of the speech signal for about 5 seconds. If the speech signal within the previous processing window detected the absence of the fundamental tone and broadband noise, and one of the other signs was determined in the speech signal in the current window, then this window is additionally checked for the presence of broadband noise in it, which is a characteristic of the group phonetic deaf noisy consonants or phonetic voiced noisy consonants.

Short-term differences in the intensity of the speech signal, indicating the presence of short bows in the signal, characteristic of trembling sonants, are determined by the ratio of the intensity of the speech signal in three consecutive processing windows. The intensity of the speech signal in the middle window is significantly lower than the intensity of the speech signal in the right and left windows, while the intensity of the speech signal in the right and left windows is almost the same.

The presence of broadband noise in a speech signal associated with the pronunciation of slotted consonants or the presence of an explosion occurring during opening the bow when pronouncing the consonant consonants is determined by the presence of intense frequency components in the range above the possible values of the fundamental frequency and its first harmonic. The intensity of the frequency components in the current window determines their relative maximum intensity in the speech signal over a relatively long length of the speech signal for about 5 seconds.

The presence of high-frequency noise in a speech signal associated with the pronunciation of slotted sibilants is determined in the range above the possible values of the frequency of the fundamental tone and its first harmonic, in relation to the intensity of the frequency components in the middle frequency region and the intensity of the frequency components in the high frequency region. The intensity of high-frequency noise significantly exceeds the intensity of medium frequencies in the case of pronouncing slotted sibilants.

The sonority of a speech signal, characteristic of pronouncing sonants and vowels, as opposed to noisy consonants, is determined by the high intensity of the frequency components in the mid-frequency range above the low-frequency region in the range of possible frequencies of the fundamental tone, but containing the range of possible frequencies of the formants of the sonants. The intensity of the frequency components in the current window is considered relative to their maximum intensity in the speech signal over a relatively long length of the speech signal for about 5 seconds.

Another acoustic feature used in speech recognition and for characterizing phoneme groups is the absence or presence of harmonic components in the spectrum in the frequency domain above the range of possible frequencies of the formants of the sonants. The absence of harmonic components in the mid and high frequencies is characteristic of the sonants, and the presence of vowels. The presence or absence of harmonic components is determined by the ratio of the intensity of the frequency components below and above the frequency threshold.

Another important acoustic characteristic of speech sounds is the quality of a vowel that has been pronounced, namely, the number of pronunciations, i.e. the position of the bulk of the tongue in the oral cavity in a horizontal position. A number of vowel pronunciations is determined by the ratio of the intensity of the harmonic components in the spectrum of the speech signal in the low frequency region, the middle frequency region and the high frequency region. The absence of harmonic components in the spectrum of the speech signal in the mid-range and high-frequency region indicates the pronunciation of the back vowel. The presence of harmonic components in the spectrum of the speech signal in the mid-frequency region indicates the pronunciation of the vowel middle series. The simultaneous presence of harmonic components in the spectrum of the speech signal in the low-frequency region and in the high-frequency region and their absence in the middle-frequency region indicates the pronunciation of the back vowel. The presence or absence of harmonic components is determined by the ratio of the intensity of the frequency components in the low frequency region, the middle frequency region and the high frequency region.

In the system and method according to the invention, the following phonemes are used: decoupled noisy noisy consonants (I), decoupled noisy noisy consonants (II), deaf noisy slit consonants (III), voiced noisy slit consonants (IV), deaf sibilants (V) sybilants (VI), nasal and slotted sonants (VII), trembling sonants (VIII), front row vowels (IX), mixed row vowels (X) and back row vowels (XI). The phoneme group symbol is indicated in parentheses in accordance with Table 1, however, any machine-readable alphanumeric or graphic symbol can be used as a phoneme group symbol, which allows you to identify the group and process the speech signal in hardware and software based on the characteristics of the phoneme groups thus identified.

The closed, dull, noisy consonants (I) are determined by the following acoustic features: the absence of a fundamental tone and broadband noise, and are characterized by a bow, that is, an actual absence of a speech signal, and subsequent short-term wide-band noise. The closed, dull, noisy consonants differ from the pauses between words with the length of the bow, which is much shorter than the pause between words, and the presence of a subsequent explosion, characterized by short-term wide-band noises.

The loud voiced noisy consonants (II) are unambiguously determined by the following acoustic features: the presence of the fundamental tone and the absence of broadband noise at the site of the bow, as well as subsequent short-term wideband noise at the site of the explosion.

Deaf noisy slotted consonants (III) are determined by the following acoustic features: the absence of a fundamental tone, the presence of broadband noise, the absence of high-frequency noise, and the absence of sonority.

Voiced noisy slotted consonants (IV) are determined by the following acoustic features: the presence of a fundamental tone, the presence of broadband noise, the absence of high-frequency noise, and the absence of sonority.

Deaf sibilants (V) are determined by the following acoustic features: lack of pitch, presence of broadband noise, presence of high-frequency noise, lack of sonority.

The sonorous sibilants (VI) are determined by the following acoustic features: the presence of the fundamental tone, the presence of broadband noise, the presence of high-frequency noise, and the absence of sonority.

Nasal and fissured sonants (VII) are determined by the following acoustic features: the presence of a fundamental tone, the presence of sonorrhea, and the acoustic characteristic of the absence of a vowel.

Trembling sonants (VIII) are determined by the following acoustic features: the presence of the fundamental tone, the presence of sonority, the presence of a difference in the intensity of the speech signal.

The vowels of the back row (IX) are determined by the following acoustic features: the presence of the main tone, the presence of sonority, the acoustic characteristic of the presence of the vowel, the acoustic characteristic of the back row of the vowel.

Mixed vowels (X) are determined by the following acoustic features: the presence of the fundamental tone, the presence of sonority, the acoustic characteristic of the presence of a vowel, the acoustic characteristic of a mixed vowel.

Vowels of the front row (XI) are determined by the following acoustic features: the presence of the main tone, the presence of sonority, the acoustic characteristic of the presence of the vowel, the acoustic characteristic of the front row of the vowel.

Affricates are considered as a sequential pronunciation of the corresponding consonant and crevice consonant, i.e. / s / is a combination of / t / and / s /, a / ch / is a combination of / t / and / sch '/, which is reflected in the transcription of the dictionary of block 4.

All signs by phoneme groups are given in table 1.

The correspondence of the used phoneme and phoneme groups of the Russian and English languages is given in table 2.

table 2 Phoneme Group Symbol Symbols of a group of phonemes Grouped phonemes I bowed deaf noisy consonants / p /, / p '/, / k /, / k' /, / t /, / t '/ II interconnected voiced noisy consonants / b /, / b '/, / g /, / g' /, / d /, / d '/ III dull noisy slotted consonants / f /, / f '/, / h /, / h' / IV voiced noisy slotted consonants / v /, / v '/ V deaf sibilants / s /, / s '/, / sh /, / sch' / VI voiced sibilants / z /, / z '/, / zh / VII nasal and fissured sonants / n /, / n '/, / m /, / m' /, / l /, / l '/ VIII trembling sonants / r /, / r '/ IX front row vowels / i /, / y /, / j '/ X mixed vowels / e /, / a / Xi back vowels / u /, / o /

The correspondence of the used phoneme and phoneme groups of the Russian and English languages is given in table 3.

Table 3 Phoneme Group Symbol Symbols of a group of phonemes Grouped phonemes I bowed deaf noisy consonants P, T, K II interconnected voiced noisy consonants B, D, G III dull noisy slotted consonants F, NN, TH IV voiced noisy slotted consonants V, DH V deaf sibilants Z, ZH VI voiced sibilants S, SH VII nasal and fissured sonants N, M, L, NG VIII trembling sonants R IX front row vowels EH, ER, IH, YT, JH, Y X mixed vowels AA, AN, AE Xi back vowels AO, UH, UW

The transcription icons in table 3 are shown using the ARPAbet alphabet, which is standardly used in many modern speech recognition systems for the English language.

As can be seen from the data presented in tables 2 and 3, the signs of phoneme groups with the same degree of reliability allow us to identify speech sounds for both the Russian language and English, which confirms the universality of the claimed method of speech recognition regardless of the language system.

Speech recognition is carried out on the basis of the method of sequential decoding of sequences of characters denoting phoneme groups into a chain of words making up a statement (text). To determine a word by a sequence of characters denoting phoneme groups, a pronunciation dictionary is used, consisting of a list of words and their corresponding transcriptions, marked up in the characters of phoneme groups. At the beginning of the stage, the dictionary is loaded into the RAM of the device and each word is encoded in the symbols of phoneme groups according to the correspondence table. Analysis of the input symbol combination is carried out on the basis of a sequential method that determines the variants of suitable words from the dictionary by the sequence of phoneme symbols until the variants lead to the only word from the dictionary of the greatest length (with the largest number of phoneme group symbols from the input sequence). After that, the analysis of the next input sequence begins, starting with the first character of the phoneme group after the word included in the previously defined one.

When the processing of the input sequence of group symbols reaches the shortest form of the word in the dictionary, a further selection is made from the variants of this word with endings and compound words of the sequence of this form of the word. If the next character in the sequence does not give any of the options for the continuation of this word, then this is the final form of the word, and there is a transition to the definition of a new word. If there are ending options for this word for the next character, then this option is selected to the end. If the selected longest word form from the dictionary, corresponding to the input sequence of characters, leads to illegibility of the subsequent chain (does not match any word from the dictionary), variants with other forms (shorter and not compound) of this word are checked. And if the choice of a shorter form leads to legibility of the subsequent sequence, a shorter version of the word is selected.

If the input character sequence does not correspond to any word from the dictionary or its left part (leads to a zero choice of words from the dictionary), then it is cut off one character from the sequence of phoneme characters until the subsequent analysis of the words (to the right of the cut off) is successful.

Implementation of the method according to the invention is illustrated in the following example, giving representation of the method and system, but is not exhaustive.

Recognition example on the example of recognition of the word "HELLO"

The word "hello" and its transcriptions:

- spelling: hello;

- phonemic transcription: / p r 'i v' e t /;

- symbolic transcription based on the signs of phoneme groups: / I VIII IX IV X I / in accordance with table 2.

On figv) designated the following levels of segmentation:

L1 - marking the speech signal to allphones.

L2 - specific groups of phonemes.

L3 is the level of the difference in the intensity of the speech signal. Legend: R - the presence of a difference in the intensity of the speech signal, not - the absence of a difference in the intensity of the speech signal.

L4 is the level of the acoustic sign of the vowel. Legend: i - vowel of the front row, a - vowel of the mixed series, u - vowel of the back row.

L5 is the level of the presence of drowsiness. Designations: sonant - the presence of sonorrhea, not - the absence of sonorrhea.

L6 - level of high-frequency noise. Designations: sibilant - the presence of high-frequency noise, not - the absence of high-frequency noise.

L7 - the level of the presence of the fundamental tone. Designations: voiced - presence of the fundamental tone, unvoiced - absence of the fundamental tone.

L8 is the level of broadband noise. Legend: not - the presence of broadband noise, closure - the absence of broadband noise.

L9 is the vowel level. Designations: vowel - the presence of a vowel, not - the absence of a vowel.

Parsing a sequence of characters into text

The input speech (acoustic) signal is processed in the analysis and normalization unit of the spectrum. The length of the speech signal of the word "hello" is about 700 ms. The speech signal is represented as a sequence of 28 windows 25 ms long. In each window, using the fast Fourier transform procedure, a spectrum is calculated at a frequency of 0-7000 Hz. The resulting spectrum is normalized. The sequence of windows of the normalized signal enters the unit for determining the presence of acoustic signs and the formation of a sequence of characters denoting groups of phonemes, which determines that the word "hello" (in transcription pr'iv'et) corresponds to the sequence I VIII IX IV X I. The determination was made on an experimental frequency dictionary of 2400 word forms. The smallest word that matches the combination of the first character is I VIII IX reveals the preposition pr'i. The sample of this word includes 49 word forms (words starting with sequence I VIII IX). Further selection comes only from a given sample of words placed in hash programs. The next group symbol narrows the sample to 12 words corresponding to word forms - hello, vaccination, including vaccinated, vaccinate, etc., greetings, including greetings, greetings, greetings, friendly, etc. The choice is continued, since all words are long in the number of phonemes relatively unassembled sequence. The fourth character in the sequence does not restrict the choice, and the fifth (X) narrows it to all word forms of the word "vaccination". The sixth character (I) does not restrict the choice of the remaining word forms of the word "hello" in the symbols of the groups, but this is the last character, and only one word corresponds to such a sequence of group symbols - "hello". As can be seen from the presented example, the speech recognition system and method does not require access to any pronunciation patterns of words and is not associated with the characteristic features of pronunciation, since the recognition process is based on objective acoustic characteristics of the speech signal, which are speaker-independent, and their comparison with the characteristics of the groups phonemes and use the pronunciation dictionary markup groups in terms of phonemes, ensuring the possibility of successively sampling dictionary words for a set of characters of phoneme groups with a narrowing of the sample to a single word by sequentially adding a character to the right of the minimum sequence of characters defining a separate word from the dictionary and subsequent possible word forms, excluding sequential phonemic sound recognition, which requires taking into account the peculiarities of the speaker’s speech, followed by word recognition.

Thus, the system and method for recognizing continuous speech in a sound stream according to the invention allows to significantly reduce the duration of speech recognition in combination with high accuracy due to speech recognition by the chain of characters of phoneme groups, excluding phonon recognition, which requires a lot of time and considerable hardware resources. Simultaneous determination of several fundamental signs of sounding speech, including the presence or absence of a fundamental tone, the presence or absence of broadband noise, the presence or absence of a difference in the intensity of a speech signal, the presence or absence of high-frequency noise, the presence or absence of sonority, an acoustic sign of the presence or absence of a vowel, acoustic feature of a vowel series, allows you to classify a speech signal in accordance with features that are independent of acoustic characteristics akteristik different speakers, as well as the language in which pronounced it.

Claims (19)

1. A method of speech recognition, comprising sequentially executed steps for receiving a speech signal at the input of a receiving unit; processing the speech signal by the information processing unit, including its processing by an analog-to-digital converter with a pre-set sampling frequency and segmentation, spectral analysis of the speech signal segments and normalization of the spectrum at high frequencies; selection in the normalized spectrum of pauses, noise and sound signals with its subsequent recognition and conversion into text using a predefined dictionary, characterized in that at the stage of recognition based on the original speech signal and normalized spectrum in each segment, the presence / absence of acoustic features of the speech signal is determined, combinatorial sets of which characterize groups of phonemes whose parameters are predefined in the memory block and compare certain combinatorial sets of acoustic features of a segment with predefined parameters of phoneme groups, with the simultaneous formation of a sequence of characters denoting phoneme groups corresponding to combinatorial sets of acoustic features of each segment, the conversion of which into connected text is carried out by sequential decoding of the combinatorial combination of characters of phoneme groups of a sequence based on a dictionary marked up with group symbols phonemes. 2. The recognition method according to claim 1, characterized in that when classifying phoneme groups, a combinatorial set of acoustic features is used, including determining the presence or absence in the speech signal of at least the fundamental tone, broadband noise, the difference in the intensity of the speech signal, high-frequency noise, sonorities, vowels, vowels, a combinatorial combination of which forms 11 groups of phonemes with a unique set of acoustic features: interconnected deaf noisy consonants, interconnected voiced noisy consonants, deaf noise slit consonants, voiced noisy slit consonants, deaf sybilants, voiced sybilants, nasal and slit sonants, trembling sonants, front vowels, mixed vowels and back vowels. 3. The recognition method according to claim 1, characterized in that some of the acoustic features are calculated from the waveform, and the rest from the spectrum of the speech signal obtained using the fast Fourier transform. 4. The method of claim 1 recognition, characterized in that the set of combinatorial determining acoustic features of the speech signal segment is performed in parallel and simultaneously. 5. Detection method according to claim 1, characterized in that in step speech recognition character sequence, designating a group of phonemes using a predefined vocabulary for the markup symbols phoneme groups. 6. The recognition method according to claim 1 or 5, characterized in that the conversion of the sequence of characters denoting phoneme groups into text is carried out on the basis of the sequential decoding method, which consists in sequentially reducing the selection of words and word forms from the dictionary marked up by the characters of the phoneme groups formed by combinatorial sequence of phonemes by groups of symbols the first symbol in the segment to the sequence fragment comprising the combination of symbols groups phonemes defining a unique word of the words pn, which is carried out after recognition of the following input sequences. 7. The recognition method according to claim 2, characterized in that the presence of the fundamental tone is determined by the high intensity of the frequency components in the low frequency region in the range of possible values of the frequency of the fundamental tone, and the intensity of the frequency components in the current window determines their relative maximum intensity in the speech signal over long segment of the speech signal. 8. A method according to claim 7 recognition, characterized in that the intensity of the frequency components in the current window is determined during the speech segment with at least 4. 9. The detection method according to claim 2, characterized in that the short-term fluctuations of the intensity determined by the ratio of the intensity of the speech signal in three consecutive processing windows. 10. The recognition method according to claim 2, characterized in that broadband noise is determined by the presence of intense frequency components in the range above the possible values of the fundamental frequency and its first harmonic, while the intensity of the frequency components in the current window is determined relative to their maximum intensity in the speech signal over a relatively long length of speech. 11. The recognition method according to claim 2, characterized in that the high-frequency noise is determined in the range above the possible values of the fundamental frequency and its first harmonic, in relation to the intensity of the frequency components in the middle frequency region and the intensity of the frequency components in the high frequency region. 12. The recognition method according to claim 2, characterized in that sonority is determined by the high intensity of the frequency components in the mid-frequency range above the low-frequency region in the range of possible frequencies of the fundamental tone, but containing the range of possible frequencies of the formants of the sonants. 13. A method of recognition according to claim 2, characterized in that the presence or sonant vowels respectively determined by the presence / absence of harmonic components in the medium and high frequencies. 14. The recognition method according to claim 2, characterized in that the vowel pronunciation series is determined by the ratio of the intensity of the harmonic components in the spectrum of the speech signal in the low frequency region, the middle frequency region and the high frequency region. 15. The recognition method according to claim 1, characterized in that at the stage of decoding a sequence of characters denoting phoneme groups, one or a group of pre-installed thematic dictionaries marked up by the characters of phoneme groups, different language systems, which are selected at the initial recognition stage, are used in the text speech. 16. A speech recognition system comprising a speech signal receiving unit connected in series with an analog-to-digital converter, the output of which is connected to an analysis and normalization unit for the frequency spectrum of a speech signal connected to a control unit and a memory unit configured to record and store recognized speech signal, storage and call dictionary, characterized in that it further comprises a series-connected unit for determining the presence of acoustic signs and the formation of consistency of symbols denoting phoneme groups, made possible to generate, according to current combinatorial combinations of acoustic signs, the corresponding sequence of symbols denoting phoneme groups present in the speech signal, a block for decoding a sequence of characters denoting phoneme groups into text, at least one of whose inputs connected to the input unit of a predefined dictionary marked up by the symbols of phoneme groups and called from the memory block, made with the possibility of determining words on the basis of a dictionary by a combinatorial combination of symbols of phoneme groups of the current sequence, as well as a text output and storage device, at least one input / output of which is connected to a control unit configured to control the modes of functioning of the units, exchange data between them and interaction with the memory unit. 17. The recognition system according to clause 16, characterized in that the input of the unit for determining the presence of acoustic signs and the formation of a sequence of characters denoting phoneme groups is connected to the output of an analog-to-digital converter with the possibility of inputting a speech signal and the output of the analysis and normalization of the private speech spectrum signal with the ability to enter a normalized speech signal, and a block of acoustic signs of phoneme groups with the possibility of comparing the current combinatorial set acoustic features of the speech signal with the parameters set of acoustic features preset groups of phonemes and connected to the memory unit. 18. The recognition system of claim 16, wherein the text output and storage device is further provided with a display for outputting and displaying the recognized text. 19. A recognition system according to any one of claims 16, 17 or 18, characterized in that the block of acoustic signs of phoneme groups contains a combinatorial set of acoustic signs of 11 groups of phonemes: decoupled noisy consonants, decoupled voiced noisy consonants, deaf noisy slotted consonants, voiced noisy gap consonants, dull sybilants, voiced sybilants, nasal and slit sonants, trembling sonants, front row vowels, mixed row vowels and back row vowels, including at least the presence characteristic for each phoneme group I / lack of pitch window, broadband noise, the differential intensity of the speech signal, high-frequency noise, sonority, vowel, vowel series.

Images