RU2606566C2 - Method and device for classifying noisy voice segments using multispectral analysis - Google Patents

Abstract

FIELD: digital communications; information technology.

SUBSTANCE: invention relates to digital communication and voice processing techniques in the conditions of noise masking. Method of classifying noisy voice segments involves multispectral analysis for partial noise suppression by increasing adequacy of the signal Fourier spectrum, which is restored from evaluation of the processed noisy voice signal bispectrum; further, basing on the initial voice signal and standardized Fourier spectrum in each segment identified is presence/absence of classification features of the voice signal and weight coefficients, performed is their correction using the hierarchy procedure and at the final stage a decision is taken upon the issue of associating the segment to typical groups, as per the comparison results formed is a sequence of symbols indicating the typical groups.

EFFECT: higher accuracy of classifying noisy voice segments as per typical classification groups.

10 cl, 7 dwg

Description

The presented inventions are united by a single concept and relate to the field of digital communications; they can be used in telecommunication systems when implementing the classification procedure for segments of a speech signal in a noisy environment.

Scope of inventions: radiotelephony and speech recognition systems, voice control of electronic devices, automatic shorthand, voice control of moving means at a distance of audible voice.

Despite the presence of a large number of technical solutions in the field of application of the claimed inventions, there is a problem associated with the processing of noisy speech at a high intensity of noise exposure, which is strongly manifested in a decrease in the reliability of the decisions made by the currently developed methods and devices that implement them.

A known method and device for speech recognition (US patent US 4,624,011 A, G10L 5/00, 01/28/1983), in which speech recognition is performed by the phased execution of certain procedures: the amplitude and phase Fourier spectra are evaluated, then the sequences of phonemes and their acoustic characteristics calculated by the auxiliary module, which are then compared with the stored reference parameters for the analyzed sequences, and then determine the degree of similarity through an interval assessment, ha RMS error. This method and device that implements it is characterized by low noise immunity, since in the presence of noise of the processed speech signal, recognition by comparison with a reference becomes impossible, which leads to a significant decrease in the reliability of decisions made.

There is a method and apparatus for recognizing continuous speech (US patent US 4852170, G10L 5/04, 12/18/1986), based on real-time speech processing by a device in which spectral estimates of each speech segment of a given duration are determined, while each speech segment is analyzed logically for the presence of phonemes and their belonging to a certain class, of which they are a part, and then the frequency spectrum of the segment is analyzed for the presence of features that make it possible to recognize specific phonemes within the type. The phoneme sequence can be stored as compact groups and then transformed to synchronize with the voice of the speaker.

Also known is a method and device for recognizing speech phonemes (RF patent 2268504 C9, published January 20, 2006), based on speech recognition by a device including a bispectrum analyzer, in which individual standards of each phoneme of speech are determined, while each allophone of individual speech is logically analyzed to the maximum coincidences when comparing with each reference stored in the memory, and after comparison make a decision on belonging to a specific phoneme. The phoneme sequence is stored in compact groups and then transformed to synchronize with the voice of the speaker.

The closest analogue in terms of essential features recognized as a prototype is a speech recognition system and method (RF patent No. 2466468, publ. 10.11.2012) for processing a continuous speech signal in real time, including sequentially executed stages, according to which the reception speech signal; perform its processing, for which an analog-to-digital conversion is performed with a predetermined sampling frequency and divided into quasistationary segments, spectral analysis of the noisy speech signal segments and spectrum normalization are performed; pauses, noises and sound signals are distinguished in the normalized spectrum, then they are classified according to typical groups, for which the presence / absence of classification features, combinatorial sets of which characterize groups of phonemes whose parameters are preset in the block, is determined on the basis of the initial speech signal and the normalized spectrum memory and compare certain combinatorial sets of classification features of the segment with the predefined parameters of phoneme groups, at the same time form the last A sequence of characters denoting phoneme groups, with the simultaneous formation of a sequence of characters denoting phoneme groups corresponding to combinatorial sets of classification features of each segment, using classification of phoneme groups based on a combinatorial set of classification features that characterize the presence or absence of at least pitch, broadband noise, difference in speech intensity, high-frequency noise, sonority, vowel, row and a vowel, the combinatorial combination of which forms eleven phoneme groups with a unique set of classification features: decoupled noisy consonants, decoupled voiced noisy consonants, deaf noisy slit consonants, voiced noisy slit consonants, deaf sibilants, voiced sybilants, nasal and slotted sonants, trembling sonants vowels of the front row, mixed vowels and back row vowels corresponding to combinatorial sets of classification features of each segment, then sequentially by decoding the combinatorial combination of phoneme group symbols into text based on a dictionary marked up by phoneme group symbols.

The main disadvantage of this method is the low noise immunity to the conditions of intense noise of the processed speech signal, which leads to errors of the first kind for the analysis of speech areas where the presence or absence of the fundamental tone is not the main classification sign, in addition, in the case of noise exposure to a homogeneous energy, many classification signs of different phoneme groups based on the presence of various types of noise, as a result of which the classification reliability decreases segments of the processed speech signal, which ultimately leads to incorrect recognition of each character that is part of the code word.

One common drawback characterizing all analogues and prototype is the low reliability of the classification according to certain typical groups of segments of noisy speech under conditions of high-intensity noise exposure.

The objective of the claimed inventions is to create a method for classifying segments of a noisy speech signal using multispectral analysis and a device that implements it, increasing the reliability of the classification of segments of a processed noisy speech signal according to certain groups of phonemes.

This task is achieved by the fact that according to the claimed method for classifying segments of noisy speech using multispectral analysis, which includes sequentially executed steps, according to which a noisy speech signal is received; perform analog-to-digital conversion with a predefined sampling frequency and separation into quasistationary segments, then calculate the Fourier spectra of the segments of the noisy speech signal and normalize it, isolate pauses, noise and sound signals in the normalized Fourier spectrum, then classify them according to typical groups, for which purpose determine, on the basis of the initial speech signal and the normalized spectrum in each segment, the presence / absence of classification features, combinatorial sets of which characterize groups of phonemes whose parameters are predefined in the memory block and compare certain combinatorial sets of classification features of a segment with predefined parameters of phoneme groups, at the same time form sequences of characters representing phoneme groups, characterized in that they use a multispectral analysis apparatus, namely, when evaluating the amplitude Fourier transform spectrum (hereinafter Fourier spectrum) of the speech signal, the Fourier spectrum is obtained through the bispectrum of the speech signal, which is synthesized by a two-dimensional discrete Fourier transform of the triple autocorrelation function in the quasistationary section of the processed noisy speech signal in order to partially suppress the Gaussian components of the noise exposure, in addition, at the stage of deciding the classification of the processed segment of the noisy speech signal with the task of assigning it to a specific group of 12 (twelve) phonemes or 1 (one) group characterizing the pause, they introduce the procedure of hierarchy of classification features of groups, for which weights are introduced, the basis nye on vnutrisimvolnyh and intersymbol connections as in the code word, and within each symbol and receive them by evaluating the statistics and the parametric features as the speech signal and noise.

обрабатываемого зашумленного речевого сигнала, вследствие чего при операциях получения биспектра

зашумленного речевого сигнала производят прямое двумерное преобразование Фурье от тройной автокорреляционной функции R_U(a,b), и восстановление Фурье-спектра

осуществляют непосредственно из самого биспектра зашумленного речевого сигнала, вследствие чего происходит частичное шумоподавление в обрабатываемом речевом сигнале за счет подавления гауссовых компонент шумового воздействия на основании сечения стационарного эргодического случайного процесса по кумулянту третьего порядка. Восстановление Фурье-спектров по оценке биспектра, получаемого косвенным методом, достаточно подробно представлено в (Тоцкий А.В., Астола Я. Восстановление сигналов по оценкам биспектров в присутствии гауссовых и негауссовых помех. Зарубежная радиоэлектроника, 2002, №11, с. 44-58; Никиас Х.Л., Рагувер М.Р. Биспектральное оценивание применительно к цифровой обработке сигналов. ТИИЭР, 1987, Т.75, №7, с. 5-30; Zhang Ji-Wu, Zheng Chong-Xun, and Xie Au, Bispectram analysis of focal ischemic cerebral EEG signal usingthird-order recursion method, IEE Trans. Biomedical Engineering, vol. 47, No. 3, March 2000, pp. 352-359). Тогда нахождение тройной автокорреляционной функции и биспектра осуществляют согласно следующим выражениям:The claimed method is characterized by the fact that at the sampling stage, a constant value of the sampling frequency is set equal to 44100 Hz, in addition, at the segmentation stage, a constant quasistationary period equal to 1024 samples is selected, and when moving from segment to segment, the relative window of intersection of neighboring segments is used, equal to 512 counts, and at the stage of spectral analysis, a multispectral analysis is used, which includes the assessment and work not only with the Fourier spectrum, but also with the bispectrum

the processed noisy speech signal, as a result of which, during bispectrum acquisition operations

noisy speech signal produce a direct two-dimensional Fourier transform of the triple autocorrelation function R _U (a, b), and Fourier spectrum reconstruction

they are carried out directly from the bispectrum of the noisy speech signal, as a result of which partial noise reduction occurs in the processed speech signal due to the suppression of the Gaussian components of the noise effect based on the section of a stationary ergodic random process by a third-order cumulant. The restoration of the Fourier spectra from the estimation of the bispectrum obtained by the indirect method is described in sufficient detail in (Totsky A.V., Astola Ya. Restoration of signals according to the estimates of the bispectra in the presence of Gaussian and non-Gaussian interference. Foreign Radio Electronics, 2002, No. 11, p. 44- 58; Nikias H.L., Raguver M.R. Bispectral Evaluation as Applied to Digital Signal Processing, TIIER, 1987, Vol. 75, No. 7, pp. 5-30; Zhang Ji-Wu, Zheng Chong-Xun, and Xie Au, Bispectram analysis of focal ischemic cerebral EEG signal using the third-order recursion method, IEE Trans. Biomedical Engineering, vol. 47, No. 3, March 2000, pp. 352-359). Then, the triple autocorrelation function and the bispectrum are found according to the following expressions:

- комплексная функция двух независимых частотных переменных p,q:where W (a, b) is the window function used to increase the adequacy of the estimate and reduce the spreading effect of the bispectrum, K is the number of samples in the segment of the processed speech signal, a, b are the values of the correlation shift. Besides,

is the complex function of two independent frequency variables p, q:

- биамплитуда, γ_U(p,q) - бифаза.Where

- biamplitude, γ _U (p, q) - biphase.

In the case of a homogeneous noise exposure, where the noise itself is a Gaussian random process or is very close to a Gaussian process, so that it can be approximated by white Gaussian noise, then for the case of additive noise exposure, partial noise reduction is valid based on the cross section of the processed noisy speech signal, represented by an ergodic random process, according to a third-order cumulant - asymmetry. In a practical application, this means studying correlations of the third order, which are equal to zero for Gaussian processes, in addition, for random processes close to a Gaussian process, these characteristics tend to zero and have small values:

- биспектр «чистого» речевого сигнала,

- биспектр шумового воздействия,

- спектральная компонента на частоте p Фурье-спектра «чистого» речевого сигнала,

- спектральная компонента на частоте p Фурье-спектра шумового воздействия.Where

- bispectrum of a "pure" speech signal,

- bispectrum of noise exposure,

- the spectral component at a frequency p of the Fourier spectrum of the "pure" speech signal,

- the spectral component at a frequency p of the Fourier spectrum of the noise exposure.

The amplitude and phase Fourier spectra are reconstructed from the bispectrum of a noisy speech signal based on iterative recursive algorithms by means of a median and submedian section of biamplitude and biphase:

In the normalized Fourier spectrum of each segment based on the introduction of a hierarchy procedure based on recursive intersymbol and intra-character codeword relationships, a description of which is presented in detail in (OI Shelukhin, NF Lukyantsev. Digital processing and transmission of speech. M., Radio and Communications, 2000 - pp. 102-112, pp. 123-146; Bykov S.F., Zhuravlev V.I., Shalimov I.A. Digital Telephony: a textbook for universities. - M .: Radio and communication, 2003 - 144 pp.), consisting of symbols denoting phoneme groups, determine the presence / absence of classification features of speech signal, combinatorial sets of which characterize groups of phonemes, using the hierarchy procedure, the parameters of combinatorial sets are predefined in the memory block, and they compare certain combinatorial sets of classification features and weight coefficients of the segment with the preset parameters of phoneme groups, with the simultaneous formation of a sequence of characters denoting phoneme groups corresponding to combinatorial sets of classification features of each segment.

When classifying segments of the processed speech signal by groups of phonemes, a combinatorial set of classification features and weighting factors is used, including determining the presence or absence of the fundamental tone, broadband noise, the difference in the intensity of the speech signal, high-frequency noise, sonority, vowel, vowel series, combinatorial combination of which forms 12 groups of phonemes with a unique set of classification features and weighting coefficients: interrupted deaf noisy consonants clear voiced noisy consonants, deaf noisy slotted consonants, voiced noisy slotted consonants, deaf sibilants, voiced sibilants, nasal and slotted sonants, trembling sonants, front vowels, mixed vowels and back vowels, affricates. In addition, a pause is introduced by an additional separate group, which is characterized as the absence of informative active components for 20 (twenty) quasistationary segments without taking into account the mutual window of intersection of neighboring segments or the presence of a noise sign and various noises for the same duration.

One of the most important classification characteristics is the presence of a fundamental tone in a speech signal. The presence of the fundamental tone is estimated by the high intensity of the frequency components in the low-frequency region in the range of possible values of the fundamental frequency. The intensity of the frequency components in the current window is determined relative to their maximum intensity in the speech signal over a relatively long length of the speech signal about 5 (five) seconds long.

Short-term differences in the intensity of the speech signal, indicating the presence of short bows in the signal, characteristic of trembling sonants, are determined by the ratio of the intensity of the speech signal in three consecutive processing windows. The intensity of the speech signal in the middle window is significantly lower than the intensity of the speech signal in the right and left windows, while the intensity of the speech signal in the right and left windows is almost the same.

The presence of broadband noise in a speech signal associated with the pronunciation of slotted consonants or the presence of an explosion occurring during opening the bow when pronouncing the consonant consonants is determined by the presence of intense frequency components in the range above the possible values of the fundamental frequency and its first harmonic.

The presence of high-frequency noise in a speech signal associated with the pronunciation of slotted sibilants is determined in the range above the possible values of the frequency of the fundamental tone and its first harmonic, in relation to the intensity of the frequency components in the middle frequency region and the intensity of the frequency components in the high frequency region. The intensity of high-frequency noise significantly exceeds the intensity of medium frequencies in the case of pronouncing slotted sibilants.

The sonority of a speech signal, characteristic of pronouncing sonants and vowels, as opposed to noisy consonants, is determined by the high intensity of the frequency components in the mid-frequency range above the low-frequency region in the range of possible frequencies of the fundamental tone, but containing the range of possible frequencies of the formants of the sonants.

Another classification feature used in speech recognition and for characterizing phoneme groups is the absence or presence of harmonic components in the spectrum in the frequency domain above the range of possible frequencies of the formants of the sonants. The absence of harmonic components in the mid and high frequencies is characteristic of the sonants, and the presence of vowels. The presence or absence of harmonic components is determined by the ratio of the intensity of the frequency components below and above the frequency threshold.

Another important classification characteristic of speech sounds is the quality of a vowel that has been pronounced, namely, the number of pronunciations, i.e. the position of the bulk of the tongue in the oral cavity in a horizontal position. A number of vowel pronunciations is determined by the ratio of the intensity of the harmonic components in the spectrum of the speech signal in the low frequency region, the middle frequency region and the high frequency region. The absence of harmonic components in the spectrum of the speech signal in the mid-range and high-frequency region indicates the pronunciation of the back vowel. The presence of harmonic components in the spectrum of the speech signal in the mid-frequency region indicates the pronunciation of the vowel middle series. The simultaneous presence of harmonic components in the spectrum of the speech signal in the low-frequency region and in the high-frequency region and their absence in the middle-frequency region indicates the pronunciation of the front vowel. The presence or absence of harmonic components is determined by the ratio of the intensity of the frequency components in the low frequency region, the middle frequency region and the high frequency region.

In the claimed invention, the following classification groups of segments of the processed noisy speech signal are used: decoupled noisy noisy consonants (O), decoupled noisy noisy consonants (B), deaf noisy slotted consonants (C), voiced noisy slotted consonants (D), deaf sibilants (E) , voiced sybilants (F), nasal and slit sonants (G), trembling sonants (H), front row vowels (I), mixed row vowels (L) and back row vowels (K), affricates (M), pause (N) ) The conditional symbol of the classification group is indicated in parentheses.

Closed-up blind noisy consonants (O) are determined using the following classification features: the absence of pitch and broadband noise, which is characterized by a bow, that is, the actual absence of a speech signal, and subsequent short-term wide-band noise. The closed, dull, noisy consonants differ from the pauses between words with the length of the bow, which is much shorter than the pause between words, and the presence of a subsequent explosion, characterized by short-term wide-band noises.

Bowed voiced noisy consonants (B) are determined by the following classification features: the presence of the fundamental tone and the absence of broadband noise at the site of the bow, as well as subsequent short-term wideband noise at the site of the explosion.

Deaf noisy slotted consonants (C) are determined by the following classification features: lack of pitch, presence of broadband noise, lack of high-frequency noise, lack of sonority.

Voiced noisy slotted consonants (D) are determined by the following classification features: the presence of the fundamental tone, the presence of broadband noise, the absence of high-frequency noise, and the absence of sonority.

Deaf sibilants (E) are determined by the following classification features: lack of pitch, presence of broadband noise, presence of high-frequency noise, lack of sonority.

Voiced sibilants (F) are determined by the following classification features: the presence of the fundamental tone, the presence of broadband noise, the presence of high-frequency noise, and the absence of sonority.

Nasal and fissured sonants (G) are determined by the following classification features: the presence of the fundamental tone, the presence of sonorrhea, and the classification characteristic of the absence of a vowel.

Trembling sonants (H) are determined by the following classification features: the presence of the fundamental tone, the presence of sonority, the presence of a difference in the intensity of the speech signal.

Vowels of the front row (I) are determined by the following classification features: the presence of the fundamental tone, the presence of sonority, the classification characteristic of the presence of the vowel, the classification characteristic of the front row of the vowel.

Mixed vowels (L) are determined by the following classification features: the presence of the fundamental tone, the presence of sonority, the classification characteristic of the presence of a vowel, the classification characteristic of a mixed vowel series.

The vowels of the back row (K) are determined by the following classification features: the presence of the basic tone, the presence of sonority, the classification characteristic of the presence of the vowel, the classification characteristic of the back row of the vowel.

Affricates (M) are defined as the sequential pronunciation of the corresponding occlusal and crevice consonants, i.e. / c / is a combination of / t / and / s /, and / h / is a combination of / t / and / w /.

Pause (N) is defined as repeated repetition of characteristic sign b - the absence of a signal over 20 segments of quasi-stationarity; under noisy conditions, a pause is determined upon repeated repetition at the same duration of a sign of the presence of various unvoiced noise-like segments and a noise sign.

The following classification features of the groups are used in the claimed invention:

a ₀ - the presence of the fundamental tone; a ₁ - the presence of harmonic components in the field of the first formant; and ₂ - the presence of harmonic components in the field of the second formant; and ₃ - the presence of harmonic components of low intensity; b - lack of signal; with - the presence of differences; d - broadband noise; e - brief noises; f - high-frequency noise; g - the presence of drowsiness; n is the presence of noise.

In addition, the claimed invention introduces weight coefficients that define a hierarchy based on the parametric properties of speech signals and noise:

where R: 1 - confirmation of the presence of short noise, 0 - the presence of short noise is false;

S: 1 - confirmation of the presence of broadband noise, 0 - the presence of broadband noise is false;

T: 1 - confirmation of the presence of high-frequency noise, 0 - the presence of high-frequency noise is false;

V: 1 - confirmation of the presence of drowsiness, 0 - the presence of drowsiness is false.

There is also the fact of false vocalization, for example, phonetic deaf noisy consonants standing in a word after vowels, in connection with which they introduce a hierarchy attribute based on statistics of voiced and unvoiced speech elements shown in the table in FIG. one:

W: 1 - confirmation of the presence of the fundamental tone, 0 - presence of the fundamental tone is false.

All signs (taking into account weights) and classification of 12 (twelve) phoneme groups and 1 (one) group characterizing a pause are shown in FIG. 2 and can be written in the form of the following expressions:

The method claimed in the invention can be represented in the form of an algorithm (figure 3), in accordance with which the following stages of processing a noisy speech signal are distinguished in order to carry out classification in certain groups:

I) Reception of a speech signal U (t);

II) Sampling of the speech signal U (kT);

III) The segmentation of the speech signal, taking into account the mutual window of intersection of neighboring segments of the analysis;

IV) Calculation of the values of the triple autocorrelation function (TACF) R _U (a, b);

V) Synthesis of the bispectrum B _U (p, q) obtained by the Fourier transform of TAKF;

и бифазы γ(p,q);VI) Obtaining biamplitude

and bifase γ (p, q);

VII) Recovery of the amplitude Fourier spectrum

VIII) Normalization of the amplitude Fourier spectrum

IX) Isolation of segment classification features: a 0, a 1, a 2, a 3, a 4, b, c, d, f, g, n;

X) Calculation of weighting factors: R, S, T, V;

XI) Weighting Ratio - W;

XII) Correction of segment classification features;

XIII) Decision making regarding classification.

The invention is also achieved by the fact that a device that implements a method for classifying segments of noisy speech using multispectral analysis, including a unit for receiving a noisy speech signal, connected in series with an analog-to-digital converter, which is connected to a control unit, a memory unit and a unit for determining classification features of phoneme groups in order to form a sequence of characters denoting phoneme groups, made with the possibility of forming according to current com binator combinations of classification features present in the speech signal of the corresponding sequence of characters and the recording of a code word labeled by the symbols of phoneme groups and called from the memory block, while all the blocks are connected to the control unit, which makes it possible to control the operating modes of the blocks and exchange data between them and interacting with a memory unit, characterized in that a speech signal segmentation unit is used, taking into account the mutual window of intersection of adjacent segments according to the number of samples, connected to a set of multispectral analysis blocks, including a third-order correlation block, the output of which is connected to the input of the direct Fourier transform block, the output of which is connected to the input of the Fourier spectrum reconstruction of the amplitude spectrum from the bispectrum estimate, the output of which is connected to the input of the determination block segment classification features; also the additional introduction of a block for the correction of classification features, blocks of a parametric and statistical hierarchy and a block for storing short-term data; in addition, in the block for determining the classification features of phoneme groups, a noise sign identification procedure is implemented, these blocks are interconnected by multiple recursive links that take into account the intrasymbol and intersymbol communication in the generated sequence from the output of the character sequence forming unit.

Also, FIG. 3 shows a block diagram of a device with reference to an algorithm based on the claimed method for classifying segments of a noisy speech signal using multispectral analysis:

1) The level of control actions and predefined classification features (the ability to implement the total processor constant memory), which is technologically composed:

1 - control unit (functionally connected to block 15),

2 - block storage of classification features based on the database;

2) The step of receiving a continuous speech signal, which is technologically composed of:

3 - a unit for receiving a continuous speech signal (it is possible to differentiate into technical sublevels, for example, not only acoustoelectric conversion, but also restriction on the input level);

3) The stage of analog-to-digital conversion and segmentation of a speech signal, having technologically composed:

4 - block analog-to-digital conversion,

5 - block segmentation of a discrete processed speech signal into quasistationary segments, taking into account the mutual windows of intersection of neighboring segments;

4) The stage of multispectral analysis of the speech signal during serial-parallel processing, technologically having in its composition and functionally 2 (two) levels:

4.1) The level of indirect acquisition of the bispectrum of the processed segment of the speech signal, having technologically in its composition:

6-1 ... 6-16 - correlation blocks of the third order,

7-1 ... 7-16- blocks of the Fourier transform;

4.2) The level of determination of segmented classification features that are technologically composed:

8-1 ... 8-16 - Fourier spectrum reconstruction blocks according to the evaluation of the bispectrum of the processed segment of a noisy speech signal,

9-1 ... 9-16 - normalization blocks of the Fourier spectrum,

10-1 ... 10-16 - blocks for determining segmented classification features, comprising a comb of band-pass filters that physically perform the function of detecting the amplitude values of the normalized Fourier spectrum, the levels of which are predefined in block 2;

5) The stage of correction of classification features based on the parametric and statistical hierarchy of signs (the possibility of implementing a combination of a processor of read-only memory (ROM)), which has the following technologically:

11-1 ... 11-16 - blocks for the correction of classification features,

12-1 ... 12-16 - blocks of the statistical hierarchy,

13-1 ... 13-16 - blocks of the parametric hierarchy,

14 - block storage of short-term data (implementation is possible on random access memory) about intersymbol and intrasymbol communications;

6) The decision-making stage on the classification according to the adjusted classification criteria is technologically composed of:

15 - decision block (implementation based on the processor is possible),

16 is a block forming a sequence of characters characterizing certain classification groups.

The procedures for receiving, analog-to-digital conversion and segmentation of a speech signal and their implementation are described in detail in (Solonina A.I., Ulakhovich D.A., Arbuzov S.M., Solovieva E.B. Fundamentals of digital signal processing: Lecture course. - SPb .: BHV - Petersburg, 2003 .-- p. 425-446). A description of the formation and reception of the transmission frame performed by blocks 3, 4, 5 is presented in (Bykov S.V., Zhuravlev V.I., Shalimov I.A.Digital telephony: Textbook for universities. - M .: Radio and Communication, 2003 .-- S. 79-87).

The implementation of the set of blocks 1, 2, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15, 16 is possible on the basis of the TORNADO-P64 module, which was developed by MicroLAB System (www.mlabsys.com) Digital signal processing CHIP NEWS Zhuchkov K., Khoruzhiy S., Chepel E. Multispectral signal analyzer based on the digital signal processor module TMS320C6416).

A device that implements the claimed method works as follows (Fig. 3):

A continuous noisy acoustic speech signal is fed to the input of block 3, in which its acoustoelectric conversion and limitation by input level take place. The received continuous electrical signal from the output of block 3 is fed to the input of the analog-to-digital conversion unit 4, in which discrete samples of the speech signal are obtained with a sampling frequency of 44100 Hz, a sequence of discrete samples from the output of block 4 is fed to the input of the segmentation block 5, where dividing the sequence of samples into quasistationary segments of 1024 samples, taking into account the mutual window of intersection of neighboring segments, equal to 512 samples, then from the output of block 5, segment by segment the speech signal is fed to the inputs of the third-order correlation blocks 6, each of the parallel branches of the speech signal processing, where the triple autocorrelation function for each quasistationary segment is found, from the output of block 6, the sequence of values of the triple autocorrelation function, where the the procedure for finding the values of the bispectrum of the speech signal for each quasistationary segment by the indirect method, from the output of block 7 to the input of the block and the restoration of the values of the Fourier spectra 8 receives the complex values of the bispectrum of each segment of the speech signal, where the bispectrum is divided into the bi-amplitude and biphase of each segment of the quasistationarity with further determination of the Fourier spectra based on iterative algorithms for obtaining the Fourier spectra from the estimates of the bispectrum, from the output of block 8 the input of the normalization block of the Fourier spectrum 9 receives a sequence of values of the Fourier spectra, where the normalization procedure occurs by dividing the values by max the maximum value obtained in the analysis of all values of the Fourier spectra of various quasistationary segments, for a total duration of 5 (five) seconds, from the output of block 9 to the input of the block for finding segment classification features 10, a sequence of values of the normalized Fourier spectrum of amplitudes is received, in which by comparing with different thresholds, the presence or absence of one or another classification feature is distinguished (in the case of the presence of a feature, a signal is given to a logical unit, otherwise 0), from the output of block 10 to the input of the short-term data storage unit 14, information is received in the form of a sequence of 1 and 0, characterizing the presence or absence of certain classification features of each analyzed quasi-stationarity segment, as well as the absolute values obtained during the measurement of a particular attribute in the block 14, information is stored on correlation relationships between quasistationary segments and recursive relationships between symbols in a codeword (phrase), coming from the output of the decision block on classification axis 15, as well as the absolute values obtained during the measurement of a particular characteristic, in addition, from the output of block 10, information about segment classification signs is fed to the input of the classification signs correction block 11, in addition, information about the hierarchy is received of the classification features of each processed quasistationary segment from the output of the block of the statistical hierarchy 12 and from the output of the block of the parametric hierarchy 13, the inputs of the blocks 12 and 13 receive information about the correlation recursive relationships in the code symbolic word and between the processed quasistationary segments, as well as the absolute values of the measured features, in blocks 12 and 13 the weighting coefficients are calculated, which are fed to the input of the classification features correction block 11, in block 11, the weighting coefficients are detected significant and insignificant classification signs, from the output of block 11, information about all classification signs is fed to the input of the decision block in which A decision is made on the classification of the segment to be processed, from the output of block 15, the information goes to the input of the short-term data storage unit 14 and to the input of the block for generating a sequence of codeword symbols, from the output of block 16 information in the form of symbols denoting various 13 (thirteen) classification groups, 12 (twelve) of which are phoneme groups and 1 (one) is a group characterizing a pause. The control unit 1 operates in real time and exercises general control over all the procedures involved in deciding on the classification; it is connected with feedback to the decision unit 15. The storage unit for classification features based on database 2 is based on read-only memory and stores information on combinatorial sets of classification features and weighting factors characterizing thirteen classification groups, block 2 is functionally connected to the blocks 10, 11, 15.

When conducting a classification in the absence or low noise exposure, the decision rules proposed in the prototype have a high recognition accuracy of about 95%, however, under conditions of exposure to high intensity noise, a decrease in reliability is observed due to the appearance of multiple classification features, which leads to incorrect determination of phoneme groups, and when the signal-to-noise ratio is 0 dB, the reliability of the recognition method used in the prototype drops to a value of 67%, which leads to iteration of inadequate decisions.

To evaluate the effect obtained by introducing various actions on the processed speech signal, we take such as the range of signal-to-noise ratio (SNR), characterized by the minimum threshold value for reliability, equal to 90%, and the average gain over the effective range of SNR for reliability, since the increase in reliability is a technical result achieved by the proposed inventions.

The figure 4 presents a graph of the reliability of the correct classification on the signal-to-noise ratio of the method proposed in the claimed invention, the prototype and analogues.

Its analysis allows us to conclude that the prototype in terms of its functional features surpasses all analogues, which indicates the correctness of the selected prototype (in the future, the effectiveness assessment to increase the average reliability will be made between the method according to claim 5 and the prototype).

Before directly calculating the increase in the average reliability of the classification, it should be noted that for the calculation we will use not the entire range presented in figure 4 from - 15 to 45 dB, but the effective range of the SNR at which the minimum requirements regarding the threshold for reliability of 90% are fulfilled, i.e. e. from 0 to 45 dB. Evaluation of the average increase in the reliability of AD,% will be carried out according to the following expression:

Ri is the value of the reliability of the method according to paragraph 5 of the i-th signal-to-noise ratio;

Pi - the reliability value of the prototype from the i-th signal-to-noise ratio

ΔD = 5.269230769230769%

Based on the assessment of the effectiveness of the proposed method, according to the solution of the inventive problem, we can confidently say that the proposed method allows the classification of segments of the processed noisy speech signal in an increase of 7 dB in the range of signal-to-noise ratios (the method proposed in the invention allows correct classification segments of noisy speech at lower signal-to-noise ratios) with an average increase in the accuracy of the correct classification by ≈5.3%.

The reliability of the technical result is confirmed by experimental information obtained during the tests (various recordings of speech signals were used, which were subjected to additive noise by white Gaussian noise at different signal-to-noise ratios (SNR), these noisy signals were subjected to repeated tests in a comparative manner between different methods implemented in MATLAB software environment) according to industry standard methods (according to GOST R 51061-97 Low-speed transfer systems Achi speech on digital channels of voice quality parameters and test methods -.. M .: State Standard of Russia, in 1997 - 230), which showed that the use of this method can improve the accuracy of the classification segments noisy speech by groups of phonemes..

A comparative analysis of the operation of the prototype and the method according to claim 5 is illustrated in figures 5-7 (is not exhaustive):

Classification example of processed segments using the example “KOKOSHNIK”

The word "kokoshnik":

- spelling notation: kokoshnik;

- phoneme transcription: / k’ak’oshn’ik /;

- symbolic transcription based on the characteristics of phoneme groups:

NNOKOLEGIONN, where NN is the presence of a long pause before and after the spoken word.

In FIG. 5 shows the operation of the prototype and method according to paragraph 5 in the absence of interference:

1) marking the speech signal to allophones;

2) identification of classification features;

3) decision-making on the classification of segments into specific groups: NNOKOLEGIONN.

As can be seen from figure 5, the prototype and method according to p. 5 provide a high reliability classification in the absence of noise exposure.

The figure 6 shows the work of the prototype in conditions of intense noise (signal-to-noise ratio 0 dB):

1) marking the speech signal to allophones;

2) identification of classification features;

3) making a decision on the classification of segments in certain groups: EEEEEEEEKELEGIBEEEEEEE.

As can be seen from FIG. 6, the prototype under conditions of noise exposure to high energy makes mistakes in classifying segments by certain phoneme groups due to the appearance of multiple noise classification features in all analyzed segments.

The figure 7 shows the operation of the method according to paragraph 5 under the conditions of noise exposure to high energy (signal-to-noise ratio 0 dB):

1) marking the speech signal to allophones;

2) obtaining the values of the Fourier spectrum of the amplitudes from the evaluation of the bispectrum of the speech signal;

3) the identification of classification signs, the moment of the detection of a noise sign, which is determined in the frequency range of the normalized Fourier spectrum of amplitudes above the range of the speech signal, is important;

4) correction of classification features:

a) errors made by the prototype are marked as:

1, 5 - error skipping pause,

2, 3, 4 - an error in the classification of information symbols in a code word;

b) the calculation of weights:

hierarchy based on speech and noise parameters:

1, 5: r = 0.95; t = 0.95; s = 0.97⇒R = 0; T = 0; S = 0.

2, 3, 4: r = 1.85; t = 0.95; s = 0.97⇒R = 1; T is 0; S is 0;

hierarchy based on statistical features of the duration of various phonemes of a speech signal:

2, 3, 4: W = 0, since the duration of vocalization of segments is 8, 10, 15 ≈30 ms;

4) making a decision on the classification of segments in certain groups: NNOKOLEGIONN

As can be seen from figure 7, the method according to p. 5 provides a high reliability classification in terms of noise exposure to high energy (signal-to-noise ratio 0 dB).

The analysis of the prior art allowed to establish that analogues, characterized by a set of features that are identical to all the features of the claimed method for classifying segments of noisy speech, are absent. Therefore, the claimed invention meets the condition of patentability "novelty."

Search results for known solutions in this and related fields of technology in order to identify features that match the distinctive features of the claimed object from the prototype showed that they do not follow explicitly from the prior art. The prior art also did not reveal the popularity of the impact provided by the essential features of the claimed invention, the transformations on the achievement of the specified technical result. Therefore, the claimed invention meets the condition of patentability "inventive step".

The claimed invention is illustrated by the following figures:

FIG. 1 is a table of statistical characteristics of the duration of various speech sounds;

FIG. 2 is a correspondence table of combinatorial sets of classification features and weighting factors for certain groups, 12 (twelve) of which are phoneme groups and 1 (one) group characterizing a pause;

FIG. 3 is a functional block diagram of a device for classifying segments of noisy speech using multispectral analysis;

FIG. 4 is a graph of a comparative analysis of the effectiveness of the methods proposed in the claimed invention, the prototype and analogues;

FIG. 5 - the operation of the prototype and method according to paragraph 5 in the absence of interference;

FIG. 6 - the work of the prototype in conditions of intense noise (signal-to-noise ratio 0 dB);

FIG. 7 - the operation of the method according to paragraph 5 under the conditions of noise exposure to high energy (signal-to-noise ratio 0 dB).

Based on the assessment of the effectiveness of the proposed method according to the solution of the inventive problem, it can be said with confidence that the proposed method allows the classification of segments of the processed noisy speech signal to be increased by 7 dB in the signal to noise ratio range (the method proposed in the invention allows the correct classification of noisy speech segments at lower signal-to-noise ratios) with an average increase in the reliability of the correct classification by about 5.3%, the task sequence of the claimed inventions achieved.

Claims (10)

1. A method for classifying segments of noisy speech using multispectral analysis, including sequentially executed steps, according to which receive a noisy speech signal; perform analog-to-digital conversion with a predefined sampling rate and separation into quasistationary segments, calculate the Fourier spectra of the segments of the noisy speech signal and normalize them; pauses, noise and sound signals are distinguished in the normalized spectrum of amplitudes, then the segments are classified according to typical groups, for which the presence / absence of classification features, combinatorial sets of which characterize groups of phonemes whose parameters are preset, is determined on the basis of the initial speech signal and the normalized spectrum in the memory block and compare certain combinatorial sets of classification features of the segment with the predefined parameters of phoneme groups, simultaneously but they form sequences of characters denoting phoneme groups, characterized in that they use a multispectral analysis apparatus, namely, when evaluating the Fourier spectrum of the amplitudes of the speech signal obtained through the bispectrum of the speech signal, which is synthesized by the Fourier transform of the triple autocorrelation function in the quasistation section of the processed noisy speech signal, in addition, at the stage of deciding the classification of the processed segment of a noisy speech signal with reference to This group corrects the classification features of the groups using the hierarchy procedure, for which weights are introduced based on the intrasymbol and intersymbol relationships, both in the code word and inside each symbol, and obtained by evaluating the statistical and parametric features of both the speech signal and noise. 2. The method according to p. 1, characterized in that at the stage of obtaining the Fourier spectra of a noisy speech signal, the Fourier spectra are brought closer to the spectra of the original speech signal by partially suppressing the Gaussian component of the noise effect based on the section of the speech signal represented by an ergodic quasistationary random process, by the cumulant of the third order and then restore the Fourier spectra directly from the bispectrum of the processed segment of the speech signal. 3. The method according to p. 1, characterized in that in the direct decision-making process for classifying segments of noisy speech to a certain class of phoneme groups under the conditions of noise of the processed speech signal of Russian speech, a hierarchy procedure is introduced based on statistical features of the duration of various phonemes of Russian speech, for which we introduce the weighting coefficients of the classification features of twelve phoneme groups based on recursive relationships inside a codeword consisting of symbols denoting a background group . 4. The method according to p. 1, characterized in that in the direct decision-making process of clustering segments of noisy speech to a certain class of twelve phoneme groups and one group characterizing a pause, a hierarchy procedure based on parametric is introduced under the conditions of noisy processed speech signal of Russian speech features of the properties of noise and speech, for which we introduce the weight coefficients of the classification features of twelve phoneme groups and one group that characterizes the pause based on recursive connections three each symbol represents a group of phonemes, the code word. 5. The method according to p. 1, characterized in that at the stage of obtaining the Fourier spectra of a noisy speech signal, the Fourier spectra are brought closer to the spectra of the original speech signal by partially suppressing the Gaussian component of the noise effect based on the section of the speech signal represented by an ergodic quasistationary random process, by the cumulant of the third order and then restore the Fourier spectra directly from the bispectrum of the processed segment of the speech signal, then at the stage of deciding the classification of segments Noisy speech to a certain class of phoneme groups under noisy conditions of the processed speech signal of Russian speech introduces a hierarchy procedure based on statistical features of the duration of various phonemes of Russian speech, for which we introduce the weighting coefficients of the classification features of twelve phoneme groups based on recursive relationships inside a codeword consisting of of the symbols denoting a group of phonemes, while taking into account the hierarchy procedure based on the parametric features of the properties of noise and speech, To introduce weighting coefficients which classification features twelve groups and one group of phonemes characterizing the pause, based on recursive relationships within each symbol indicating a group of phonemes codeword. 6. A device for classifying speech segments, including a unit for receiving a noisy speech signal, connected in series with an analog-to-digital converter connected to a control unit, a memory unit and a unit for determining classification features of phoneme groups to form a sequence of characters representing phoneme groups, configured to generate current combinatorial combinations of classification features present in a noisy speech signal of the corresponding sequence characters denoting phoneme groups; recording a codeword marked up by the symbols of phoneme groups and called up from the memory block, correction blocks for segment classification attributes and a decision block are connected to a control unit configured to control the operating modes of these blocks, exchange information between them and interact with the memory block, characterized in that a block of segmentation of a speech signal is introduced, taking into account the mutual window of intersection of neighboring segments by the number of samples connected to the set of blocks of the multispectrum complete analysis, including a third-order correlation block, the output of which is connected to the input of the Fourier transform block, the output of which is connected to the input of the Fourier reconstruction of the amplitude spectrum from the bispectrum estimate, the output of which is connected to the input of the segment classification attribute determination block; blocks of correction of classification features, blocks of parametric and statistical hierarchies, and a block for storing short-term data have also been introduced; in addition, in the block for determining classification features of phoneme groups, a procedure for detecting a noise sign by adding an additional band-pass filter is implemented; the entered blocks are interconnected by multiple recursive connections, taking into account the intrasymbol and intersymbol communication in the generated sequence from the output of the block forming the sequence of characters. 7. The device according to claim 6, characterized in that data on combinatorial sets of classification features and weights of twelve groups of phonemes and one group characterizing a pause in the processed speech signal are pre-installed in the memory unit. 8. The device according to claim 6, characterized in that the classification feature determination unit is connected to the classification feature correction unit associated with the hierarchy determination units by introducing mutual recursive relationships when processing segments of a noisy speech signal in series and parallel. 9. The device according to claim 6, characterized in that the short-term data storage unit is connected with the blocks of the parametric and statistical hierarchy and the decision block by introducing mutual recursive relationships during serial-parallel processing of segments of a noisy speech signal. 10. The device according to claim 6, characterized in that data on combinatorial sets of classification features and weights of twelve groups of phonemes and one group characterizing a pause in the processed speech signal are predefined in the memory unit, in addition, the classification features determination unit is connected to the correction unit classification features associated with hierarchy determination units by introducing reciprocal recursive relationships when sequentially processing segments of a noisy speech signal, in this case, the short-term data storage unit is connected with the parametric and statistical hierarchy blocks and the decision block by introducing mutual recursive relationships during serial-parallel processing of segments of a noisy speech signal.

Images