JP2005202335A - Method, device, and program for speech processing - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To improve the articulation of a speech radiated from a speaker by processing a speech signal picked up by a microphone before it is outputted to the speaker. <P>SOLUTION: The speech signal which is digitized by an A/D converter 11 is inputted to a windowing processing part 12 for frame division and then passed through an FFT 13, and a logarithmic spectrum calculation part 14 calculates a logarithmic spectrum, which is subjected to IFFT 15 to generate a cepstrum coefficient. Then regression coefficient calculation parts 16-1 to 16-n calculates regression coefficients when the cepstrum coefficient is viewed in the time direction and a square averaging part 17 calculates the average (D value) of squares of the regression coefficient; and the D value is passed through a threshold processing part 18 to find a stationary part of the speech signal, and a multiplier 19 suppresses the amplitude of the speech signal for the found stationary part, so that the result is outputted through a D/A converter 20. <P>COPYRIGHT: (C)2005,JPO&NCIPI

Description

  The present invention relates to an audio processing method, apparatus, and program for performing processing for improving the intelligibility of audio that is loudened indoors.

  When lectures, lectures, etc. are performed in lecture halls, multipurpose halls, classrooms, churches, etc., the voice generated by the speaker is detected by a microphone, and after electrical processing such as amplification, it is installed in the venue The sound is emitted from the speaker as sound and finally reaches the audience's ear.

  In such a situation, the intelligibility of the sound radiated from the speaker is usually lowered due to the effect of reverberation in the room. In particular, for people with senile deafness or hearing impairment, the effects of such effects are great and the sound becomes very difficult to hear. Reverberation is also undesirable in voice communication using a language other than the listener's native language. For example, in a language listening experiment, if the same voice is played in a different reverberant environment, it may be disadvantageous for the examinee.

  For this problem, the sound signal detected by the microphone is subjected to specific preprocessing before it is output to the speaker, thereby improving the clarity of the sound emitted from the speaker and reaching the audience's ear. Various attempts have been made in the past. As one of them, the inventors are Takayuki Arai, Keisuke Kinoshita, Nao Hojima, Akiko Enomoto, Kyoko Kitamura, “Effects of speech suppression on reverberation”, Acoustical Society of Japan (Autumn Conference) Vol. 1, 449-450, October 2001 (Non-Patent Document 1), “Standing part” is intended to reduce overlap-masking due to reverberation to an input audio signal. It has been proposed to perform "suppression processing", and it has been confirmed that a reduction in intelligibility due to reverberation can be avoided under a certain noise environment.

  That is, the influence of overlap masking is considered as one of the factors that lower the intelligibility of speech due to reverberation. Overlap masking is the effect of masking the phoneme that is followed by the reverberation associated with the preceding phoneme, especially when the energy of the preceding phoneme is large and the energy of the following phoneme is small. Yes. In order to reduce such overlap masking, it is conceivable to thin out the original voice samples appropriately. However, if the thinning is simply performed, voice information is lost. As a result, the clarity is lowered.

  Therefore, in Non-Patent Document 1, a process of thinning out only the stationary part of the audio signal is performed. The stationary part of the speech signal is typically the center of the vowel part (syllable nucleus), and its energy is large but the amount of information as speech is small. On the other hand, it is known that the transition part of the speech signal plays a very important role in perception of speech information (for example, S. Furui, “On the role of spectral transition for speech perception,” J. Acoust. Soc Am., 80 (4): 1016-1025, 1986: Non-Patent Document 2). According to Non-Patent Document 2, as a result of listening experiments using stimuli in which the initial and final parts of the syllable were deleted at various positions, the transition part of speech played a very important role in terms of speech perception. It has been reported that the stationary part of is not necessary for vowel or syllable recognition.

  Since the steady portion of the vowel is generally large in energy among the steady portions of the audio signal, the transition portion and the consonant with low energy that follow it are easily susceptible to overlap. For this reason, when the steady part suppression process is performed, it is possible to reduce the masking amount to the transition part by overlap masking while minimizing the loss of voice information.

In Non-Patent Document 1, the following signal processing is specifically performed. First, an audio signal is band-divided by 1 / 3-oct using a filter bank such as an FIR filter, and a time envelope is extracted in each band. Next, the time envelope of each band is down-sampled to 100 Hz, and regression coefficients for a total of five points are calculated for each sample from two points before and after the logarithmic locus. The root mean square (hereinafter referred to as D value) of the regression coefficient is obtained over all the bands. Here, the D value represents a parameter indicating the spectrum transition of the audio signal according to Non-Patent Document 2. Next, after returning to the original sampling frequency, a portion where the D value is smaller than a certain threshold is regarded as a stationary portion, and the amplitude of the original waveform is suppressed for the stationary portion. By performing the steady-state suppression processing on the audio signal in this way, it is possible to reduce the influence of overlap masking due to reverberation and to prevent the speech intelligibility from being lowered.
Takayuki Arai, Keisuke Kinoshita, Nao Hojima, Akiko Enomoto, Kyoko Kitamura, “Effects of speech suppression on reverberation”, Acoustical Society of Japan (Autumn Conference), Vol. 1, 449-450, October 2001 S. Furui, “On the role of spectral transition for speech perception,” J. Acoust. Soc. Am., 80 (4): 1016-1025, 1986

  The steady-state suppression processing disclosed in Non-Patent Document 1 is effective in reducing overlap masking due to reverberation and avoiding a decrease in intelligibility due to reverberation, but particularly in a filter bank for band division. Due to the large processing delay, it is not necessarily suitable for real-time processing. Considering the original goal of improving speech intelligibility by preprocessing speech signals when the speech produced by the speaker is detected by a microphone and radiated by a speaker, the speaker is This means that the movement and movement of the mouth does not match the sound emitted from the speaker. Therefore, the real-time property of the steady-state suppression processing for improving the intelligibility is very important.

  The present invention provides an audio processing method, apparatus, and program for facilitating real-time processing of steady-state suppression processing for improving intelligibility before an audio signal detected by a microphone is output to a speaker. The purpose is to provide.

  In order to solve the above-described problems, the present invention is an audio processing method for performing processing on an input audio signal before being output to a speaker. A plurality of frames, a step of calculating a logarithmic spectrum for the audio signal of each divided frame, a step of calculating a cepstrum coefficient from the logarithmic spectrum, and a regression coefficient when the cepstrum coefficient is viewed in the time direction Calculating a root mean square of the regression coefficient, obtaining a stationary part of the speech signal by performing threshold processing on the mean square, and suppressing the amplitude of the speech signal for the steady part. It is characterized by comprising.

  The present invention is also an audio processing apparatus that performs processing on an input audio signal before being output to a speaker, and performs a windowing process on the audio signal to divide the audio signal into a plurality of frames. A windowing processing unit, a Fourier transform unit that performs a Fourier transform on the audio signal of each frame divided by the windowing processing unit, a logarithmic spectrum calculation unit that calculates a logarithmic spectrum based on an output signal from the Fourier transform unit, A cepstrum coefficient calculation unit that generates a cepstrum coefficient by performing an inverse Fourier transform on a logarithmic spectrum, a regression coefficient calculation unit that calculates a regression coefficient when the cepstrum coefficient is viewed in the time direction, and a mean square of the regression coefficient A mean square unit for obtaining the threshold value, a threshold value processing unit for obtaining a stationary part of the audio signal by performing threshold processing for the mean square, and a voice for the stationary part Characterized by comprising a suppression unit for suppressing the amplitude of No..

  Furthermore, according to the present invention, there is provided a program for causing a computer to perform audio processing for performing processing on an input audio signal before being output to a speaker, and performing a windowing process on the audio signal. A process of dividing the audio signal into a plurality of frames, a process of calculating a logarithmic spectrum for the audio signal of each divided frame, a process of calculating a cepstrum coefficient from the logarithmic spectrum, and the cepstrum coefficient viewed in the time direction A process for calculating a regression coefficient in the case, a process for obtaining a root mean square of the regression coefficient, a process for obtaining a stationary part of the speech signal by performing threshold processing on the mean square, and the speech for the steady part It is also possible to provide an audio processing program that causes the computer to perform processing for suppressing the amplitude of a signal.

  It is possible to effectively improve the intelligibility of the sound radiated from the speaker for the hearing impaired and the elderly by performing the steady-state suppression processing on the sound signal detected by a microphone or the like. In addition, real-time processing can be easily realized.

  Embodiments of the present invention will be described below with reference to the drawings. FIG. 1 shows an example of a voice enhancement system to which a voice processing device according to an embodiment of the present invention is applied. In the room 1 such as a lecture hall, a multipurpose hall, a classroom, or a church, a voice generated by a speaker 2 giving a lecture / lecture is detected by a microphone 3. An audio signal output as an electrical signal from the microphone 3 is amplified by the preamplifier 4 and then input to the audio processing device 5 according to an embodiment of the present invention.

  The audio processing device 5 performs signal processing for improving the intelligibility of the input audio signal, that is, as described in detail later, in order to reduce the influence of overlap masking due to reverberation, A process for suppressing the amplitude is performed. The audio signal processed by the audio processing device 5 is amplified by the power amplifier 6, then supplied to the speaker 7 installed in the room 1, and finally radiated as sound from the speaker 7, so that the audience 8 finally becomes. Reach the ears.

  FIG. 2 shows the state of overlap masking due to reverberation. As speech, “October” (speaker: EngM2, male) from the University of Tsukuba Multilingual Speech Corpus was used. FIG. 2A shows the original speech waveform, and the bottom line of FIG. 2A is / o /, / k /, / t /, / o /, / b /, / er / by segmentation of the top 5 lines. This is a waveform obtained by adding the voice waveforms divided into two. Fig. 2 (b) is a speech waveform obtained by convolving an impulse response with a reverberation time of 1.1 seconds into the speech waveform of Fig. 2 (a). Consonants with weak energy such as / k /, / t /, / b / It can be seen that it is masked by the reverberation added to the immediately preceding vowel. That is, when the preceding sound is a strong energy phoneme such as a vowel, the subsequent phoneme is greatly affected by the tail of the reverberation.

  Therefore, in the speech processing device 5, the influence of overlap masking due to reverberation is reduced by suppressing in advance the steady portion of the speech signal that is relatively large in energy but not so important for speech recognition. Achieve improvement. Hereinafter, the audio processing device 5 will be described in detail with reference to FIG.

  In FIG. 3, the audio signal amplified by the preamplifier 4 shown in FIG. This input audio signal is sampled by the A / D converter 11 at a sampling frequency of 16 kHz, for example, and converted to a digital signal of about 16 bits. The digitized audio signal output from the A / D converter 11 is first input to the windowing processing unit 12 and subjected to windowing processing using, for example, a 20 ms Hanning window or Hamming window.

  That is, the windowing processing unit 12 uses a cepstrum coefficient, which will be described later, to detect a stationary part of a vowel, so that the digitized speech signals overlap each other by a time length of, for example, 10 ms (50%). A plurality of long frames are cut out, and then a windowing process using a Hanning window or a Hamming window having the same width of 20 ms is performed.

  The audio signal of each frame output from the windowing processing unit 12 is input to the fast Fourier transform (FFT) unit 13 and subjected to FFT. The logarithmic spectrum of the audio signal of each frame is calculated by the logarithmic spectrum calculation unit 14 from the output signal of the fast Fourier transform unit 13. Specifically, the logarithmic spectrum calculation unit 14 performs square calculation after taking an absolute value to obtain a power spectrum for the output signal of the fast Fourier transform unit 13, and then calculates 10 * log 10 to calculate dB ( Convert the unit to decibels to obtain the logarithmic spectrum of the output.

  Next, the log spectrum calculated by the logarithmic spectrum calculation unit 14 is subjected to IFFT by an inverse Fourier transform (IFFT) unit 15 to generate a cepstrum coefficient. Of the generated cepstrum coefficients, low-dimensional coefficients represent the spectral envelope of the audio signal. Therefore, by performing liftering on the cepstrum coefficients, for example, cepstrum coefficients up to the 30th order representing the spectrum envelope are left and output. FIG. 4 shows a logarithmic spectrum 41 (solid line) with respect to the original waveform of the audio signal input to the input terminal 10 and a spectrum envelope 42 (broken line) which is a cepstrum coefficient up to the 30th order.

  Next, each cepstrum coefficient up to, for example, the 30th order generated and lifted by the inverse Fourier transform unit 15 is input to the regression coefficient calculation units 16-1 to 16-n (in this case, n = 30), and each cepstrum is input. For example, a total of 5 regression coefficients, for example, 2 points before and after the coefficient time trajectory are calculated for each sample by the least square method. As another example, a total of 7 regression coefficients may be calculated for each sample, 3 points before and after the time trajectory of each cepstrum coefficient.

  In FIG. 5, the cepstrum coefficient of the five time trajectories is indicated by a solid line, and the regression line is indicated by a broken line. The slope of the regression line is the regression coefficient (delta coefficient). In this case, since 30th-order cepstrum coefficients are used, 30 delta coefficients are obtained per frame.

  Next, the mean square of the 30 delta coefficients, which are the regression coefficients calculated by the regression coefficient calculators 16-1 to 16-n, is calculated by the mean square unit 17, which is representative of one frame. Let D value. The D value is a parameter that is defined according to Non-Patent Document 2 and indicates the spectrum transition of the audio signal, and is obtained one for each frame.

  FIG. 6 shows an example of an original waveform 61 (filled portion) of the vowel part of the audio signal and a D value 62 (portion drawn with a line) that is a mean square obtained by the mean square unit 17. A portion with a small D value corresponds to a stationary part of a vowel. Therefore, the D value is input to the threshold processing unit 18 and compared with a predetermined threshold, and a portion where the D value is smaller than the threshold is set as a vowel stationary part. The output of the threshold processing unit 18 is, for example, a stationary part detection signal composed of a binary signal such that α (0 ≦ α <1) in the stationary part of the vowel and 1 in other parts. In this example, α = 0.4, but any value is acceptable as long as 0 ≦ α <1. The stationary part detection signal is input to the multiplier 19 and is multiplied by the digitized speech signal output from the A / D converter 11, whereby the amplitude of the speech signal is suppressed for the stationary part.

  FIG. 7 shows an original waveform 71 (lightly painted portion and darkly painted portion) of the audio signal and a waveform 72 (darkly painted portion) after the steady portion is suppressed. The audio signal after the stationary part suppression processing from the multiplier 19 is output from the output terminal 21. The audio signal output from the output terminal 21 is input to, for example, the power amplifier 6 of FIG.

  As described above, according to the sound processing apparatus of the present embodiment, the process of suppressing the amplitude of the stationary part of the input sound signal can be performed, so that the processed sound signal is converted into the power amplifier 6 as shown in FIG. By supplying to the speaker 7 installed in the room 1 via the voice, a voice with high intelligibility can be emitted.

  In addition, in the speech processing apparatus of the present embodiment, in order to perform the intelligibility improvement processing to reduce the influence of overlap masking due to reverberation in units of frames of the input speech signal, the speech signal is divided into bands by a filter bank. Compared to Non-Patent Document 1 that performs the same processing, the processing delay is very short, and real-time processing becomes easy.

  The audio processing apparatus shown in FIG. 3 performs processing from the output of the A / D converter 11 to the D / A converter 20 by software processing using a DSP (Digital Signal Processor) or a general-purpose CPU (Central Processing Unit). It can also be realized. It is also possible to implement the speech processing apparatus shown in FIG. 3 using dedicated hardware.

Next, the result of a listening experiment conducted to confirm the effect of the embodiment of the present invention will be described. First, the results of listening experiments in a laboratory environment will be described.
The reverberation environment was realized by convolving the speech signal and the reverberant impulse response on a computer. The impulse response used was based on the impulse response measured at the Higashiyamato City Dai Hall (without reflector) and was artificially processed to change the reverberation time from 0.4 seconds to 1.3 seconds. It is.

  The stimulus was targeted to Japanese monosyllable CV (consonant-vowel) and inserted into the Japanese career sense "I'll call it _". / A /, / i / as V and / p /, / t /, / k /, / b /, / d /, / g /, / s /, / ∫ /, / h /, as C 14 consonants of / t∫ /, / dz /, / d3 /, / m /, / n / were used. After all, 24 types of CV were used in the experiment. For each stimulus, a Japanese speech database for ATR research (speaker: MAU, 40-year-old male) was used. There are two types of stimulus sounds: a stimulus set in which reverberation is convoluted in the original speech signal (no processing), and a stimulus set in which reverberation is convoluted after performing steady-state suppression processing according to the embodiment of the present invention (with processing). Prepared. The subjects were 44 normal listeners whose native language was Japanese (22 for a set with a short reverberation time and 22 for a long set).

  The instruction of the experiment was performed on the computer screen in the soundproof room. The instruction of the stimulation sound was adjusted to a sound pressure level suitable for each subject using headphones (STAX SR-303). In each trial, first, the stimulus sound was presented only once, and after the presentation, 24 types of CV used in the experiment were displayed on the screen as options as kana. The subject was forced to click one of the choices on the screen with a mouse and responded. When the selection is over, the next stimulus is automatically presented. A total of 240 stimuli (5 types of reverberation × 24 single syllables × 2 types of treatment) were randomly presented to each subject.

As a result of the single syllable intelligibility test in the laboratory environment under the above conditions, the average values of consonant correct answer rates under each reverberation condition and processing condition are shown in Table 1 (set with a short reverberation time) and Table 2 (Reverberation). Long set).

  However, the accuracy rate of vowels was 100% under all conditions. The main effects of treatment were all significant (p <.001). As a result of t-test between processing conditions, in Table 1, “with processing” is shown when reverberation time is 0.8, 0.9, 1.0 seconds, and in table 2, “processing is done” when reverberation time is 0.9, 1.0, 1.1, 1.2 seconds. “” Was significantly higher in accuracy.

  From these experimental results, the accuracy rate was higher for “with treatment” under all reverberation conditions, and the effect of the treatment was confirmed when the reverberation time was 0.8 to 1.2 seconds.

  Next, a result of an experiment conducted in the university auditorium in order to confirm the effect of the steady-state suppression processing, which has been effective in the laboratory environment described above, even in an actual reverberant environment is shown. In the experiment, a single syllable intelligibility test and a sentence writing test were performed.

  In the single syllable intelligibility test, among the stimuli described above, those having a vowel of / a / (14 single syllables, with carrier sentence, with / without processing) were used. In the sentence comprehension test, NTT-AT phoneme balance 1000 sentences to 20 sentences were used. The subjects were 24 normal hearing listeners whose native language was Japanese.

  The experiment was conducted in the Hall No. 10 auditorium, which has the largest capacity (822 people) on the campus of Sophia University. A speaker was installed on the platform, and the stimulation sound prepared in advance from the PC was reproduced. The subject was placed in the rear block in front of the auditorium. First, after giving instructions to the subjects, the output was adjusted to a level that all the subjects could hear without any problems using test stimulus sentences.

  In the single syllable intelligibility test, 28 stimuli (with or without treatment for each of the 14 single syllables) were presented in a total of 56 stimuli randomly arranged twice. In each trial, the stimulating sound was presented only once, and one answer was forcibly selected from the list of 14 single syllables and written on the form. The time until the next stimulus presentation was 5 seconds.

  In the sentence intelligibility test, 24 subjects were divided into group A (13 people) and group B (11 people), and experiments were conducted for each group. Each group presented 20 different sentences, that is, 10 sentences with “processing” and 10 sentences without “processing” at random. Also, 10 sentences that were “processed” in group A become “no process” in group B, and conversely, 10 sentences that were “no process” in group A became “processed” in group B. Balanced by combining so. In each trial, the stimulating sound was presented twice at 20-second intervals, and the responses were written on a paper in kana.

  In the single syllable intelligibility test, the correct answer rate of consonants was compared, and “corrected” (69.3%) was higher than “not processed” (62.7%). In the sentence comprehension test, the written sentences were compared with and without processing. As a result, the accuracy rate for each mora was high at 95% or more for both “with processing” and “without processing”, and almost no difference was observed.

  In the monosyllable intelligibility test, the same stimulus was used as in the case of diotic listening in the laboratory environment, but the effect was also confirmed in the dichotic environment. Since context information can be used in writing a sentence, even if there is some difficulty in hearing, there is no problem in the case of a normal hearing person. The stimulus sentence used this time was relatively simple, was spoken by a well-trained announcer slowly and clearly, had a reverberation time that was not so long, and had strong direct sound energy. This is considered to be the reason why the intelligibility was high. However, in a worse reverberant environment, if there are words with low intimacy, or if the speech speed is fast or unclear as seen in spontaneous speech, the processing effect according to the embodiment of the present invention is effective. It is expected to appear prominently. This is especially true for the elderly and the hearing impaired.

1 is a conceptual diagram of a voice enhancement system using a voice processing device according to an embodiment of the present invention. Diagram showing an example of overlap masking due to reverberation The block diagram which shows the structure of the audio | voice processing apparatus according to one Embodiment of this invention. Diagram showing logarithm spectrum and spectrum envelope of original speech waveform Diagram showing examples of regression coefficient calculation The figure which shows the example of the root mean square (D value) of an original speech waveform and a regression coefficient The figure which shows the example of the voice waveform where the original voice waveform and the stationary part were suppressed

Explanation of symbols

DESCRIPTION OF SYMBOLS 10 ... Input terminal 11 ... A / D converter 12 ... Windowing process part 13 ... Fast Fourier transformer 14 ... Logarithmic spectrum calculation part 15 ... Inverse fast Fourier transformer 16-1 to 16-n ... Regression coefficient calculation part 17 ... Root mean square calculation unit 18 ... Threshold processing unit 19 ... Multiplier 20 ... D / A converter 21 ... Output terminal

Claims (3)

An audio processing method for performing processing on an input audio signal before being output to a speaker,
Performing a windowing process on the audio signal to divide the audio signal into a plurality of frames;
Calculating a logarithmic spectrum for the audio signal of each divided frame;
Calculating a cepstrum coefficient from the logarithmic spectrum;
Calculating a regression coefficient when the cepstrum coefficient is viewed in the time direction;
Obtaining a root mean square of the regression coefficients;
Obtaining a stationary part of the audio signal by performing threshold processing on the mean square;
A voice processing method comprising: suppressing the amplitude of the voice signal for the stationary part. An audio processing device that performs processing on an input audio signal before being output to a speaker,
A windowing processing unit that performs windowing on the audio signal and divides the audio signal into a plurality of frames;
A Fourier transform unit that performs Fourier transform on the audio signal of each frame divided by the windowing processing unit;
A logarithmic spectrum calculation unit for calculating a logarithmic spectrum based on an output signal from the Fourier transform unit;
A cepstrum coefficient calculation unit that generates a cepstrum coefficient by performing an inverse Fourier transform on the logarithmic spectrum;
A regression coefficient calculation unit for calculating a regression coefficient when the cepstrum coefficient is viewed in the time direction;
A mean square part for obtaining a mean square of the regression coefficient;
A threshold processing unit that obtains a stationary part of the audio signal by performing threshold processing on the mean square;
An audio processing apparatus comprising: a suppression processing unit that suppresses an amplitude of the audio signal for the stationary unit. A program for causing a computer to perform audio processing for performing processing on an input audio signal before being output to a speaker,
Performing a windowing process on the audio signal to divide the audio signal into a plurality of frames;
A process of calculating a logarithmic spectrum for the audio signal of each divided frame;
Calculating cepstrum coefficients from the logarithmic spectrum;
A process of calculating a regression coefficient when the cepstrum coefficient is viewed in the time direction;
A process for obtaining a mean square of the regression coefficients;
Processing for obtaining a stationary part of the audio signal by performing threshold processing on the mean square;
An audio processing program for causing the computer to perform processing for suppressing the amplitude of the audio signal for the stationary part.

Images