JPH07146700A - Pitch emphasizing method and device and hearing acuity compensating device - Google Patents

Abstract

PURPOSE:To attain the helping to the pitch sensations of hearing-impaired persons by providing a means extracting fundamental frequencies of a voice and a means emphasizing levels of fundamental frequencies of a voice spectrum and improving the tone quality of the voice. CONSTITUTION:A cepstrum analyzing part 108 obtains the power spectrum of a frequency spectrum 107 and calculates a cepstrum 109 by performing the inverse Fourier trnsformation of its logarithm. A fundamental frequency extracting part 110 extracts a fundamental frequency 111 from the cepstrum 109. Then, a pitch emphasizing part 113 emphasizes pitches by changing the level of the fundamental frequency position of a frequency spectrum 112. A frequency spectrum 114 whose pitches are emphasized is reconstructed into a continuous wave form 118 in a wave reconstructing part 117 after being inversely converted into a shirt time sectional signal 116 in an inverse Fourier transformation part 115 and further converted into an analog voice signal 120 in a D/A converter part 119 to be outputted as a voice whose pitches are emphasized.

Description

Detailed Description of the Invention

[0001]

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention relates to a sound processing device,
In particular, the present invention relates to an audio signal processing device for the purpose of compensating the hearing of a hearing impaired person and recovering deteriorated audio.

[0002]

2. Description of the Related Art Most conventional hearing aids such as hearing aids amplify an input voice by an analog amplifier and reproduce it.

In recent years, with the progress of digital technology, it has become possible to execute complicated processing in a shorter time. Along with this, for example, Working Group on Communicat
ion Aids for the Hearing- "Speech by Impaired"
Perception AIDS for Hearing Impaired People: Current Status and Needed
Research (Speech-perception aids for hearing-impair
ed people: Current status and needed research) "
(J. Acoust. Soc. Am. 90 (2), Pt. 1, August 1991,
pp637-685), a method of finely adjusting the amplification factor for each frequency according to the frequency characteristics of the audible range of the hearing impaired person, and moving the audio information in the frequency direction to the area where the hearing remains. There have been proposed methods, methods for selectively emphasizing various characteristics of voice, and the like.

On the other hand, human beings estimate the intention of the speaker by carrying out speech distinction, homonymation, and intention expression based on the pitch of the voice, that is, the time change of the pitch. In general, the pitch in speech is perceived by humans by the auditory system to analyze the acoustic characteristics based on the fundamental frequency of the vocal cord vibration of the speaker, and plays an important role in transmitting will by voice. It is a characteristic of having. In the auditory system, it is considered that pitch perception is comprehensively performed based on the temporal characteristics of the fine structure of the waveform, the fundamental frequency, and the frequency characteristics of the fundamental frequency and its harmonic components. There is. A hearing person can perceive pitch if some of this information is obtained, but if frequency selectivity or time resolution is reduced due to hearing impairment,
Lack of full utilization of these voice features often makes pitch perception difficult. For example, by Moore and Peters, "(Pitch Discrimination and Phase Sensitivity in Young and Elderly Subjects and It's Relationship to Frequency Selectivity. (Pitch discrimination and
phase sensitivity in young and elderly subjects an
d its relationship to frequency selectivity) ",
J. Acoust. Soc. Am. 91 (5), May 1992, pp2881-289
See 3.

By the way, when voice is transmitted through a communication line or the like, usually, due to the restriction of the information transmission rate of the transmission system,
Bandwidth is limited to reduce the amount of information. For example, the telephone line is band-limited to about 300 Hz to 3.4 kHz. Usually, the pitch of voice is 80 to 400 Hz, so that pitch information is missing or attenuated when the pitch frequency deviates from the telephone band in men who have a relatively low pitch.
In such a case, especially for a hearing-impaired person whose voice analysis function has deteriorated, it becomes difficult to perceive the pitch because the information about the pitch that can be obtained from the voice is insufficient. Also,
The hearing quality is also felt by the hearing-impaired.

As a method for hearing-impairing a hearing impaired person with respect to pitch information, a method of giving a sine wave modulated in accordance with a change in fundamental frequency of voice as auxiliary information at the time of lip reading, for example, to a person with a particularly high hearing impairment, is known. , Grant (Gran
"Encoding voice pitch for profoundly hears"
ing-impaired listerners) "J. Acoust. Soc. Am. 82
(2), August 1987, pp423-432. Further, a method of applying a filter having an inverse characteristic of the telephone characteristic to the voice signal in order to compensate the band limitation for the telephone sound and bring it closer to the original sound can be considered.

[0007]

Of the above-mentioned conventional techniques,
The hearing aid system that provides a sine wave that is modulated according to the change in the fundamental frequency of the voice does not directly process the voice. Therefore, it is not applied to a hearing aid device for the purpose of understanding speech contents using only voice information.

In addition, in an actual transmission system of a telephone, since information other than a voice signal exists outside the band due to noise addition, transmission distortion, etc., the method of applying a filter having an inverse characteristic of the telephone characteristic causes noise. Amplification at the same time may impair clarity and ease of listening.

An object of the present invention is to enhance the fundamental frequency component of voice, thereby improving the sound quality of voice in which the fundamental frequency component is missing or attenuated due to band limitation of telephone voice and the like, and a voice analysis function required for pitch perception. It is an object of the present invention to provide a pitch emphasizing method and device, and a hearing compensating device, which can assist the pitch perception of a hearing impaired person with reduced hearing loss.

[0010]

In order to solve the above problems, in the pitch emphasizing apparatus of the present invention, a means (fundamental frequency extraction unit) for extracting a fundamental frequency of a voice and a voice based on the extracted fundamental frequency are used. And a means for emphasizing the level of the fundamental frequency position of the spectrum (fundamental frequency level changing unit).

As a first means for changing the level of the fundamental frequency position of the speech spectrum in the fundamental frequency level changing unit, a method of multiplying the frequency spectrum of the speech by a window of the fundamental frequency width centered at the fundamental frequency position is used. You can

As a second means for changing the level of the fundamental frequency position of the speech spectrum in the fundamental frequency level changing unit, a method of adding a window of the fundamental frequency width centered at the fundamental frequency position to the frequency spectrum of the input speech is used. be able to.

As a third means for changing the level of the fundamental frequency position of the voice spectrum in the fundamental frequency level changing unit, the component of the fundamental frequency width of the fundamental frequency position of the frequency spectrum of the input voice is centered at the fundamental frequency position. A method of replacing with a window of the fundamental frequency width can be used.

[0014]

The pitch emphasizing device according to the present invention performs pitch emphasizing by changing the level of the fundamental frequency position of the speech spectrum with respect to the component of the fundamental frequency position of the speech spectrum. As a result, it is possible to improve the sound quality of the voice in which the fundamental frequency component is missing or attenuated due to the band limitation of the telephone voice or the like, or to assist the pitch perception of the hearing impaired person whose voice analysis function necessary for pitch perception is deteriorated.

Generally, in a voice transmitted over a telephone line or the like, a fundamental frequency component is attenuated due to band limitation. If the level of such speech is low, but the fundamental frequency component remains, amplify only the level of the fundamental frequency component while preserving the original speech shape before passing through the transmission system. To emphasize the pitch. For that purpose, the window processing is performed on the fundamental frequency position of the frequency spectrum.
At this time, in the limited band other than the fundamental frequency position,
There may be components other than the original voice due to noise. Therefore, in order to not amplify the components other than the fundamental frequency, a window of the fundamental frequency width having the center at the fundamental frequency position is added or added. This makes it possible to enhance the pitch while improving the sound quality.

When the attenuation is steep, or due to the influence of line distortion, the fundamental frequency component is deformed with respect to the original voice component, or the original voice component almost remains. If not, it is possible to emphasize the pitch while improving the sound quality by replacing the fundamental frequency position of the speech frequency spectrum with a window that simulates the fundamental frequency component.

Further, if the level after the spectrum is changed by using the window can be changed according to the taste and hearing characteristics of the listener, it can be personalized. It is also possible to adjust the level to be equal to the level of the fundamental frequency component of the reference voice. At that time, if the level of the fundamental frequency component of the reference voice is measured, for example, by measuring the attenuation characteristic of the transmission system using a device such as a spectrum analyzer and estimated from the attenuation rate at the fundamental frequency position, the pitch-emphasized voice will be obtained. , It is possible to approximate the sound quality of the voice before passing through the transmission system.

[0018]

Embodiments of the present invention will be described below with reference to the drawings.

FIG. 1 is a block diagram for explaining an embodiment of the pitch emphasizing device according to the present invention. In FIG. 1, an input voice signal 101 is converted into a digital waveform signal 103 by an A / D converter 102, and a frame processor 104
In, a continuous short-time section waveform signal 105 of about several tens of milliseconds to hundreds of milliseconds is obtained by using an appropriate time window. The short-time section waveform signal 105 is converted into frequency spectra 107 and 112 by the Fourier transform unit 106. The cepstrum analysis unit 108 calculates the cepstrum 109 by obtaining the power spectrum of the frequency spectrum 107 and performing the inverse Fourier transform of the logarithm thereof. From this cepstrum 109, the fundamental frequency extraction unit 110 extracts the fundamental frequency 111. The fundamental frequency extraction method based on cepstrum analysis is described in, for example, "Digital Speech Processing" (Furui, Tokai University Press, 1985). The pitch emphasizing unit 113 has a frequency spectrum 112.
The pitch is emphasized by changing the level (amplitude) at the fundamental frequency position of. 114 is a pitch enhancement unit 113
It is a frequency spectrum with a pitch emphasis output from. The frequency spectrum 114 in which the pitch is emphasized is inversely transformed into the short time period waveform signal 116 by the inverse Fourier transform unit 115, and then the continuous waveform 11 is obtained by the waveform reconstructing unit 117.
Rebuilt to 8. Then, after being converted into the analog audio signal 120 in the D / A converter 119, it is output as pitch-enhanced audio.

The fundamental frequency extraction method includes, in addition to the cepstrum analysis described in the above embodiment, a method using an autocorrelation function, a method for measuring the number of zero-crossings of a waveform, and the like. However, this pitch emphasizing device can be realized.

FIG. 2 is a conceptual diagram of pitch enhancement. It is assumed that the frequency spectrum of a short time period with the input voice is 201. Reference numeral 202 is an assumption of frequency characteristics of a certain transmission system, and 201 is assumed to be attenuated in a limited frequency band as a result of being transmitted through a system having such characteristics. In the pitch emphasizing apparatus of the present invention, the pitch is emphasized by increasing and changing the spectrum level at the fundamental frequency F0 position of the input voice like 203. As a method of changing the spectrum level, a window on the frequency axis as shown in FIG. 3 is used, and the spectrum is multiplied by a function of the window, added, or replaced by the window. The details will be described below.

FIG. 3 is an example of a window for emphasizing the fundamental frequency component. In FIG. 3, F0 represents the fundamental frequency, and the center and width on the frequency axis are F0. When the shape of the window is defined by the function W (f), the emphasis window is expressed by the following equation. However, W (f) has a maximum value of 1 when f = F0.

[0023]

[Equation 1]

When the fundamental frequency component is changed by multiplying the voice frequency spectrum by the function of this window, the changed spectrum S (f) is the spectrum S0 before the change.
It is expressed by the following equation with respect to (f).

[0025]

[Equation 2]

Here, a is a variable that defines the amplification degree.

When the fundamental frequency component is changed by adding a window function to the voice frequency spectrum, the changed spectrum S (f) is also

[0028]

[Equation 3]

It is represented by

When the fundamental frequency component is changed by replacing the voice frequency spectrum with a window function, the changed spectrum S (f) is

[0031]

[Equation 4]

It is represented by Here, b represents an offset with respect to the level, and by appropriately providing both variables a and b, a window close to the shape of the fundamental frequency component of the original speech is realized.

When the noise level near the fundamental frequency component is high, the pitch can be further emphasized by selecting a window function that suppresses components other than the fundamental frequency position, as shown in FIG. In FIG. 4, fc represents the frequency width to be suppressed. The window function is defined by the following equation.

[0034]

[Equation 5]

The shape of the window does not have to be bilaterally symmetric, and may have a shape as shown in FIGS. 5 and 6, for example.
However, if a function with a steep slope near the boundary is used, the spectrum after the change becomes discontinuous and the sound quality may deteriorate.

The following equation shows an example of W (f) corresponding to FIG.

[0037]

[Equation 6]

However,

[0039]

[Equation 7]

It is Note that σp and σn represent standard deviations. W (f) in Expression 6 is a combination of Expressions representing a Gaussian distribution and has a so-called Mexican hat shape.

FIG. 7 is a diagram showing level compensation based on the frequency characteristic of the transmission system. Now, let 701 be the frequency characteristic of the transmission system. In addition, the fundamental frequency components of the audio frequency spectrum before and after passing through the transmission system are respectively 702 and 703.
And The frequency characteristic of the transmission system can be measured using a spectrum analyzer or the like. From 701, the attenuation rate G at the fundamental frequency position F0 is obtained. If the level at F0 of the voice frequency spectrum before change is L0, the level L at the F0 position of the voice before passing through the transmission system is

[0042]

[Equation 8]

It becomes Therefore, the pitch is emphasized by using a window in which the parameters a and b of Expressions 2, 3, and 4 are adjusted so that the level at the fundamental frequency position becomes L.

FIG. 8 is an example of a frequency spectrum of a voice in which pitch is emphasized according to the present invention. As can be seen from the figure, the spectrum level is enhanced at the fundamental frequency position F0.

In the above embodiments, the mathematical expression is used to realize the window for emphasizing the fundamental frequency component. However, in addition to using the mathematical expression, data is stored in a table in advance and a so-called table look is taken out to extract a necessary portion. It is also possible to use up. In this method, since the data is treated as a discrete quantity, the emphasis window is also discretized and stored in the table. However, the window length changes depending on the fundamental frequency, and the number of discrete points also changes depending on the sampling frequency and the FFT size. Therefore, in the table, a discrete amount in which a window having a predetermined shape is represented by a sufficiently large number of points is stored as a standard value, and is thinned out and used according to the situation. FIG. 9 shows an example of a table storing standard values of such window functions for emphasizing the fundamental frequency component. Here, N1 and N2 are the numbers of points from the center 0 of the window to both ends, and are sufficiently large integers. If the windows are symmetrical, N1 and N2 will be equal. Further, X [·] is the window function value at each point.

FIG. 10 shows an example of setting the window function value for each point when the window shape 1001 is used. Now, assuming that the extracted fundamental frequency is F0, the sampling frequency Fs, and the FFT size is NFFT, the number of points M1 and M2 from the center to both ends of the window actually used is expressed by a time equation.

[0047]

[Equation 9]

[0048]

[Equation 10]

However, int (·) represents a function for converting a real number into an integer.

When the center position point is 0, the position j in the table referred to by the i-th point of the window actually used is

[0051]

[Equation 11]

It becomes Therefore, the window function value W (fi) at the frequency fi at the i-th point is

[0053]

[Equation 12]

It becomes

FIG. 11 shows an example of the hardware configuration of the pitch emphasizing device of this embodiment. This device 1100 includes an amplifier 1102 that receives and amplifies an input audio signal 1101 (corresponding to 101 in FIG. 1), a filter 1103 that receives the output of this amplifier, and an analog digital (converts the analog output of this filter into a digital signal). A / D) converter 11
04, a program memory 1106 that stores a processing program including pitch enhancement processing, and a digital signal processor (DSP) 1105 that performs digital signal processing based on this program. The DSP 1105 allows the elements 104, 106, 108, 110, 11 of FIG.
The processes of 3, 115, and 117 are executed. Device 1
Reference numeral 100 denotes a data memory 1107 and a DSP 11 which are used as a storage area for input data before processing of the DSP 1105 and data of processing results, or a work area.
Digital-analog (D / A) converter 1108 for converting the data of the processing result of 05 into an analog signal, filter 1109 and amplifier 111 for receiving the output of this D / A converter.
Equipped with 0. The output 1111 of the amplifier 1110 corresponds to the analog audio signal 120 of FIG.

FIG. 12 shows an example of actual use of the pitch emphasizing device 1100 shown in FIG. In this example, a microphone 1201 is connected to the input side and a speaker 1202 is connected to the output side for use.

FIG. 13 shows another usage example of the pitch emphasizing device 1100. This example is applied to a telephone for an analog line. The voice signal input to the analog telephone 1302 through the transmission line 1301 is input to the pitch enhancing device 11
00, and after emphasizing, handset 13
The speaker 1304 of No. 03 outputs the sound whose pitch is emphasized. The input from the pickup microphone 1305 of the handset 1303 is directly sent to the telephone 1303.

FIG. 14 shows an example in which the pitch emphasizing device 1100 is applied to a telephone for a digital line. In the telephone 1402 using a digital line, the input signal is already digitized, so that the filter unit 1103 and the A / D conversion unit 1104 (FIG. 11) can be omitted.

FIG. 15 shows an application example in which the pitch emphasizing device 1100 is used as a preprocessing unit of a transmitter. By using the pitch emphasizing device in the pre-processing unit of the audio signal input of a transmitter such as a television or a radio, it becomes possible to transmit the sound in which the pitch is emphasized in advance.

FIG. 16 shows an application example in which the pitch emphasizing device 1100 is used as a post-processing unit of a receiver. By using this pitch enhancement device as a post-processing unit of a receiver 1604 of a television, a radio, etc., it becomes possible to enhance the received audio signal.

FIG. 17 shows a pitch emphasizing device 1 with a variable emphasizing amount.
An example of 100 is shown. An external volume 1701 is used to change the pitch emphasis amount from the outside by an emphasis amount controller 1702.

[0062]

By changing the component of the fundamental frequency position of the speech spectrum by using the window function of the fundamental frequency width, it is possible to amplify unnecessary information such as ambient noise without amplification.
Only pitch information is emphasized. As a result, it has become possible to assist the pitch perception of a hearing impaired person whose voice analysis function has deteriorated.

[Brief description of drawings]

FIG. 1 is a block diagram illustrating an embodiment of a pitch emphasizing device according to the present invention.

FIG. 2 is a conceptual diagram of pitch enhancement according to the present invention.

FIG. 3 is a waveform diagram showing an example of a window for emphasizing a fundamental frequency component in the example.

FIG. 4 is a waveform diagram showing an example of a window for suppressing the vicinity of the fundamental frequency component in the embodiment.

FIG. 5 is a waveform diagram showing an example of an asymmetric window for emphasizing a fundamental frequency component in the example.

FIG. 6 is a waveform diagram showing an example of an asymmetric window that suppresses the vicinity of the fundamental frequency component in the embodiment.

FIG. 7 is an explanatory diagram showing level compensation based on frequency characteristics of the transmission system in the example.

FIG. 8 is a waveform diagram showing an example of a frequency spectrum of a voice in which pitch is emphasized according to the present invention.

FIG. 9 is an explanatory diagram of a table that stores standard values of window functions that can be used in the embodiment.

10 is an explanatory diagram of a window setting method using the table of FIG.

FIG. 11 is a block diagram showing a hardware configuration example of the pitch emphasizing device of the embodiment.

FIG. 12 is a block diagram showing an application example of the pitch emphasizing device of FIG. 11.

FIG. 13 is a block diagram showing another application example of the pitch emphasizing device of FIG. 11.

14 is a block diagram showing still another application example of the pitch emphasizing device in FIG. 11. FIG.

15 is a block diagram showing another application example of the pitch emphasizing device in FIG. 11. FIG.

16 is a block diagram showing another application example of the pitch emphasizing device of FIG. 11. FIG.

17 is a block diagram showing a modified example of the pitch emphasizing device of FIG. 11. FIG.

[Explanation of symbols]

101 ... Input voice, 107, 112 ... Frequency spectrum, 109 ... Cepstrum, 111 ... Fundamental frequency, 11
4 ... Pitch-enhanced spectrum, 801 ... Speech power spectrum before pitch enhancement, 802 ... Speech power spectrum after pitch enhancement

Claims (8)

[Claims] 1. A means for inputting a voice signal, a means for obtaining a frequency spectrum of an input voice from the voice signal, a means for extracting a fundamental frequency of the voice signal, and the extracted fundamental of the frequency spectrum. Level enhancement means for enhancing the frequency spectrum component of the frequency; means for converting the frequency spectrum into a corresponding voice signal; and means for converting the voice signal obtained by the conversion into voice. Characteristic pitch enhancement device. 2. The pitch emphasizing device according to claim 1, wherein the level emphasizing means increases a spectrum level of a fundamental frequency band centered on a fundamental frequency position in the frequency spectrum. 3. The level emphasizing means performs the emphasizing by multiplying a frequency spectrum of the input voice by a window function of a fundamental frequency width having a center at a fundamental frequency position. The described pitch enhancement device. 4. The level enhancement means performs the enhancement by adding a window function of a fundamental frequency width centered at a fundamental frequency position to the frequency spectrum of the input voice. Pitch enhancement device. 5. The level emphasizing means replaces a component of the fundamental frequency width of the fundamental frequency position of the frequency spectrum of the input speech with a window function of the fundamental frequency width centered on the fundamental frequency position. The pitch enhancement device according to 1 or 2. 6. The level emphasizing means adjusts the level of the fundamental frequency component of the input voice to the level of the fundamental frequency component of the input voice before the degradation, with respect to the input voice degraded by the transmitting means. 3. The pitch enhancing device according to claim 1, wherein the pitch enhancing device is changed to. 7. A pitch emphasizing method in a sound processing apparatus, comprising: a means for inputting a voice, a means for analyzing and processing the voice, and a means for reproducing and outputting the voice, which is a basis of the input voice. A pitch emphasizing method characterized in that the frequency of a voice is extracted and the level of the fundamental frequency band centering on the fundamental frequency position in the frequency spectrum of the speech is changed to enhance the pitch of the inputted speech. 8. A means for converting voice into a voice signal, a means for obtaining a frequency spectrum of the voice signal obtained by the conversion, a means for extracting a fundamental frequency of the voice signal, and a portion of the frequency spectrum. A means for emphasizing the frequency spectrum component of the extracted fundamental frequency, a means for converting the frequency spectrum into a voice signal corresponding thereto, and a means for converting the voice signal obtained by the conversion into voice. A hearing compensation device characterized by being provided.