JPH0556007A - Mixed voice signal transmission system - Google Patents

Abstract

PURPOSE:To separate a speech signal and a background sound from a mixed voice signal with high sound quality and to mix the both in a preferred rate by the listener while suppressing a capacity required for additional information. CONSTITUTION:A sender side mixes a speech signal and a background sound, generates additional information for separating signals (by additional information extraction section 2) from the speech signal and the background sound and multiplexes the mixed voice signal and the additional information signal with each other and sends the multiplexed signal. A receiver side separates the mixed voice signal and the additional information from the received signal, separates the speech signal and the background sound based on the additional information (speech/background separator 5), and mixes the separated speech signal and background sound in a rate preferred by the listener.

Description

Detailed Description of the Invention

[0001]

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention relates to a mixed voice signal transmission system suitable for listening to voices of hearing-impaired persons and the elderly.

SUMMARY OF THE INVENTION The present invention is directed to a broadcasting system or the like which transmits a voice signal in which speech (human voice) and background sound such as music and sound effects are mixed, and time information regarding the speech and the background sound. Additional information such as spectrum information is multiplexed and transmitted with a small number of quantization bits, and on the receiving side,
By applying processing such as filtering to the mixed sound based on the additional information, the speech and the background sound are separated from the mixed sound so that the receiver can mix and listen to each other again according to his / her desired mixing ratio. It is a thing.

[0003]

2. Description of the Related Art As conventional techniques for removing noise mixed in voice or extracting only a specific voice, there are a two-input method that uses information of noise sources and a one-input method that can use only mixed voice. Method is known.

A noise removal method using an adaptive filter is known as a two-input method (for example, B. Widrow et.
al .: “Adaptive Noise Cancelling: Principles and App
lications ”, Proc.IEEE, Vol.63, No.12, pp.1692-1716 (1
975)).

As a one-input method, a section in which a voice signal does not exist is detected by some method, noise is estimated from the section, and a Wiener filter (for example, N. Wiener:
“Extrapolation, Interpolation and Smoothing of Sta
tionary Time Series, with Engineering Applications
, John Wiley & Sons, (1949)) and the spectral subtraction method (SFBoll: “Suppression of Acoustic
Noise in Speech Using Spectral Subtraction ”, IEEE
Trans., Vol.ASSP-27, No.2, April (1979)) removes noise.

[0006]

By the way, it is often difficult for a hearing impaired person or an elderly person to hear a broadcast program sound, especially a human voice. In particular, if music or sound effects are mixed with speech for program effects, it becomes extremely difficult to hear.

In order to solve this, it is possible to apply the above-mentioned conventional one-input method to separate the speech from only the received speech, but it is very difficult because the broadcasting system handles various speeches. is there. It is also possible to send information on speech and background sound by a two-input method, but the transmission capacity is limited in systems such as broadcasting.
For example, if an attempt is made to transmit a speech voice waveform itself in addition to a normal voice signal, naturally one more channel of transmission capacity is required, which is not preferable for efficient transmission.

Therefore, although there is a value-added voice service such as stereo broadcast, bilingual broadcast, voice multiplex broadcast, etc. by increasing the number of voice channels, the receiver service in the current broadcast system, especially voice broadcast, is available. However, the present situation is that no consideration is given to improve the listening of the speech, which is the essence of information transmission, to the recipients such as the hearing impaired and the elderly who have difficulty hearing the normal program sound.

The present invention has been made in view of the above circumstances, and an object thereof is to separate a speech and a background sound with high sound quality while suppressing the volume of additional information to be small.
It is an object of the present invention to provide a mixed audio signal transmission system that enables a listener to mix both at a ratio of his or her preference.

[0010]

In order to achieve the above object, a mixed voice signal transmission system according to the present invention, on the transmitting side, mixes speech and background sound and separates the signal from the speech and background sound. While generating the additional information for use and multiplexing and transmitting the mixed voice signal and the additional information signal, the receiving side separates the mixed voice and the additional information from the received signal, and gives a speech based on the additional information. The feature is that the background sound is separated, and the separated speech and the background sound are mixed at a ratio of the listener's preference.

[0011]

On the transmitting side, section information indicating whether each speech frame is a silent section, a periodic section, or an aperiodic section, pitch data when the speech is a periodic section, and speech is a speech section. Additional information such as a vector number representing a power spectrum envelope of a speech at a certain time and a vector number representing a power spectrum envelope of a background sound when the speech is a voiced section is generated and multiplexed with the mixed voice signal.

At the receiving side, first, the mixed voice and the additional information are separated, and the mixed voice is separated into the speech and the background sound based on the additional information. The separated speech and background sound are mixed sounds at a ratio of the listener's preference.

[0013]

1 is a block diagram showing the construction of an embodiment to which the system of the present invention is applied.

The transmission system of the device shown in the figure comprises a sound mixer 1.
The additional information extractor 2 and the transmitter 3 are provided to mix the speech and the background sound, generate additional information from the speech and the background sound, and multiplex and transmit the mixed audio signal and the additional information signal. To do. One receiving system includes a receiver 4, a speech / background sound separator 5, and a voice mixer 6, and separates mixed voice and additional information from a received signal,
The speech and background sound are separated based on the additional information, and the separated speech and background sound are mixed at a desired ratio.

FIG. 2 shows the configuration of the additional information extractor 2.

The additional information extractor 2 includes a frame cutout unit 201 for cutting out a frame from a speech signal,
A silence / speech determination unit 202 that determines silence / speech from a frame of a speech signal, a pitch extraction unit 203 that extracts a pitch of a speech segment, and an LPC analysis unit that also performs linear prediction analysis of a frame from a speech segment. 204, a vector quantization unit 205 that vector-quantizes the spectrum envelope coefficient of the LPC analysis unit 204, a frame cutout unit 206 that cuts out a frame from a background sound signal, and an LPC analysis that performs linear prediction analysis of the cutout frame. Part 207 and L
A vector quantization unit 208 for vector-quantizing the spectrum envelope coefficient of the PC analysis unit 207, section information of the silence / sound determination unit 202, pitch data of the pitch extraction unit 203, and each of the vector quantization units 205 and 208. An encoder 209 that generates additional information encoded from the vector number and outputs the encoded additional information to the transmitter 3.

Next, the operation of the additional information extractor 2 will be described.

In the frame cutout unit 201, a speech is cut out into a frame of a fixed length by a Hamming window having a frame length of 20 ms and a frame period of 10 ms. Each clipped frame is output to the silence / sound determination unit 202.

The silence / speech judging section 202 judges whether each of the supplied frames is a silent section or a sound section based on its power. The silent section referred to here is a silent section of speech, that is, a section in which no words are spoken, and the voiced section is a section other than the silent section. This section information is output to the encoder 209.

For the voiced section, the pitch extraction unit 203
At, the pitch is extracted. When the pitch is extracted, it is set as a periodic section, and when the pitch is not extracted, it is set as an aperiodic section. The pitch data of the periodic section is output to the encoder 209.

Further, the LPC analysis section 204 carries out a linear prediction analysis for the frame determined to be in the voiced section. Here, the coefficient representing the envelope of the power spectrum is obtained using the fifth-order LPC cepstrum coefficient. The coefficient of the spectrum envelope thus obtained is output to the vector quantization unit 205.

In the vector quantizer 205, the coefficient of the spectrum envelope is vector quantized and converted into a vector number. The codebook for vector quantization is
Make a number of speeches in advance. Also, the codebook size is 256 bits, and LBG (Linde, Bu
zo, Gray) algorithm is used for clustering. At this time, when the distance between the true spectrum and the closest spectrum in the codebook is larger than a certain threshold value, quantization is not performed. The obtained vector number is output to the encoder 209.

On the other hand, regarding the background sound, similarly to the speech, the frame cutout unit 206 performs frame cutout, and the linear prediction analysis of each cutout frame is performed by the LPC.
This is performed by the analysis unit 207. The coefficient of the spectrum envelope is converted into a vector number by the vector quantization unit 208.
However, unlike the codebook in the vector quantization unit 205, the codebook for vector quantization is created in advance from many background sounds such as music and sound effects. The obtained vector number is output to the encoder 209.

The encoder 209 has section information indicating whether each speech frame is a silent section, a periodic section, or an aperiodic section, pitch data when the speech is a periodic section, and speech is a sound section. Is supplied, and a vector number representing the power spectrum envelope of the background sound when the speech is a voiced section is supplied, and these information are encoded additional information. ..

The additional information thus obtained is transmitted to the transmitter 3 after being multiplexed with the mixed voice.

In FIG. 3, the speech / background sound separator 5 is shown.
The configuration of is shown.

The speech / background sound separator 5 decodes the section information, the pitch data, the vector number of the speech and the vector number of the background sound from the input additional information, and the decoder 501 from the decoded section information. A frame cutout unit 502 for cutting out a frame of mixed voice
A section determination unit 503 that determines a section of a frame, a removal unit 504 that removes a mixed sound in a silent section, an FFT 505 that performs a fast Fourier transform of a mixed sound in a sound section, and F
A spectrum subtraction unit 506 that subtracts the power spectrum of the background sound from the FT output, a Wiener filter 507 that filters the output spectrum of the spectrum subtraction unit 506, an IFFT 508 that performs a fast inverse Fourier transform of the Wiener filter output, and an IF
Comb filter 509 for filtering FT output
And a joining portion 51 for joining the frames
0 and an adaptive filter unit 511.

Next, the operation of this speech / background sound separator 5 will be described.

The received signal is separated by the receiver 4 into mixed voice and additional information, and then supplied to the speech / background sound separator 5.

The decoder 501 decodes the section information, pitch data, vector number of speech and vector number of background sound from the additional information. The section information is output to the frame cutout unit 502 and the section determination unit 503, the pitch data is output to the comb filter 509, the vector number of the speech is output to the Wiener filter 507, and the vector number of the background sound is output to the spectrum subtraction unit 506.

In the frame cutout unit 502, the input mixed voice is cut out in frame units. The cutout timing is synchronized with the transmitting side while referring to the section information.

In the section determination unit 503, the mixed voice cut into frames is separated into a silent section of speech, a sound section, a periodic section, and an aperiodic section by referring to the section information. The separated silent section is output to the removing unit 504, and the removing unit 504 removes the mixed voice in the silent section.
The mixed voice of the removed silent section is connected by the joining unit 510.
Is output to.

The voiced section is transformed into the frequency domain by the fast Fourier transform of the FFT section 505, and the power spectrum of the mixed voice is obtained.

The spectrum subtraction unit 506 subtracts the power spectrum corresponding to the vector number of the background sound from the power spectrum of the mixed voice to obtain the power spectrum of the speech. At this time, the place where the subtraction result becomes negative is replaced with 0.

Next, in the Wiener filter section 507,
Based on the input vector number of the speech, the Wiener filter processing having the characteristic shown by the following expression is applied to the output of the spectral subtraction unit 506.

H (ω) = (power spectrum corresponding to vector number of speech) / (power spectrum of mixed speech) The output spectrum of the Wiener filter unit 507 is IF
The time waveform is restored by the fast inverse Fourier transform of the FT unit 508. The periodic section of this time waveform is output to the comb filter section 509, and the non-periodic section is output to the joining section 510.

The comb filter unit 509 extracts speech consisting of only the pitch frequency and frequency components that are integral multiples of the pitch frequency based on the input pitch data from the output of the IFFT unit 508.

The signal waveforms of the silent section, the periodic section, and the aperiodic section restored as described above are added by the joining section 510 to form separated speech.

The adaptive filter unit 511 applies an adaptive filter to the separated speech. Subtractor 5 for filter output
The background sound is separated by subtracting 12 from the mixed voice.
The separated background sound is fed back to the adaptive filter unit 511 and corrected to an optimum coefficient so that the output of the adaptive filter unit 511 is waveform-equalized.

The speech and background sound separated from the mixed voice in this way are output to the mixer 6, and are mixed by the mixer 6 at a ratio of the listener's preference to obtain an optimum voice.

As described above, according to the present embodiment, the signal separation on the receiving side, which has been difficult with the conventional speech enhancement and noise removal techniques, is achieved by transmitting the additional information suitable for the signal separation from the transmitting side. It will be possible.

Further, in the prior art, it was difficult to separate the background sound from the mixed voice, but according to the present embodiment,
Both speech and background sound can be extracted and separated, and the separated speech and background sound can be mixed in a ratio that is most audible to the listener. Therefore, when applied to a broadcasting system, it is sufficiently effective for listening to programs with various background sounds such as news programs and dramas.

[0043]

As described above, according to the present invention, it is possible to separate the speech and the background sound with high sound quality from the mixed voice while suppressing the volume of the additional information to be small, and the listener can use both at a ratio of his or her preference. Can be mixed.

[Brief description of drawings]

FIG. 1 is a block diagram showing an overall configuration of a mixed audio signal transmission system according to the present invention.

FIG. 2 is a block diagram showing the configuration of an embodiment of an additional information extractor.

FIG. 3 is a block diagram showing the configuration of an embodiment of a speech / background sound separator.

[Explanation of symbols]

 2 Additional information extractor 3 Transmitter 4 Receiver 5 Speech / background sound separator 6 Mixer

Claims (1)

[Claims] 1. A transmitting side mixes a speech and a background sound, generates additional information for signal separation from the speech and the background sound, and multiplexes and transmits the mixed audio signal and the additional information signal. On the other hand, on the receiving side, the mixed voice and the additional information are separated from the received signal, and the speech and the background sound are separated based on the additional information, and the separated speech and the background sound are mixed at a ratio of the listener's preference. A mixed audio signal transmission method characterized by the above.