JP2008022069A - Voice recording apparatus and voice recording method - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To provide a voice recording apparatus capable of recording voice of each talker without being affected by reverberation sounds. <P>SOLUTION: The voice recording apparatus 1 includes: a reference signal generating section 5 for generating a reference signal received by a speaker 15 located at a sound source and acting like a sound generating section; an impulse response storage section 6 storing each of voice signals resulting from recording the sound generated from the speaker 15 by a plurality of voice pickup sections located differently from each other and temporally synchronized; and a transfer function calculation section 7 for obtaining a transfer function from the position of the sound source to microphones 11 to 14 acting like the voice pickup sections on the basis of the voice signals and the reference signal stored in the impulse response storage section 6. Further, the voice recording apparatus 1 includes: an inverse filter coefficient calculation section 8 for calculating an inverse filter coefficient for canceling the characteristic of the obtained transfer function; and an inverse filter processing section 9 for applying filter processing to the voice signals picked up by the plurality of voice pickup sections on the basis of the inverse filter coefficient. <P>COPYRIGHT: (C)2008,JPO&INPIT

Description

  The present invention relates to a sound recording apparatus and a sound recording method for recording a target sound.

Conventionally, as a method for individually recording a speaker's voice, a method using a close-talking microphone, a headset microphone, or the like, or a method for processing an audio signal based on the characteristics of the speaker's voice has been proposed. As another conventional technique, in a remote conference or the like, a sound signal received by a plurality of microphones is filtered and output to reduce noise and distortion, and a sound emitted from a target sound source can be reduced. An apparatus that collects sound with high quality has been proposed (Patent Document 1). The apparatus described in Patent Document 1 performs speaker position estimation based on a signal received by a microphone. Next, a delay is set in the delay device so that the delay sum array is focused on the estimated speaker position based on the speaker position estimation result. A signal-to-noise ratio (S / N ratio) is estimated for an audio signal recorded by each microphone. The estimated S / N ratio is used to determine the filter coefficient. The optimal filter calculation unit calculates an optimal filter that optimizes the S / N ratio of the output of the array and the distortion of the target sound component, and sets the filter.
JP 2001-309383 A

  However, in the technique described in Patent Document 1, since the reverberant sound reflected from the wall of the room or the like enters the microphone, the voice of each speaker cannot be collected without being affected by the reverberant sound. There was a problem.

  Accordingly, the present invention has been made in view of the above problems, and an object thereof is to provide an audio recording apparatus and an audio recording method capable of collecting the voice of each speaker without being affected by reverberant sound. And

  In order to solve the above problems, an audio recording apparatus according to the present invention includes a reference signal generating unit that generates a reference signal to be input to an audio generating unit placed on a sound source, and audio generated by the audio generating unit at different positions. A sound storage unit that stores sound signals picked up by a plurality of sound pickup units that are placed in synchronization with each other in time, and the sound source based on the sound signal stored in the storage unit and the reference signal Based on the inverse filter coefficient, a transfer function calculation unit that calculates a transfer function from the position of the sound pickup unit, an inverse filter coefficient calculation unit that calculates an inverse filter coefficient that cancels the characteristics of the obtained transfer function, And a filter processing unit that performs filter processing on the audio signals collected by the plurality of sound collecting units.

  According to the present invention, a reference signal to be input to a sound generator placed on a sound source is generated, and a plurality of sound pickups in which the sound generated by the sound generator is placed at different positions and synchronized in time Each of the sound signals collected by the storage unit is stored, and based on the sound signal and the reference signal stored in the storage unit, a transfer function from the position of the sound source to the sound collection unit is obtained, and the obtained transfer function By collecting the inverse filter coefficient that cancels the characteristics and filtering the speech signal, it is possible to create speech without the reverberation, so that the speaker's speech can be collected individually without being affected by the reverberation. Can do.

  The filter processing unit picks up a target sound by making the phases of the sounds arriving at the plurality of sound collecting units in phase and adding them. It is preferable that the reference signal is an impulse input, and the sound signals collected by the plurality of sound collecting units are impulse responses. The audio recording apparatus of the present invention further includes an audio recording unit that records the audio signal filtered by the filter processing unit as an audio signal generated by the sound source.

  The filter processing unit estimates the position of the sound source based on sound source information indicating the position of the sound source and performs the filtering process. As a result, the sound source position can be estimated and filtered using the sound source information other than the image, for example, the position of the chair or the image using a tag using radio waves, ultrasonic waves, or infrared rays. If the room has little reverberation, voice source estimation is possible. The sound source information is, for example, an image signal captured by an imaging unit. Thereby, sound source estimation candidates can be narrowed down using the image signal, and the accuracy of the filter processing can be improved.

  According to the audio recording method of the present invention, the step of generating a reference signal to be input to a sound generator placed on a sound source and the sound generated by the sound generator are placed at different positions and synchronized in time. From the step of storing each of the sound signals picked up by the plurality of sound collecting units in the storage unit, and from the position of the sound source to the sound collecting unit based on the sound signal stored in the storage unit and the reference signal Obtaining a transfer function, calculating an inverse filter coefficient that cancels the characteristics of the transfer function, and filtering audio signals picked up by the plurality of sound pickup units based on the inverse filter coefficient Steps. According to the present invention, since the reverberant sound is removed using the reference signal, the voice of the speaker can be collected without being affected by the reverberant sound.

  The reference signal is an impulse input, and the voice signals collected by the plurality of voice collecting units are impulse responses. The filtering step estimates the position of the sound source based on sound source information indicating the position of the sound source, and performs the filtering process. As a result, the sound source position can be estimated and filtered using the sound source information other than the image, for example, the position of the chair or the image using a tag using radio waves, ultrasonic waves, or infrared rays. In addition, it is possible to estimate the sound source of a voice in a room with little reverberation. The sound source information is an image signal captured by an imaging unit. Thereby, sound source estimation candidates can be narrowed down using the image signal, and the accuracy of the filter processing can be improved.

In addition, BSS (Blind) which applied ICA (independent component analysis) of known technology to speech
The accuracy can be improved by optimizing the filter parameters in the filter processing using the technique of Source Separation. BSS has been proposed by the following literature. H.
Saruwatari, T. Kawamura, T.
Nishikawa, K. Shikano,
`` Fast-Convergence Algorithm for Blind Source Separation Based
on Array Signal Processing, '' IEICE Trans.
Fundamentals, Vol.E86-A, No.3, pp.286--291 March 2003.

  Since the calculation is complicated and the accuracy is reduced in a multi-channel, the calculation can be performed in a realistic time by using the filter parameter obtained in the present invention as the initial value of the BSS. A known BSS is divided into bands and then filtered for each band to separate the sound. In the present invention, the process is performed in units of sound wave sampling as in the microphone array technology. BSS is based on the fact that multiple sound sources are independent (even if there are two or more utterances, there is no correlation between them), but this can also be used for fine-tuning / optimizing the parameters of the present invention. . It is possible to set the present invention as an initial value for optimization by BSS, and there is no precedent for this. By using a known BSS with the parameters of the present invention as initial values, the accuracy can be further improved and fine adjustment can be performed when the body (sound source) moves.

  ADVANTAGE OF THE INVENTION According to this invention, the audio | voice recording apparatus and audio | voice recording method which can collect each speaker's audio | voice without being influenced by a reverberation sound can be provided.

  Hereinafter, the best mode for carrying out the present invention will be described. FIG. 1 is a diagram for explaining an audio recording apparatus that measures a transfer function up to a sound receiving point in advance and calculates an inverse filter. As shown in FIG. 1, the audio recording apparatus 1 includes first to fourth audio input terminals 21 to 24, an audio output terminal 3, a reference signal output terminal 4, a reference signal generation unit 5, an impulse response storage unit 6, a transfer function. A calculation unit 7, an inverse filter coefficient calculation unit 8, an inverse filter processing unit 9, and an audio recording unit 10 are provided. Reference numerals 11 to 14 denote first to fourth microphones (microphone array, a plurality of sound collecting units), 15 denotes a speaker, 16 to 18 denote a speaker, and 20 denotes an imaging unit such as a video camera. It is assumed that the speakers 16 to 18 exist at the positions of the sound sources A to C, respectively.

  The sound recording device 1 generates a reference signal from the speaker position, measures a transfer function up to the sound receiving point in advance, calculates an inverse filter of the transfer function, and convolves with the sound receiving signal to reverberate. Is to record the sound.

  The reference signal generator 5 generates a reference signal to be input to the speaker 15 which is a sound generator placed on the sound source. The reference signal generator 5 passes the generated reference signal through the reference signal output terminal 4 and outputs it from the speaker 15 placed at the same position as the speakers 16 to 18. The first to fourth microphones 11 to 14 are installed at arbitrary positions different from each other. The first to fourth microphones are arranged two-dimensionally, for example, on the ceiling or floor surface of the conference room. The generated reference signal is received by the first microphone 11 to the fourth microphone 14. The impulse response storage unit 6 stores voice signals (for example, impulse responses) collected by a plurality of microphones 11 to 14 which are placed at different positions and are synchronized in time. The impulse response storage unit 6 is composed of, for example, a hard disk or a semiconductor memory, and stores the impulse response of each received channel.

  FIG. 2 is a diagram for explaining the contents of the impulse response storage unit 6, where FIG. 2A is a diagram for explaining the impulse input and impulse response in the sound source A, and FIG. 2B is the impulse input in the sound source B. FIG. 6C is a diagram for explaining impulse input and impulse response in the sound source C. FIG. In each figure, (a) is a waveform of an impulse input as a reference signal input to the speaker 15, (b) is a waveform of an impulse response received by the first microphone 11, and (c) is received by the second microphone 12. (D) shows the waveform of the impulse response received by the third microphone 13, and (e) shows the waveform of the impulse response received by the fourth microphone 14, respectively.

  The transfer function calculation unit 7 obtains a transfer function from the position of the sound source to the microphones 11 to 14 based on the audio signal and the reference signal stored in the impulse response storage unit 6. Specifically, the transfer function calculation unit 7 transfers the transfer function from the position of the speakers 16 to 18 to the first microphone 11 based on the received signal of each channel and the reference signal, and the speakers 16 to 18. A transfer function from the position to the second microphone 12, a transfer function from the positions of the speakers 16 to 18 to the third microphone 13, and a transfer function from the positions of the speakers 16 to 18 to the fourth microphone 14 are calculated. The inverse filter coefficient calculation unit 8 calculates an inverse filter coefficient that cancels the measured transfer function characteristics of each channel.

  The inverse filter processing unit 9 filters audio signals collected by the plurality of microphones 11 to 14 based on the inverse filter coefficient. The inverse filter processing unit 9 filters the audio signal received by the plurality of microphones 11 to 14 using the inverse filter coefficient and outputs the filtered audio signal from the audio output terminal 3. Output. The audio recording unit 10 records the audio signal filtered by the inverse filter processing unit 9 as an audio signal generated by a sound source. For example, it is constituted by a hard disk device or a semiconductor memory, and the audio signal output from the audio output terminal 3 is stored for each speaker 16-18. Thereby, the sound emitted from the target sound source can be collected with high quality.

The processing of the reference signal generator 5, the transfer function calculator 7, the inverse filter coefficient calculator 8, and the inverse filter processor 9 is realized by executing a predetermined program by a CPU (Central Processing Unit) and its peripheral circuits. ASIC (Application Specific)
(Integrated Circuit). Further, the inverse filter processing unit 9 may estimate the position of the sound source based on the image signal captured by the imaging unit 20 and perform the filter processing. Thereby, sound source estimation candidates can be narrowed down using the image signal, and the accuracy of the filter processing can be improved.

  FIG. 3 is a flowchart of the inverse filter coefficient calculation process according to the embodiment of the present invention. Next, the inverse filter coefficient calculation process will be described. First, in the inverse filter coefficient calculation process, the speaker 15 is set at the position A where each speaker 16-18 is scheduled to speak (step S11). Next, the reference signal generator 5 passes the generated reference signal (impulse input) through the reference signal output terminal 4 and outputs it from the speaker 15 placed at the same position as the speakers 16 to 18 (step 12). The reference signal generated from the speaker 15 is received by the first microphone 11 to the fourth microphone 14 (step S13). The impulse response storage unit 6 stores the received impulse response of each channel (step S14).

  Next, similarly, the same procedure is performed for the positions B and C where the speakers 16 to 18 are to speak, and the impulse response storage unit 6 stores the impulse response of each channel. In the case of a meeting or a presentation, such processing is possible because the position of the sound source is roughly known from the position of the chair or the position of the presenter.

  When the transfer function calculation unit 7 determines that impulse responses for all utterance scheduled positions have been collected ("Y" in step S15), the speaker is based on the received signal of each channel and the reference signal. The transfer function from the position of 16-18 to the first microphone 11, the transfer function from the position of the speakers 16-18 to the second microphone 12, and the transfer function from the position of the speakers 16-18 to the third microphone 13 Then, a transfer function from the positions of the speakers 16 to 18 to the fourth microphone 14 is calculated (step S16). The inverse filter coefficient calculation unit 8 calculates an inverse filter coefficient that cancels the measured transfer function characteristics of each channel (step S17).

  Next, a process when recording the voice of each speaker individually will be described. FIG. 4 is a processing flowchart when recording the voice of each speaker individually. For example, it is assumed that the speaker 16 speaks at the sound source position A (step S21). Audio signals are input to the first to fourth microphones 11 to 14, and the input audio signals are input to the inverse filter processing unit 9 via the first to fourth audio input terminals 21 to 24 (step S22).

  The inverse filter processing unit 9 uses a technique called a delay sum array that increases the sensitivity to the focal position by adding in-phase the sounds arriving at the first to fourth microphones 11 to 14, and directs the focus to the target sound source position. As a result, noise other than the target sound source position is suppressed, and a sound signal of the sound source A with an improved S / N ratio is generated (step S23). As a result, an audio signal in which the sound from the target sound source is emphasized can be generated. Next, the inverse filter processing unit 9 filters and outputs the audio signals received by the plurality of microphones 11 to 14 using the inverse filter coefficient (step S24). As a result, the audio signal from which reverberation has been removed is output from the audio output terminal 3.

  The voice recording unit 10 stores the voice signal output from the voice output terminal 3 for each of the speakers 16 to 18, and individually records the voice of the speaker (step S25). The sound recording device ends the process when an instruction to end sound recording is input. Thereby, the voice of the speaker emitted from the target sound source can be individually recorded with high quality.

  According to the audio recording apparatus according to the above-described embodiment, it is possible to create a voice from which reverberation is removed using a reference signal, and thus it is possible to individually collect the voice of a speaker without being affected by the reverberation. .

  Although the preferred embodiments of the present invention have been described in detail above, the present invention is not limited to the specific embodiments, and various modifications, within the scope of the gist of the present invention described in the claims, It can be changed. In the above description, the case where there are three sound source positions has been described as an example. However, the present invention is not limited to this, and the present invention can be applied even when more sound source positions exist. In addition, although an example using four microphones has been described above, in practice, it is better to use more microphones for improving accuracy. In the above description, an example in which an impulse input is used as an example of the reference signal has been described. However, the present invention is not limited to this, and any signal may be used as long as it is a reference signal. Further, the present invention can be applied not only to a meeting scene but also to various scenes. Moreover, although the example using the image signal imaged by the imaging unit as the sound source information indicating the position of the sound source has been described above, the present invention is not limited to the sound source information using the image signal. That is, the inverse filter processing unit can also perform filter processing by estimating the position of the sound source based on sound source information indicating the position of the sound source.

It is a figure explaining the technique of the audio | voice recording apparatus which measures the transfer function to a sound receiving point in advance, and calculates an inverse filter. It is a figure for demonstrating the content of an impulse response memory | storage part. It is a flowchart of the inverse filter coefficient calculation process which concerns on embodiment of this invention. It is a process flowchart at the time of recording the audio | voice for every speaker separately.

Explanation of symbols

DESCRIPTION OF SYMBOLS 1 Audio | voice recording device 21-24 Audio | voice input terminal 3 Audio | voice output terminal 4 Reference signal output terminal 5 Reference signal generation part 6 Impulse response recording part 7 Transfer function calculation part 8 Inverse filter coefficient calculation part 9 Inverse filter process part

Claims (10)

A reference signal generator for generating a reference signal to be input to a sound generator placed on a sound source;
A storage unit for storing each of the sound signals picked up by a plurality of sound pickup units placed in different positions and synchronized in time with the sound generated by the sound generation unit;
A transfer function calculation unit for obtaining a transfer function from the position of the sound source to the sound pickup unit based on the sound signal stored in the storage unit and the reference signal;
An inverse filter coefficient calculation unit for calculating an inverse filter coefficient that cancels the characteristics of the obtained transfer function;
An audio recording apparatus comprising: a filter processing unit that performs a filter process on audio signals collected by the plurality of audio pickup units based on the inverse filter coefficients. The sound recording apparatus according to claim 1, wherein the filter processing unit picks up a target sound by in-phaseizing and adding sounds arriving at the plurality of sound collecting units. The reference signal is an impulse input;
The sound recording apparatus according to claim 1, wherein the sound signals collected by the plurality of sound collecting units are impulse responses. The audio recording apparatus according to claim 1, further comprising an audio recording unit that records the audio signal filtered by the filter processing unit as an audio signal generated by the sound source. 5. The audio recording apparatus according to claim 1, wherein the filter processing unit estimates the position of the sound source based on sound source information indicating the position of the sound source and performs the filter processing. 6. . 6. The audio recording apparatus according to claim 5, wherein the sound source information is an image signal captured by an imaging unit. Generating a reference signal to be input to a sound generator placed on a sound source;
Storing each of the sound signals collected by the plurality of sound collecting units that are placed at different positions and synchronized in time with the sound generated by the sound generating unit in the storage unit;
Obtaining a transfer function from the position of the sound source to the sound pickup unit based on the sound signal stored in the storage unit and the reference signal;
Calculating an inverse filter coefficient that cancels the characteristics of the transfer function;
Filtering audio signals picked up by the plurality of sound pickup units based on the inverse filter coefficients. The reference signal is an impulse input;
The voice recording method according to claim 7, wherein the voice signals picked up by the plurality of voice pickup units are impulse responses. 9. The audio recording method according to claim 7, wherein the filtering process performs the filtering process by estimating the position of the sound source based on sound source information indicating the position of the sound source. The audio recording method according to claim 9, wherein the sound source information is an image signal captured by an imaging unit.

Images