JP6630605B2 - Impulse response estimation device and program - Google Patents

Description

  The present invention relates to an impulse response estimation device and a program for estimating an impulse response.

  In the production of program audio, a plurality of audios recorded at different times and places are mixed to produce one program. However, for example, when an audio signal recorded in a place where there is little sound is mixed with an audio signal recorded in a place where the sound is rich, unnatural content that does not fit in the sense of hearing may occur. In order to avoid such a problem, by adding reverberation to an audio signal having almost no reverberation, it is possible to produce a natural program that is compatible with hearing.

  Also, when converting the number of audio signals of a relatively small number of channels, such as monaural or 2ch stereo, to a multi-channel sound format such as 5.1ch surround or 22.2ch surround, a horizontal or vertical shift with respect to the listening point is required. By assigning a signal to which a reverberation simulating an indirect sound due to reflection on a wall surface or the like is assigned to a channel installed on the back surface, it is possible to convert the number of channels with a sense of spatial expansion.

  In the reverberation addition in the mixing of the audio signal and the conversion of the number of channels, what kind of reverberation is actually added largely depends on the experience and intuition of the program producer. For example, even if an audio signal recorded in a space with a short reverberation is mixed with an audio signal to which an extremely long reverberation is added, the resulting sound will be unnatural.

  Normally, an acoustic signal recorded in a certain space (original sound field) is composed of a direct sound arriving directly at a listening point and an indirect sound (reverberant sound) arriving after being reflected on a wall or the like. This indirect sound is a signal obtained by convoluting the impulse response between the sound source and the listening point with the direct sound. Therefore, if the direct sound and the impulse response are known, a reverberation sound can be generated by convoluting the impulse response with the direct sound. Therefore, the impulse response is estimated from the sound signal containing the reverberation (original sound), and this impulse response is convolved with another sound signal (direct sound) containing no reverberation to obtain a reverberation sound close to the reverberation sound of the original sound field. Can be added. According to this method, even when a plurality of sounds recorded at different times and places are mixed, reverberation without unnaturalness can be added.

  In general, when both the indirect sound and the direct sound included in an original sound recorded in a space are known, an impulse response, which is a reverberation component of the space, can be estimated using an adaptive filter method. . In the adaptive filter method, the indirect sound signal included in the original sound and the indirect sound signal generated by convolving the impulse response with the direct sound are trained for a certain period for each sample so that the average value of the square error is minimized. Although the impulse response is estimated, it is known that effective estimation is possible only when a linear relationship holds between two signals (for example, see Non-Patent Document 1).

  However, in normal recording, direct sound and indirect sound are not separately recorded, and only a mixed sound of them is often recorded. In such a case, it is difficult to estimate the impulse response. Therefore, there is a demand for a technique that separates a direct sound and an indirect sound from a recorded mixed sound using an existing reverberation separation technique, and estimates an impulse response from the direct sound and the indirect sound. As the reverberation separation method, for example, a technology described in Non-Patent Document 2 and software “UNVEIL” by Zynaptiq are known.

John Usher, "Acoustic impulse response measurement using speech and music signals", AES 128th convention, 2010 K. Kinoshita et al, "Blind Upmix Of Stereo Music Signals Using Multi-Step Linear Prediction Based Reverberation Extraction", ICASSP, pp.49-52, 2010.

  However, when the reverberation separation method is realized by nonlinear processing or time-varying processing, the relationship between the output direct sound and the indirect sound changes momentarily depending on the mixed sound that is the input signal. , The estimated impulse response does not converge to a constant value. Therefore, an unnatural impulse response may be estimated by lengthening the learning time, and it is necessary to estimate an impulse response in a signal section as short as possible. On the other hand, when an acoustic signal having a temporal bias in frequency components such as a tone signal is used, it is necessary to set a longer learning time in order to estimate an impulse response including more frequency components. There was a problem that conflicting conditions were required.

  In view of such circumstances, an object of the present invention is to provide an impulse response estimation device and a program capable of accurately estimating an impulse response from only an acoustic signal that is a mixed sound recorded in an original sound field having reverberation. Is to do.

  In order to solve the above problems, the impulse response estimation device according to the present invention, a reverberation separation unit that separates an audio signal into a direct sound and an indirect sound, dividing the direct sound and the indirect sound into a plurality of frames, A splitting unit that generates a split direct sound and a split indirect sound; and an impulse response estimating unit that estimates an impulse response of an original sound field containing an audio signal from the split direct sound and the split indirect sound. .

  Furthermore, the impulse response estimation device according to the present invention further includes a frame averaging unit that generates an average direct sound and an average indirect sound by averaging the divided direct sound and the divided indirect sound, and the impulse response estimation unit includes an average The impulse response is estimated using a direct sound and an average indirect sound.

  Furthermore, the impulse response estimation device according to the present invention further includes a frame selection unit that selects a frame having a largest difference between frequency characteristics of the divided direct sound and the divided indirect sound, wherein the impulse response estimation unit performs the frame selection. The impulse response is estimated from the divided direct sound and the divided indirect sound of the frame selected by the section.

  Further, in the impulse response estimation device according to the present invention, the frame selecting unit selects a frame having the largest spectral distortion or spectral distortion weighted equally to all octaves.

  In order to solve the above problems, a program according to the present invention causes a computer to function as the impulse response estimation device.

  According to the present invention, an impulse response can be accurately estimated from only an acoustic signal recorded in an original sound field having reverberation. Further, even when a reverberation separation method based on non-linear processing or time-varying processing is used, an appropriate impulse response can be estimated. Further, even when the input signal is a sound signal having a temporally biased frequency component, the impulse response can be estimated in a short learning time.

FIG. 1 is a block diagram illustrating a configuration example of an impulse response estimation device according to a first embodiment of the present invention. It is a block diagram showing the example of composition of the impulse response estimating device concerning a 2nd embodiment of the present invention.

  Hereinafter, embodiments of the present invention will be described in detail with reference to the drawings.

(1st Embodiment)
An impulse response estimation device according to the first embodiment of the present invention will be described below. FIG. 1 shows a configuration example of an impulse response estimation device according to the first embodiment of the present invention. The impulse response estimating apparatus 1 shown in FIG. 1 includes a reverberation separating unit 11, a direct sound dividing unit 12, an indirect sound dividing unit 13, a direct sound frame averaging unit 14, an indirect sound frame averaging unit 15, an impulse A response estimating unit 16.

  The impulse response estimation device 1 inputs an acoustic signal x (k) recorded in a certain space. The acoustic signal x (k) is a mixed sound of the direct sound d (k) and the indirect sound r (k), and is expressed as in Expression (1). In this specification, the “acoustic signal” may include any sound such as a voice or a musical instrument.

  The reverberation separation unit 11 performs a reverberation separation process, separates the acoustic signal x (k) into a direct sound d (k) and an indirect sound r (k), outputs the direct sound to the direct sound division unit 12, and The sound is output to the indirect sound dividing unit 13. Here, any known reverberation processing can be used.

The direct sound dividing unit 12 divides the direct sound d (k) input from the reverberation separating unit 11 into frames having a predetermined length T, and divides the divided direct sound d _n (k) (n = 1, 2, .., N) are generated and output to the direct sound frame averaging unit 14. Here, k = 1, 2,..., T.

The indirect sound division unit 13 divides the indirect sound r (k) input from the reverberation separation unit 11 into frames of a predetermined length T, and divides the indirect sound r _n (k) (n = 1, 2, .., N) are generated and output to the indirect sound frame averaging unit 15. Here, k = 1, 2,..., T.

  The division of the direct sound d (k) and the indirect sound r (k) may be performed using T as the frame length without division as in Expression (2), or with overlap as in Expression (3). It may be divided.

  Here, the length of the frame length T can be arbitrarily determined by the user, but is preferably equal to or longer than the length of the impulse response to be extracted. If the direct sound d (k) and the indirect sound r (k) cannot be clearly divided by the predetermined frame length T, the signal ends of the direct sound d (k) and the indirect sound r (k) are added. Zero padding may be performed.

Direct sound frame averaging unit 14, the division direct sound generated by the direct sound division unit 12 d _{n (k),} to produce an average direct sound / d to direct sound d (k) is averaged (k). In this specification, “/” means a bar which is an average symbol. The average direct sound / d (k) is represented by Expression (4), where N is the number of frames (the number of divisions) of the direct sound.

Indirect sound frame averaging unit 15, the division indirect sound generated by the indirect sound division unit 13 r _{n (k),} and generates an indirect sound r a (k) averaged mean indirect sound / r (k). The average indirect sound / r (k) is represented by Expression (5), where N is the number of divided indirect sounds (the number of frames).

  Further, the indirect sound r (k) can be expressed by Expression (6) using the impulse response h (k). Here, * represents a convolution operation.

Similarly, division indirect sound r _{n (k)} can be expressed as in Equation (7) using the impulse response h _n of each divided frame _(k).

Here, e _{n (k)} is the influence of the reverberation of the previous frame. Incidentally, when the impulse response h _n of each frame _(k) is assumed to be unchanged, the relationship between the average direct sound / d (k) and the average indirect sound / r (k) is be expressed by Equation (8) it can. Also, / e (k) is the effect of the reverberation of the previous frame in each averaged frame.

  Here, / h (k) indicates an averaged impulse response, and can be expressed by equation (9).

  The impulse response estimation unit 16 averages the averaged direct sound / d (k) and the average indirect sound / r (k) using any known method such as an adaptive filter method or a least square method. An impulse response / h (k) is obtained, and this is output to the outside as an impulse response of an original sound field in which an acoustic signal is recorded. Here, assuming that / e (k) is uncorrelated with the average direct sound / d (k), it is possible to remove the effect of / e (k) by using the adaptive filter method or the least squares method. it can. As an adaptive algorithm in the adaptive filter method, since many methods such as an LMS algorithm, an N-LMS algorithm, and a projection algorithm have been proposed, an algorithm that can be separated while maintaining sound quality may be used.

  The impulse response estimating unit 16 estimates the impulse response for the averaged direct sound / d (k) and the average indirect sound / r (k), so that the reverberation separating unit 11 performs nonlinear processing or time-varying processing. Even when the sound is separated into a direct sound and an indirect sound in which the relationship between input and output changes, an impulse response / h (k) containing many frequency components can be obtained by averaging in all signal sections.

(Second embodiment)
Next, an impulse response estimation device according to a second embodiment of the present invention will be described below. FIG. 2 shows a configuration example of an impulse response estimation device according to the second embodiment of the present invention. The impulse response estimation device 2 shown in FIG. 2 includes a reverberation separation unit 11, a direct sound division unit 12, an indirect sound division unit 13, a frame selection unit 17, and an impulse response estimation unit 16. The reverberation separation unit 11, the direct sound division unit 12, and the indirect sound division unit 13 are the same as those in the first embodiment, and thus the description is omitted.

The frame selecting unit 17 receives the divided direct sound d _n (k) from the direct sound dividing unit 12 and the divided indirect sound r _n (k) from the indirect sound dividing unit 13. The wider the frequency domain of the direct sound and the indirect sound, the higher the estimation accuracy of the impulse response. Therefore, the frame selection unit 17 selects the largest frame difference of the frequency characteristics of the division direct sound d _{n (k)} and divide the indirect sound r _{n (k).} That is, the difference of the frequency characteristics is obtained for each frame. For example, when the difference of the frequency characteristics is the largest for n = 3 frames, the divided direct sound d ₃ (k) and the divided indirect sound r ₃ (k) are impulse. Output to the response estimating unit 16.

Hereinafter, a specific example of the frame selection process in the frame selection unit 17 will be described. Relationship split direct sound d _{n (k)} and divide the indirect sound r _{n (k)} is expressed by Equation (7) using the impulse response h _n of each frame that is divided as described above _(k).

Further, division indirect sound _r n (k) of the discrete Fourier transform; by (DFT discrete` Fourier transform), it can be derived relation of formula (10).

_{Here, R n (ω m),} D n (ω m) each divided indirect sound _r n (k), represents the frequency domain representation of the division direct sound _{d n (k), H n} (ω m) is transmitted Represents a function. Further, ω _m is an angular frequency, and m is an index in frequency discretization generated by the DFT processing. From equation (10), the transfer function represents the relative relationship between the direct sound and the indirect sound.

As an index indicating the difference between the frequency characteristics of the direct sound and the indirect sound, spectral distortion SD (Spectral Distortion) is shown in Expression (11). It is considered that the larger the spectral distortion SD is, the larger the integrated value of the difference between the amplitude characteristics of the direct sound and the indirect sound on the frequency axis is, and the more the reverberation component is included. Therefore, the frame selection unit 17, a spectral distortion SD calculated in advance each frame, most divided direct sound d _{n (k)} and divided indirect sound of a large frame of spectral distortion SD r _{n (k)} an impulse response estimator 16 May be output. Here, M is the DFT score.

Further, as another index indicating the difference between the frequency characteristics of the direct sound and the indirect sound, a spectrum distortion β equally weighted for all octaves is shown in Expression (12). The larger the evaluation function β, the greater the difference between the amplitude characteristics of the direct sound and the indirect sound on the frequency axis when the frame is evaluated with an averaging error equally weighted over all octaves according to the human auditory characteristics. It is considered that the value is large and contains many reverberation components. Therefore, the evaluation function β is calculated in advance for each frame, and the divided direct sound d _n (k) and the divided indirect sound r _n (k) of the frame having the largest evaluation function β are output to the impulse response estimation unit 16. Is also good.

  Note that a computer can be suitably used to function as the impulse response estimation devices 1 and 2 described above. Such a computer is a program that describes processing contents for realizing the functions of the impulse response estimation devices 1 and 2. Is stored in the storage unit of the computer, and the program is read out and executed by the CPU of the computer. This program can be recorded on a computer-readable recording medium.

  As described above, the impulse response estimation devices 1 and 2 according to the present invention, or the program thereof, estimates the impulse response using the direct sound and the indirect sound divided into a plurality of frames. Therefore, even when the reverberation separation unit 11 separates the direct sound and the indirect sound in which the relationship between the input and the output changes by the non-linear processing or the time-varying processing, an appropriate impulse response can be obtained. Further, even when an acoustic signal having a temporal bias in frequency components such as a tone signal is used, it is possible to estimate an impulse response in consideration of all frequencies included in the acoustic signal in a short learning time.

  Although the above embodiments have been described as representative examples, it will be apparent to those skilled in the art that many changes and substitutions can be made within the spirit and scope of the present invention. Therefore, the present invention should not be construed as being limited by the above-described embodiments, and various modifications and changes can be made without departing from the scope of the claims. For example, a plurality of configuration blocks described in the configuration diagram of the embodiment can be combined into one, or one configuration block can be divided.

  INDUSTRIAL APPLICABILITY The present invention can be used, for example, for audio editing work in program production.

1, 2 impulse response estimation device 11 reverberation separation unit 12 direct sound division unit 13 indirect sound division unit 14 direct sound frame averaging unit 15 indirect sound frame averaging unit 16 impulse response estimation unit 17 frame selection unit

Claims (5)

A reverberation separation unit that separates an acoustic signal into a direct sound and an indirect sound,
A dividing unit that divides the direct sound and the indirect sound into a plurality of frames to generate a divided direct sound and a divided indirect sound;
From the divided direct sound and the divided indirect sound, an impulse response estimation unit that estimates an impulse response of an original sound field that has recorded an acoustic signal,
An impulse response estimation device comprising: Further comprising a frame averaging unit that generates an average direct sound and an average indirect sound by averaging the divided direct sound and the divided indirect sound,
The impulse response estimation device according to claim 1, wherein the impulse response estimation unit estimates the impulse response using an average direct sound and an average indirect sound. The divided direct sound further includes a frame selection unit that selects a frame having a largest difference in frequency characteristics between the divided indirect sound,
The impulse response estimation device according to claim 1, wherein the impulse response estimation unit estimates the impulse response from a divided direct sound and a divided indirect sound of the frame selected by the frame selection unit.   The impulse response estimation device according to claim 3, wherein the frame selection unit selects a frame having the largest spectral distortion or the spectral distortion weighted equally to all octaves.   A program for causing a computer to function as the impulse response estimation device according to any one of claims 1 to 4.

Images