JP2019024160A - Extracted sound correction device, extracted sound correction method, program - Google Patents

Abstract

To provide an extracted sound correction technique capable of correcting a sound generated in a large meeting place and extracted, so as to suppress discomfort of audibility.SOLUTION: A extracted sound correction device includes a power calculation part 110 for calculating a sound collection venue sound power Yand an extracted sound power Efrom a collection venue sound y(k) and an extracted sound e(k), where the collection venue sound y(k) reproduced in the meeting place and the sound generated in the meeting place and collected are sound of sample number k, the extracted sound e(k) extracted from the sounds generated in the meeting place from the collection venue sound y(k) is the sound of sample number k, and an extracted sound correction part 120 for generating a corrected extracted sound e'(k) from the extracted sound e(k), by using sound collection venue sound power Yand the extracted sound power E. The extracted sound correction part 120 generates the corrected extracted sound e'(k) as e'(k)=0 when a ratio E/Yis within a prescribed small range, otherwise generates the corrected extracted sound as e'(k)=e(k).SELECTED DRAWING: Figure 4

Description

  The present invention relates to a technique for extracting sound generated in a sound field space.

  Echo canceller technology is used to extract the sound generated at the venue (Non-patent Document 1). Hereinafter, the echo canceller technique will be described with reference to FIG.

  First, a sound x (k) (k = 1, 2,..., Hereinafter referred to as a sample number) generated at the venue A is transmitted to the venue B. For example, when the input sound x (k) transmitted to the venue B is processed for each frame composed of K samples, K is the frame length. Further, the input sound x (k) is, for example, a telephone voice or a sound of a content distribution system installed in the venue A.

  Next, the input sound x (k) is reproduced from the speaker 910 installed at the venue B, and the convolutional input sound x ((from the speaker 910 to the microphone 920) is convoluted with the acoustic echo h0 (k) of the venue B. k) The sound is picked up by the microphone 920 installed at the venue B as * h0 (k). Here, * represents a convolution operator. In addition, the acoustic echo is sometimes referred to as a transfer function or an impulse response, but is hereinafter referred to as an acoustic echo or a transfer function.

  The sound collected by the microphone 920 is not limited to the convolution input sound x (k) * h0 (k). Environmental sound (for example, voice and applause sound) c1 (k) generated at the venue B is a convolutional environmental sound c1 (k) * h1 (k) in which the acoustic echo h1 (k) from the sound source to the microphone 920 is convoluted. Sound is collected as. Further, noise n (k) is also mixed in the microphone 920. Therefore, the sound y (k) picked up by the microphone 920 (hereinafter referred to as the picked-up venue sound y (k)) is y (k) = x (k) * h0 (k) + c1 (k) * h1 It can be expressed as (k) + n (k).

  If the picked-up venue sound y (k) is transmitted to the venue A as it is and reproduced at the venue A, howling and double talk will occur, making it difficult for the other party to hear. Therefore, the echo canceller 930 estimates x ′ (k) = x (k) * h0 ′ (k) that approximates the convolution input sound x (k) * h0 (k), and extracts the sound generated at the venue B. Extracted extraction sound e (k) = y (k) -x ′ (k) = c1 (k) * h1 (k) + n (k) + d (k) To venue A.

  However, since the estimated transfer function h0 '(k) and the true transfer function h0 (k) are different, a residual signal d (k) is generated. In order to solve this problem, an adaptive algorithm (adaptive filter) such as the NLMS (Normalized Least-Mean-Squares) method is used. When the NLMS method is used, the transfer function h0 '(k) can be brought close to the transfer function h0 (k). In particular, when the venue B is small (that is, the distance between the speaker and the microphone is relatively short), the transfer function h0 ′ (k) can be accurately estimated, so the amplitude value of the residual signal d (k) is sufficiently small, A sense of incongruity in hearing can be reduced.

Nobuhiko Kitawaki, “Digital Voice / Audio Technology”, Ohm Corporation, pp.223-225.

  By using the echo canceller technique, for example, when the speaker and the microphone are close to each other as in a speakerphone application, it is possible to appropriately extract the sound generated at the venue B.

  However, the speakers and microphones are separated from each other, and the environment is designed to add reverberation (for example, a viewing venue (venue A) that is different from a live venue in a live viewing venue (venue A) such as a concert venue or a theater venue) In the venue B)), the transfer function h0 ′ (k) cannot be sufficiently close to the transfer function h0 (k). For this reason, the amplitude value of the residual signal d (k) becomes large, and it is impossible to extract only the sound generated at the venue B.

  When using an echo canceller for voice calls using speakerphones, the remaining sound d (k) may be somewhat low because it is not necessary if the environmental sound c1 (k) including the sound in the conference room (venue B) does not reach the other party. Even if it remains, it is preferable to transmit the extracted sound e (k) as it is.

  However, in the case of live viewing, since the hearing at the live venue (venue A) is greatly impaired, it is not preferable to transmit the extracted sound e (k) as it is. In particular, instead of transmitting the environmental sound c1 (k) itself such as applause and hand clapping sounds generated at the viewing venue (venue B), the environmental sound c1 (k) is transmitted at the live venue (venue A) as the transmission destination. When transmitting the parameters used for generation, if the environmental sound c1 (k) generated at the viewing venue (Venue B) is mis-extracted, applause sounds and hand clapping sounds (ie A sound including a clapping sound and a sound other than a hand clapping sound) is generated, and the audience of the live venue (venue A) enjoying the original content is inconvenienced.

  FIG. 2 shows such a situation, and the input sound when the residual signal d (k) remains large (the difference between the input sound x (k) and the sound collecting hall sound y (k) is large). It is a figure which shows the mode of x (k), the sound collection hall sound y (k), and the extraction generated sound e (k). In the shaded portion of FIG. 2, when a normal echo canceller is used, the residual signal d (k) is large, and the sound x (k) reproduced from the speaker remains in the extraction generated sound e (k). This indicates that the environmental sound c1 (k) at the venue B is not correctly extracted.

  In other words, in a normal echo canceller, the uncomfortable feeling of hearing when playing a sound in a live venue (venue A) where the loudspeaker and microphone are not close to each other like the viewing venue (venue B). There is a problem that it is not possible to extract in a form that suppresses.

  Therefore, an object of the present invention is to provide an extracted sound correction technique that can correct an extracted sound obtained by extracting a sound generated in a large venue so as to suppress a sense of incongruity in hearing.

In one embodiment of the present invention, the sample number k is k = 1, 2,..., The sound of the sound collection venue sound y (k) is reproduced at the venue, and the sound of the sample number k is the sound collected at the venue. , Extraction sound e (k) is the sound of the sample number k obtained by extracting the sound generated at the venue from the sound collection venue sound y (k), the sound collection venue sound y (k) and the extraction occurrence sound e From (k), a power calculation unit that calculates the sound collection venue sound power Y _k that is the power of the sound collection venue sound y (k) and the extraction generation sound power E _k that is the power of the extraction generation sound e (k) And an extracted generated sound correcting unit that generates a corrected extracted generated sound e ′ (k) from the extracted generated sound e (k) using the sound collecting venue sound power Y _k and the extracted generated sound power E _k. The extracted generated sound correcting unit includes the corrected extracted generated sound e ′ (k) having a small ratio E _k / Y _{k of the} extracted generated sound power E _{k to} the sound collecting venue sound power Y _k . If it is within the specified range e 'and (k) = 0, otherwise, e' is generated as (k) = e (k).

In one embodiment of the present invention, the sample number k is k = 1, 2,..., The sound of the sound collection venue sound y (k) is reproduced at the venue, and the sound of the sample number k is the sound collected at the venue. , The extraction generated sound e (k) is the sound of the sample number k obtained by extracting the sound generated at the venue from the sound collecting venue sound y (k), and the extracted generated sound e (k) A sparse scale calculation unit for calculating a sparse measure S _k representing the degree of sparseness in the time direction of (k), and using the sparse measure S _k , a corrected extracted generated sound e from the extracted generated sound e (k) an extracted generated sound correcting unit that generates '(k), wherein the extracted generated sound correcting unit displays the corrected extracted generated sound e' (k) in a predetermined range indicating that the sparse measure _Sk is small. If e ′ (k) = 0, then e ′ (k) = 0. Otherwise, e ′ (k) = e (k).

  According to the present invention, it is possible to correct an extraction generated sound obtained by extracting a sound generated in a large venue so as to suppress a sense of incongruity in hearing.

The figure for demonstrating an echo canceller technique. The figure which shows an example of the mode of the input sound x (k), the sound collection hall sound y (k), and the extraction generated sound e (k) when the residual signal d (k) remains largely. The block diagram which shows an example of a structure of the extraction generation sound correction system 10 containing the extraction generation sound correction apparatus 100. FIG. FIG. 2 is a block diagram showing an example of the configuration of the extracted generated sound correcting apparatus 100. 5 is a flowchart showing an example of the operation of the extracted sound correcting apparatus 100. The figure which shows an example of the mode of the extraction generation sound e (k) which is the input of the extraction generation sound correction apparatus 100, and the corrected extraction generation sound e '(k) which is an output. FIG. 3 is a block diagram showing an example of the configuration of the extracted generated sound correcting apparatus 101. 5 is a flowchart showing an example of the operation of the extracted sound correcting apparatus 101. The figure which shows an example of the mode of an amplitude in case the environmental sound c1 (k) is a clap sound. The block diagram which shows an example of a structure of the extraction generated sound correction apparatus. The flowchart which shows an example of operation | movement of the extraction generated sound correction apparatus 200. The block diagram which shows an example of a structure of the extraction generation | occurrence | production sound correction apparatus 201. FIG. 5 is a flowchart showing an example of the operation of the extracted sound correcting apparatus 201. The block diagram which shows an example of a structure of the extraction generation | occurrence | production sound correction apparatus 300. FIG. 5 is a flowchart showing an example of the operation of the extracted sound correcting apparatus 300. The figure which shows an example of the mode of the corrected extraction generation | occurrence | production sound e '(k) in case environmental sound c1 (k) is a clapping sound. The block diagram which shows an example of a structure of the extraction generation | occurrence | production sound correction apparatus 301. FIG. The flowchart which shows an example of operation | movement of the extraction generation | occurrence | production sound correction apparatus 301.

  Hereinafter, embodiments of the present invention will be described in detail. In addition, the same number is attached | subjected to the structure part which has the same function, and duplication description is abbreviate | omitted.

  In the above description, the input sound x (k) transmitted to the venue B is described as a monaural signal, and the environmental sound generated at the venue B is one of c1 (k). k) may be a stereo signal, or there may be a plurality of environmental sounds generated at the venue B.

  For example, if the input sound is a monaural signal x (k) and the environmental sound generated at venue B is M (c1 (k), ..., cM (k), where M is an integer greater than or equal to 1, input Sound echoes of sound x (k), environmental sound c1 (k), ..., cM (k) are h0 (k), h1 (k), ..., hM (k) and noise is n (k), respectively. The sound collection venue sound y (k) collected by the microphone is y (k) = x (k) * h0 (k) + c1 (k) * h1 (k) +… + cM (k) * hM (k ) + n (k). Furthermore, assuming that the transfer function h0 (k) is a transfer function h0 ′ (k) and the residual signal is d (k), the extracted generated sound e (k) is e (k) = y (k) −x ( k) * h0 ′ (k) = c1 (k) * h1 (k) +... + cM (k) * hM (k) + n (k) + d (k).

  Also, the input sound is stereo signals xR (k), xL (k), and the environmental sound generated at venue B is M (c1 (k), ..., cM (k), where M is an integer of 1 or more) If there is an acoustic echo of input sound xR (k), xL (k), environmental sound c1 (k), ..., cM (k), hR0 (k), hL0 (k), h1 (k), ... , hM (k), where n (k) is the noise, the sound collected from the microphone y (k) is y (k) = xR (k) * hR0 (k) + xL (k) * hL0 (k) + c1 (k) * h1 (k) +... + cM (k) * hM (k) + n (k). Furthermore, when the transfer functions hR0 (k) and hL0 (k) are estimated as hR0 ′ (k) and hL0 ′ (k), respectively, and the residual signal is d (k), the extracted sound e (k) is e (k) = y (k)-{xR (k) * hR0 '(k) + xL (k) * hL0' (k)} = c1 (k) * h1 (k) +… + cM (k) * It can be expressed as hM (k) + n (k) + d (k).

  As can be seen from the three examples, the sound collecting venue sound y (k) can be expressed as the sum of the convolution input sound, the convolution environmental sound, and the noise. The extracted generated sound e (k) can be expressed as the sum of the convolution environmental sound, noise, and reverberation signal.

  In addition to monaural and stereo, there are various acoustic environments such as surround environment and 22.2ch environment, but the sound collection venue sound y (k) and extraction sound e (k) are the same as the above three examples. Can be handled.

  Therefore, in each embodiment described below, the case where the input sound is a monaural signal and there is one environmental sound will be described.

  First, terms used in each embodiment will be described.

  Let the sample number k be k = 1, 2,. When the input sound x (k) is processed for each frame composed of K samples, the i-th frame Xi is represented by a vector Xi = [x (1) x (2)… x (K)] can do. Here, i (i = 1, 2,...) Represents a frame number.

  Hereinafter, in order to simplify the description, each embodiment will be described assuming that Xi = x (k), that is, each frame is configured by one sample. The contents are the same as if you did.

  The sound collection venue sound y (k) is the sound of the sample number k obtained by collecting the sound reproduced at the venue and the sound generated at the venue. The sound reproduced at the venue is a sound transmitted from another venue and reproduced. In the example above, it is the sound of the live venue. In addition, the sound generated at the venue refers to environmental sounds and noises such as applause sounds generated at the viewing venue.

  The extracted generated sound e (k) is the sound of sample number k obtained by extracting the sound generated at the venue from the collected venue sound y (k).

<First embodiment>
Hereinafter, the extraction generated sound correction system 10 will be described with reference to FIG. FIG. 3 is a block diagram showing the configuration of the extracted sound correction system 10. As shown in FIG. 3, the extraction generated sound correction system 10 includes a speaker 910, a microphone 920, an echo canceller 930, and an extraction generated sound correction device 100.

  The speaker 910, the microphone 920, and the echo canceller 930 have the same functions as those in FIG. Specifically, the speaker 910 is a device that reproduces the transmitted sound x (k) of the venue A at the venue B. The microphone 920 is a convolution environment in which the convolution input sound x (k) * h0 (k) convolved with the input sound x (k) reproduced from the speaker 910 and the environmental sound c1 (k) generated in the venue B are convolved. It is a device that picks up sound c1 (k) * h1 (k) and noise n (k). The echo canceller 930 receives the input sound x (k) and the collected sound of the venue y (k) collected by the microphone 920 (= x (k) * h0 (k) + c1 (k) * h1 (k) + n From (k)), an extraction generated sound e (k) is generated. The extracted generated sound e (k) = c1 (k) * h1 (k) + n (k) + d (k) is a sound extracted by the echo canceller 930 from the sound generated at the venue B. Here, d (k) is a residual signal.

  In consideration of a transmission delay from the speaker 910 to the microphone 920, a predetermined delay may be added to the input sound x (k) instead of the input sound x (k), and then input to the echo canceller 930. . Further, considering the transmission delay from the speaker 910 to the microphone 920, a convolution obtained by convolving the input sound x (k) with a transfer function measured when the environmental sound or noise is sufficiently small instead of the input sound x (k). The input sound may be input to the echo canceller 930.

  The extracted generated sound correcting apparatus 100 generates a corrected extracted generated sound e ′ (k) from the collected sound hall sound y (k) and the extracted generated sound e (k). Here, the corrected extraction generated sound e ′ (k) is a sound of the sample number k obtained by correcting the extracted generated sound e (k) using the sound collection hall sound y (k) and the extracted generated sound e (k). is there. In general, the corrected extraction generated sound e '(k) is obtained by making the amplitude value of the residual signal d (k) sufficiently small so as not to cause a sense of incongruity in hearing.

  Hereinafter, the extracted sound correcting apparatus 100 will be described with reference to FIGS. FIG. 4 is a block diagram illustrating a configuration of the extraction generated sound correcting apparatus 100. FIG. 5 is a flowchart showing the operation of the extracted sound correcting apparatus 100. As shown in FIG. 4, the extracted / generated sound correction apparatus 100 includes a power calculation unit 110, an extracted / generated sound correction unit 120, and a recording unit 190. The recording unit 190 is a component that appropriately records information necessary for processing of the extraction generated sound correcting apparatus 100.

The operation of the extracted sound correcting apparatus 100 will be described with reference to FIG. Power calculation unit 110, a sound collection site sound y (k) and the extraction generated sound e (k), voice collecting hall sound y sound pickup hall sound is the power of (k) power Y _k and extraction generated sound e (k ) to calculate the extracted sound generated power E _k is a power of (S110). For example, the sound collecting hall sound power Y _k is calculated as the power of the sound collecting hall sound for a certain time (K samples (where K is an integer of 1 or more)) including the sound collecting hall sound y (k). When K samples before sample k are used, the following equation is obtained.

Further, the extracted generated sound power E _k is calculated as the power of the extracted generated sound for a certain time (K samples) including the extracted generated sound e (k). When K samples before sample k are used, the following equation is obtained.

Further, as described below, values obtained by dividing the above-described sound collection hall sound power Y _k and extraction generated sound power E _k by the number of samples K may be set as the sound collection hall sound power Y _k and the extraction generated sound power E _k. .

The extracted generated sound correcting unit 120 generates a corrected extracted generated sound e ′ (k) from the extracted generated sound e (k) using the sound collection venue sound power Y _k and the extracted generated sound power E _k (S120). . For example, when E _k > Y _k , e ′ (k) = e (k) is set, and when E _k ≦ Y _k , corrected extraction generated sound is generated with e ′ (k) = 0. The reason why e ′ (k) = e (k) when E _k > Y _k is that the environmental sound c1 (k) is sufficiently larger than the input sound x (k) and the residual signal d (k) is large. It is because it is considered. When E _k ≦ Y _k , e ′ (k) = 0 is set because the environmental sound c1 (k) is sufficiently smaller than the input sound x (k) output from the speaker 910, and the environmental sound c1. This is because (k) is considered buried. In relation to (Modification 2) described later, when E _k ≦ Y _k , it is considered that good results cannot be obtained even if e (k) is used for parameter generation. k) = 0.

  FIG. 6 is a diagram illustrating a state of the extracted generated sound e (k) that is the input of the extracted generated sound correcting apparatus 100 and the corrected extracted generated sound e ′ (k) that is the output. It can be seen from FIG. 6 that the influence of the input sound x (k) reproduced from the speaker 910 is reduced and the environmental sound c1 (k) can be extracted.

Note that α is a predetermined positive constant (hereinafter α is referred to as a magnification), and when E _k > αY _k , e ′ (k) = e (k) and E _k ≦ αY _k , E ′ (k) = 0 may be used to generate a corrected extraction generated sound.

Of course, E _k> Y _k, instead of _{E k ≦ Y k, E k} ≧ Y k, the _{E k <Y k, E k} > αY k, instead of _{E k ≦ αY k, E k} ≧ αY k , E _k <αY _k may be used.

In summary, including E = 1, if E _k / Y _k ≦ α or E _k / Y _k <α, e ′ (k) = 0, otherwise e ′ ( k) = e (k) is generated as corrected extraction generated sound e ′ (k). E _k / Y _k ≦ α or E _k / Y _k <α with respect to a predetermined positive constant α, the ratio E _k / Y of the extracted generated sound power E _{k to} the sound collecting venue sound power Y _k It is said that Y _k is in a predetermined range indicating that it is small. At this time, α is referred to as a value indicating that the ratio E _k / Y _k is within a predetermined range indicating that the ratio E _k / Y _k is small.

(Modification 1)
In the comparison of the sound collection venue sound power Y _k and the extraction generated sound power E _{k in} the extraction generated sound correction unit 120, the magnification α used for the comparison condition is treated as being unchanged regardless of the passage of time, but at a fixed timing. The magnification α may be updated. If the update is performed at a fixed timing, the magnification α can follow the change in the number of people in the venue, the change in the temperature of the venue, and the like.

For example, α ₀ (where α ₀ is a predetermined positive constant) may be used as an initial value of the magnification, and the magnification α may be updated and compared for each sample. Specifically, as k = 1,..., When E _k > α _k−1 Y _k , e ′ (k) = e (k), and when E _k ≦ α _k−1 Y _k , e ′ (k) = 0, and the corrected extraction generated sound e ′ (k) is generated. Next, α _k = E _k / Y _k is obtained as a magnification for the next sample. Similarly, when E _{k + 1} > α _k Y _{k + 1} , e ′ (k + 1) = e (k + 1) and E _{k + 1} ≦ α _k Y _{k + 1} , the corrected extraction generated sound e ′ (k + 1) is generated with e ′ (k + 1) = 0.

Note that the magnification α may be an average value of L samples as α _k = (E _{k−L + 1} / Y _{k−L + 1} +... + E _k / Y _k ) / L. Further, instead of a simple average, a filter in which a forgetting factor is set may be used for the calculation of the magnification α, or a linear function or a nonlinear function may be used. For example, let ω and σ be real constants (where ω> 0), and forgetting factor W (i) (i = 0,1, ...) defined using a Gaussian window, the magnification α _k is You may ask for.

(Modification 2)
The corrected generated sound parameter p (k), which is a parameter used by the extracted generated sound correcting device to generate the corrected extracted generated sound e ′ (k) instead of the corrected extracted generated sound e ′ (k) May be generated.

Hereinafter, the extracted sound correcting apparatus 101 will be described with reference to FIGS. FIG. 7 is a block diagram showing the configuration of the extracted sound correcting apparatus 101. As shown in FIG. FIG. 8 is a flowchart showing the operation of the extraction generated sound correcting apparatus 101. As can be seen from FIG. 7, the extracted generated sound correcting apparatus 101 differs from the extracted generated sound correcting apparatus 100 only in that it further includes a corrected extracted generated sound parameter generating unit 130. Further, as can be seen from FIG. 8, the operation of the extracted sound correcting device 101 is different from the operation of the extracted sound correcting device 100 only in that S130 is added. The corrected extracted generated sound parameter generation unit 130 generates a corrected extracted generated sound parameter p (k) from the corrected extracted generated sound e ′ (k) (S130). As long as it is a parameter that can be used to generate the corrected extracted generated sound e ′ (k), any corrected extracted generated sound parameter may be used. For example, the corrected extraction generated sound parameter p (k) can be generated using the power E ′ _k of the corrected extracted generated sound for a certain period including the corrected extracted generated sound e ′ (k) (formula ( 2)).

Specifically, an index obtained by quantizing the range of possible values of power E ′ _k into a predetermined number is set as a corrected extracted generated sound parameter p (k).

In addition, as shown in Equation (8) and Equation (6), which will be described later, a corrected extraction is performed in the same procedure using a sum of absolute values at a fixed time or an arbitrary sum of powers instead of the power E ′ _k. The generated sound parameter p (k) may be generated. Furthermore, the logarithm of them (logΣ _{i = k−K + 1} ^k e (i) × e (i), logΣ _{i = k−K + 1} ^k | e (i) |, logΣ _{i = k− K + 1} ^k e (i) × e (i) × e (i) × e (i)) is used to generate the corrected extracted generated sound parameter p (k) by the same procedure. Good.

  In addition, the sampling frequency of p (k) may be different from the sampling frequency of the transmission format. In this case, a new parameter p ′ (j) (j = 1, 2,..., Where j is a sample number) is generated by changing the sampling frequency of p (k) in consideration of the sampling frequency of the transmission format. You may do it. For example, if you want to change p (k) with a sampling frequency of 48 kHz to a parameter p '(k) sampled with a transmission format sampling frequency of 8 kHz, find the average value as shown below and change it. Alternatively, it may be changed using a resampler used for an acoustic signal.

  Furthermore, if the quantization accuracy of p (k) or p '(j) is different from the quantization accuracy of the transmission format, parameters p ^ (k) and p' ^ (j) are generated by converting the quantization accuracy. You may make it do. For example, if the quantization accuracy of p (k) is 16 bits and the quantization accuracy of p ^ (k) is 8 bits, simply shift p (k) to the right by 8 bits, that is, p ^ (k ) = p (k) >> 8 may be used to convert the quantization accuracy (instead of shifting to the right by 8 bits, it may be divided by 256). Alternatively, as with ITU-T G.711 A-law / μ-law conversion, the 16-bit precision is once reduced to 14-bit precision, and then converted to a 8-bit precision by conversion with a logarithmic function. Good. Note that the above processing can be performed using a lookup table such as p ^ (k) = LUT (p (k)) or p '^ (j) = LUT (p' (j)). Good.

  According to the present invention, it is possible to correct an extraction generated sound obtained by extracting a sound generated in a large venue so as to suppress a sense of incongruity in hearing. As a result, it is possible to appropriately extract and correct the sound generated in the venue where the speaker and the microphone are not close to each other and the reverberation is intentionally added.

<Second embodiment>
When the environmental sound c1 (k) is, for example, a clapping sound, the environmental sound c1 (k) is a sparse signal in the time direction, as shown by the shaded portion in FIG. Therefore, the degree of sparseness in the time direction of the extracted sound e (k) is examined, and if it is determined that it is sparse, it is determined that the environmental sound c1 (k) is sufficiently louder than the input sound x (k). If it is determined that e '(k) = e (k) and it is not sparse, the environmental sound c1 (k) is sufficiently smaller than the input sound x (k) and the environmental sound c1 (k) is If extraction cannot be performed, corrected extraction sound is generated with e ′ (k) = 0.

  Hereinafter, the extracted sound correcting apparatus 200 will be described with reference to FIGS. FIG. 10 is a block diagram showing the configuration of the extracted sound correcting apparatus 200. FIG. 11 is a flowchart showing the operation of the extraction generated sound correcting apparatus 200. As shown in FIG. 10, the extracted generated sound correcting apparatus 200 includes a sparse scale calculating unit 210, an extracted generated sound correcting unit 220, and a recording unit 190.

The operation of the extracted sound correcting apparatus 200 will be described with reference to FIG. The sparse scale calculation unit 210 calculates a sparse scale S _k of the extracted generated sound e (k) from the extracted generated sound e (k) (S210). The sparse scale is a value representing the degree of sparsity in the time direction. For example, the sparse measure S _k may be calculated by the following equation using the extracted generated sounds of K samples before the sample k.

Note that E2 _k and E4 _k are referred to as kurtosis, and the sparse scale S _k is sometimes referred to as a kurtosis coefficient. Further, sparse measure S _k may be calculated by the following equation.

Further, as another example, it may be using the ratio calculated from the mean value of the maximum value and the amplitude absolute value of the amplitude absolute value as sparse measure S _k.

Note that instead of the maximum amplitude absolute value, the average value of the top P (where P <K) may be E0 _k . That is, for j = 1, ..., P, let f _j be the jth largest value in | e (i) | (i = k-K + 1, ..., k) and E0 _k = (f ₁ + f ₂ + ... + f _P ) / P.

Extraction generated sound correction section 220 uses sparse measure S _k, to produce a corrected extraction generated sound e '(k) from the extract generated sound e (k) (S220). For example, if β is a predetermined constant (hereinafter, β is also referred to as a threshold), if S _k > β, e ′ (k) = e (k), and if S _k ≦ β, e ′ (k) = 0 to generate a corrected extraction generated sound.

Of course, S _k> β, instead of _{S k ≦ β, S k ≧} β, she may be used S _k <beta.

In summary, if S _k ≦ β or S _k <β, then e ′ (k) = 0, otherwise e ′ (k) = e (k) and corrected extracted generated sound e ′ Generate (k). S _k ≦ β or S _k <β with respect to a predetermined constant β is said to be within a predetermined range indicating that the sparse measure S _k is small. At this time, β is referred to as a value indicating that it is in a predetermined range indicating that the sparse measure _Sk is small.

(Modification 1)
In the comparison between the sparse scale _Sk and the threshold value β in the extraction sound correcting unit 220, the threshold value β used for the comparison condition is treated as being invariant regardless of the passage of time. However, the threshold value β is updated at a fixed timing. May be. If the update is performed at a certain timing, the threshold value β can follow the change in the number of people in the venue, the change in the temperature of the venue, and the like.

For example, β ₀ (where β ₀ is a predetermined number) may be used as an initial value of the threshold value, and the threshold value β may be updated and compared for each sample. Specifically, as k = 1,..., When S _k > β _k−1 , e ′ (k) = e (k), and when S _k ≦ β _k−1 , e ′ (k) = A corrected extraction generated sound e ′ (k) is generated as 0. Next, the threshold for the next sample is β _k = S _k, and similarly if S _{k + 1} > β _k , e ′ (k + 1) = e (k + 1) and S _{k + 1} In the case of ≦ β _k , corrected extraction generated sound e ′ (k + 1) is generated with e ′ (k + 1) = 0.

Note that the threshold value β may be an average value of L samples such as β _k = (S _{k−L + 1} +... + S _k ) / L. Further, instead of a simple average, a filter with a forgetting factor may be used for calculating the threshold value β, or a linear function or a nonlinear function may be used. For example, ω and σ are real constants (where ω> 0), and forgetting factor W (i) (i = 0,1, ...) defined using a Gaussian window, threshold β _k is as follows: You may ask for.

(Modification 2)
As described in the first embodiment, the extracted generated sound correcting device is a parameter used to generate a corrected extracted generated sound e ′ (k) instead of the corrected extracted generated sound e ′ (k). The corrected extracted generated sound parameter p (k) may be generated.

  Hereinafter, the extracted sound correcting apparatus 201 will be described with reference to FIGS. FIG. 12 is a block diagram showing the configuration of the extracted generated sound correcting apparatus 201. FIG. 13 is a flowchart showing the operation of the extraction generated sound correcting apparatus 201. As can be seen from FIG. 12, the extracted generated sound correcting device 201 is different from the extracted generated sound correcting device 200 only in that it further includes a corrected extracted generated sound parameter generating unit 130. Further, as can be seen from FIG. 13, the operation of the extracted sound correcting device 201 differs from the operation of the extracted sound correcting device 200 only in that S130 is added. The corrected extracted generated sound parameter generation unit 130 generates a corrected extracted generated sound parameter p (k) from the corrected extracted generated sound e ′ (k) (S130). As long as it is a parameter that can be used to generate the corrected extracted generated sound e ′ (k), any corrected extracted generated sound parameter may be used.

  According to the present invention, it is possible to correct an extraction generated sound obtained by extracting a sound generated in a large venue so as to suppress a sense of incongruity in hearing. As a result, it is possible to appropriately extract and correct the sound generated in the venue where the speaker and the microphone are not close to each other and the reverberation is intentionally added.

<Third embodiment>
In the first embodiment, the power ratio E _k / Y _k is used, and in the second embodiment, the sparse scale S _k is used to determine the condition for correcting the extracted sound. Here, the condition determination is performed using two power ratios E _k / Y _k and sparse scale S _k .

  Hereinafter, the extracted sound correcting apparatus 300 will be described with reference to FIGS. FIG. 14 is a block diagram illustrating a configuration of the extracted generated sound correcting apparatus 300. FIG. 15 is a flowchart showing the operation of the extracted sound correcting apparatus 300. As shown in FIG. 14, the extraction generated sound correction apparatus 300 includes a power calculation unit 110, a sparse scale calculation unit 210, an extraction generated sound correction unit 320, and a recording unit 190.

The operation of the extracted sound correcting apparatus 300 will be described with reference to FIG. Power calculation unit 110, a sound collection site sound y (k) and the extraction generated sound e (k), voice collecting hall sound y sound pickup hall sound is the power of (k) power Y _k and extraction generated sound e (k ) to calculate the extracted sound generated power E _k is a power of (S110). The sparse scale calculation unit 210 calculates a sparse scale S _k of the extracted generated sound e (k) from the extracted generated sound e (k) (S210). The generated sound extraction unit 320 generates a corrected extracted generated sound e ′ (k) from the extracted generated sound e (k) using the sound collecting venue sound power Y _k , the extracted generated sound power E _k, and the sparse scale S _k. (S320). For example, if E _k > αY _k and S _k > β (E _k ≧ αY _k and S _k ≧ β) with respect to the magnification α and the threshold β, e ′ (k) = e (k) Otherwise, the corrected extraction sound is generated with e ′ (k) = 0. At that time, for example, Y _k and E _k can be calculated using Expressions (1) and (2), and S _k can be calculated using Expressions (5) to (7). As for S _k , equations (8) to (10) and equations (11) to (13) may be used instead of equations (5) to (7).

Of course, for example, if E _k ≦ αY _k and S _k ≦ β (E _k <αY _k and S _k <β) with respect to the magnification α and the threshold β, e ′ (k) = 0, otherwise In this case, the corrected extraction generated sound may be generated as e ′ (k) = e (k).

Furthermore, a plurality of conditions may be used for condition determination for correcting the extracted generated sound in the extracted generated sound correcting unit 320. For example, the generated sound extraction unit 320 uses the sound collection hall sound power Y _k , the extracted generated sound power E _k, and the sparse scale S _k to correct the extracted extracted generated sound e ′ (k) from the extracted generated sound e (k). Is generated when either of the following first condition or second condition is satisfied with respect to the magnification α1, α2 and the threshold values β1, β2 (where α1 <α2, β1> β2). ) = e (k), otherwise, e ′ (k) = 0 and a corrected extracted sound is generated.
First condition: E _k > α1 · Y _k and S _k > β1
Second condition: E _k > α2 · Y _k and S _k > β2

At that time, Y _k and E _k can be calculated using equations (3) and (4), and S _k can be calculated using equations (11) to (13). In this case, for example, α1 = 0.5, α2 = 1, β1 = 12, and β2 = 9.

  FIG. 16 shows a corrected extraction generated sound e ′ (k) generated using the power ratio as in the first embodiment with respect to the extracted generated sound e (k) using the applause sound as the environmental sound (FIG. 16 (b)). )) And the corrected extracted generated sound e ′ (k) (FIG. 16C) generated using the power ratio and the sparse scale as in the present embodiment. Comparing FIG. 16 (b) and FIG. 16 (c), it can be seen that the present embodiment (FIG. 16 (c)) has better extraction / correction accuracy of the applause sound which is the environmental sound.

Of course, if any of the following third condition and fourth condition is satisfied with respect to the magnification α1, α2 and the threshold values β1, β2 (where α1 <α2, β1> β2), e ′ (k) = 0 Otherwise, the corrected extraction generated sound may be generated as e ′ (k) = e (k).
Third condition: E _k ≦ α1 · Y _k and S _k ≦ β1
Fourth condition: E _k ≦ α2 · Y _k and S _k ≦ β2

(Modification 1)
Similar to the first embodiment and the second embodiment, the magnification α and the threshold β may be changed for each sample.

(Modification 2)
In addition, as described in the first embodiment and the second embodiment, the extracted generated sound correcting device generates a corrected extracted generated sound e ′ (k) instead of the corrected extracted generated sound e ′ (k). The corrected extracted sound parameter p (k), which is a parameter used for this purpose, may be generated.

  Hereinafter, the extracted sound correcting device 301 will be described with reference to FIGS. FIG. 17 is a block diagram illustrating a configuration of the extraction generated sound correcting apparatus 301. FIG. 18 is a flowchart showing the operation of the extraction generated sound correcting apparatus 301. As can be seen from FIG. 17, the extracted generated sound correcting device 301 is different from the extracted generated sound correcting device 300 only in that it further includes a corrected extracted generated sound parameter generation unit 130. Further, as can be seen from FIG. 18, the operation of the extracted sound correcting device 301 is different from the operation of the extracted sound correcting device 300 only in that S130 is added. The corrected extracted generated sound parameter generation unit 130 generates a corrected extracted generated sound parameter p (k) from the corrected extracted generated sound e ′ (k) (S130). As long as it is a parameter that can be used to generate the corrected extracted generated sound e ′ (k), any corrected extracted generated sound parameter may be used.

  According to the present invention, it is possible to correct an extraction generated sound obtained by extracting a sound generated in a large venue so as to suppress a sense of incongruity in hearing. As a result, it is possible to appropriately extract and correct the sound generated in the venue where the speaker and the microphone are not close to each other and the reverberation is intentionally added.

<Supplementary note>
The apparatus of the present invention includes, for example, a single hardware entity as an input unit to which a keyboard or the like can be connected, an output unit to which a liquid crystal display or the like can be connected, and a communication device (for example, a communication cable) capable of communicating outside the hardware entity Can be connected to a communication unit, a CPU (Central Processing Unit, may include a cache memory or a register), a RAM or ROM that is a memory, an external storage device that is a hard disk, and an input unit, an output unit, or a communication unit thereof , A CPU, a RAM, a ROM, and a bus connected so that data can be exchanged between the external storage devices. If necessary, the hardware entity may be provided with a device (drive) that can read and write a recording medium such as a CD-ROM. A physical entity having such hardware resources includes a general-purpose computer.

  The external storage device of the hardware entity stores a program necessary for realizing the above functions and data necessary for processing the program (not limited to the external storage device, for example, reading a program) It may be stored in a ROM that is a dedicated storage device). Data obtained by the processing of these programs is appropriately stored in a RAM or an external storage device.

  In the hardware entity, each program stored in an external storage device (or ROM or the like) and data necessary for processing each program are read into a memory as necessary, and are interpreted and executed by a CPU as appropriate. . As a result, the CPU realizes a predetermined function (respective component requirements expressed as the above-described unit, unit, etc.).

  The present invention is not limited to the above-described embodiment, and can be appropriately changed without departing from the spirit of the present invention. In addition, the processing described in the above embodiment may be executed not only in time series according to the order of description but also in parallel or individually as required by the processing capability of the apparatus that executes the processing. .

  As described above, when the processing functions in the hardware entity (the apparatus of the present invention) described in the above embodiments are realized by a computer, the processing contents of the functions that the hardware entity should have are described by a program. Then, by executing this program on a computer, the processing functions in the hardware entity are realized on the computer.

  The program describing the processing contents can be recorded on a computer-readable recording medium. As the computer-readable recording medium, for example, any recording medium such as a magnetic recording device, an optical disk, a magneto-optical recording medium, and a semiconductor memory may be used. Specifically, for example, as a magnetic recording device, a hard disk device, a flexible disk, a magnetic tape or the like, and as an optical disk, a DVD (Digital Versatile Disc), a DVD-RAM (Random Access Memory), a CD-ROM (Compact Disc Read Only). Memory), CD-R (Recordable) / RW (ReWritable), etc., magneto-optical recording medium, MO (Magneto-Optical disc), etc., semiconductor memory, EEP-ROM (Electronically Erasable and Programmable-Read Only Memory), etc. Can be used.

  The program is distributed by selling, transferring, or lending a portable recording medium such as a DVD or CD-ROM in which the program is recorded. Furthermore, the program may be distributed by storing the program in a storage device of the server computer and transferring the program from the server computer to another computer via a network.

  A computer that executes such a program first stores, for example, a program recorded on a portable recording medium or a program transferred from a server computer in its own storage device. When executing the process, the computer reads a program stored in its own recording medium and executes a process according to the read program. As another execution form of the program, the computer may directly read the program from a portable recording medium and execute processing according to the program, and the program is transferred from the server computer to the computer. Each time, the processing according to the received program may be executed sequentially. Also, the program is not transferred from the server computer to the computer, and the above-described processing is executed by a so-called ASP (Application Service Provider) type service that realizes the processing function only by the execution instruction and result acquisition. It is good. Note that the program in this embodiment includes information that is used for processing by an electronic computer and that conforms to the program (data that is not a direct command to the computer but has a property that defines the processing of the computer).

  In this embodiment, a hardware entity is configured by executing a predetermined program on a computer. However, at least a part of these processing contents may be realized by hardware.

Claims (7)

The sample number k is k = 1, 2,..., The sound of the sound collection venue sound y (k) is reproduced at the venue and the sound of the sample number k is the sound generated at the venue, and the extraction generated sound e (k ) As the sound of sample number k obtained by extracting the sound generated at the venue from the sound collection venue sound y (k),
From the sound collecting venue sound y (k) and the extraction generated sound e (k), the sound collecting venue sound power Y _k which is the power of the sound collecting venue sound y (k) and the extraction generated sound e (k) A power calculation unit for calculating extraction generated sound power E _k that is power;
An extracted generated sound correcting unit that generates a corrected extracted generated sound e ′ (k) from the extracted generated sound e (k) using the sound collecting venue sound power Y _k and the extracted generated sound power E _k. Extraction sound correction device,
The extraction generated sound correction unit is a predetermined value indicating that the ratio E _k / Y _{k of the} extracted generated sound power E _{k to} the collected sound source sound power Y _k of the corrected extracted generated sound e ′ (k) is small. If it is within the range, e ′ (k) = 0, otherwise, it is generated as e ′ (k) = e (k). The sample number k is k = 1, 2,..., The sound of the sound collection venue sound y (k) is reproduced at the venue and the sound of the sample number k is the sound generated at the venue, and the extraction generated sound e (k ) As the sound of sample number k obtained by extracting the sound generated at the venue from the sound collection venue sound y (k),
From the extracted sound generated e (k), and sparse scale calculation unit for calculating a sparse measure S _k representing the degree of sparse is in the time direction of the extraction sound generated e (k),
An extraction generation sound correction apparatus including an extraction generation sound correction unit that generates a corrected extraction generation sound e ′ (k) from the extraction generation sound e (k) using the sparse scale S _k ,
The extraction generated sound correction unit, the corrected extraction generated sound e 'a (k), when in a predetermined range which indicates that the sparse measure S _k is small, e' and (k) = 0, otherwise In this case, the extracted sound correcting device is generated as e ′ (k) = e (k). The extraction generated sound correction apparatus according to claim 1 or 2,
further,
Generation of corrected extraction generated sound parameters for generating corrected extraction generated sound parameters p (k) used for generating the corrected extracted generated sound e ′ (k) from the corrected extracted generated sound e ′ (k) And an extracted generated sound correcting device. The extraction generated sound correction device according to any one of claims 1 to 3,
The extraction sound correcting apparatus according to claim 1, wherein the value indicating the small range is updated for each sample. The sample number k is k = 1, 2,..., The sound of the sound collection venue sound y (k) is reproduced at the venue and the sound of the sample number k is the sound generated at the venue, and the extraction generated sound e (k ) As the sound of sample number k obtained by extracting the sound generated at the venue from the sound collection venue sound y (k),
Extraction sound generation correction device, the sound collection venue sound power Y _k that is the power of the sound collection venue sound y (k) from the sound collection venue sound y (k) and the extraction occurrence sound e (k) and the extraction A power calculating step for calculating an extracted generated sound power E _k that is the power of the generated sound e (k);
The extracted generated sound correcting device generates a corrected extracted generated sound e ′ (k) from the extracted generated sound e (k) using the sound collecting venue sound power Y _k and the extracted generated sound power E _k. Extraction sound correction method including extraction sound correction step,
The extraction generated sound correction step is a predetermined step that indicates that the ratio E _k / Y _{k of the} extracted generated sound power E _{k to} the collected sound source sound power Y _k of the corrected extracted generated sound e ′ (k) is small. If the frequency is within the range, e ′ (k) = 0; otherwise, e ′ (k) = e (k) is generated. The sample number k is k = 1, 2,..., The sound of the sound collection venue sound y (k) is reproduced at the venue and the sound of the sample number k is the sound generated at the venue, and the extraction generated sound e (k ) As the sound of sample number k obtained by extracting the sound generated at the venue from the sound collection venue sound y (k),
Extraction generated sound correction device, from the extracted sound generated e (k), and sparse scale calculation step of calculating a sparse measure S _k representing the degree of sparse in the time direction of the extraction sound generated e (k),
The extraction sound generated correction device, using said sparse measure S _k, the extracting generated sound e (k) from the corrected extraction generated sound e '(k) extracting generated sound including an extraction sound generated correction step of generating Correction method,
The extraction sound generated correction step, the corrected extraction generated sound e 'a (k), when in a predetermined range which indicates that the sparse measure S _k is small, e' and (k) = 0, otherwise In the case of, the extracted sound correction method generated as e ′ (k) = e (k).   A program for causing a computer to function as the extraction generated sound correcting apparatus according to any one of claims 1 to 4.