JP2010217551A - Sound processing device and program - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To create natural sound in which objective sound is easily listened to, from mixed sound of the objective sound and non-objective sound. <P>SOLUTION: A sound source separation section 30 creates an objective sound spectrum QA1 in which the objective sound is composed of components of an objective sound frequency FA where the objective sound is dominant, and the non-objective sound spectrum QB2 coming from a direction different from the objective sound, in which the non-objective sound is composed of components of a non-objective sound frequency FB where the non-objective sound is dominant, from a plurality of sound signals (S1, S2). A variation sound suppressing section 60 suppresses the non-objective sound in the non-objective sound spectrum QB2 after separation by the sound source separation section 30. A combining section 54 combines the objective spectrum QA2 after separation by the sound source separation section 30, and the non-objective sound spectrum QB3 after processing by the variation sound suppressing section 60. <P>COPYRIGHT: (C)2010,JPO&INPIT

Description

  The present invention relates to a technique for separating sound coming from a predetermined direction (hereinafter referred to as “target sound”) and sound other than the target sound (hereinafter referred to as “non-target sound”).

  Conventionally, a technology for selecting each of a plurality of frequencies (frequency bands) in a plurality of sound signals generated by a plurality of sound collecting devices into a target sound frequency in which the target sound is dominant and a non-target sound frequency in which the non-target sound is dominant. Proposed by For example, Non-Patent Document 1 discloses a technique (SAFIA) for selecting a frequency having a high intensity of a sound signal generated by a sound collecting device close to a target sound source among a plurality of sound signals as a target sound frequency. Further, in Patent Document 1, a target sound dominance signal in which a target sound is emphasized and a target sound inferior signal in which the target sound is suppressed are generated by delaying and adding a plurality of sound signals (that is, beam formation), and the target sound dominance is generated. A technique is disclosed in which a frequency whose signal intensity exceeds the intensity of the target sound inferior signal is selected as the target sound frequency. According to the techniques of Non-Patent Document 1 and Patent Document 1, generation of a target sound spectrum composed of components of each target sound frequency and a non-target sound spectrum composed of components of each non-target sound frequency (that is, sound source separation) Is possible.

Mariko Aoki, et al., "Sound source segregation based on controlling incident angle of each frequency component of input signals acquired by multiple microphones", Acoustical Science and Technology, Vol.22, No.2 p.149-p.157, 2001

JP 2006-197552 A

  Since the intensity at the non-target sound frequency in the target sound spectrum is zero, the reproduced sound generated only from the target sound spectrum has an unnatural impression sound. In order to solve the above problems, for example, a method of constraining each frequency component (non-target sound frequency) of the non-target sound spectrum and then synthesizing it to the target sound spectrum can be considered. However, when the non-target sound spectrum is excessively suppressed, the intensity change on the frequency axis and the time axis becomes excessive, and an unpleasant musical noise may occur. On the other hand, if the suppression of the non-target sound spectrum is insufficient, the non-target sound is reproduced while maintaining a considerable intensity (volume), so that it is difficult to listen to the target sound.

  Note that, under the techniques of Non-Patent Document 1 and Patent Document 1, the target sound and the non-target sound are distinguished on the basis of whether or not the sound is an incoming sound from a predetermined direction. Therefore, there is noise that is stationary in time (hereinafter referred to as “non-target steady sound”), such as operating noise of air conditioning equipment and crowded noise in crowds, and acoustic characteristics (for example, volume and pitch) ) Is changing every moment, in an environment where sounds such as sounds and musical sounds (hereinafter referred to as “non-target fluctuation sounds”) arrive from a different direction from the target sound, both non-target stationary sounds and non-target fluctuation sounds are not distinguished. Extracted as the target sound. Since non-target fluctuation sounds are more likely to have similar acoustic characteristics to target sounds (speech and musical sounds) compared to non-target stationary sounds, The problem that listening to the target sound (distinguishing between the target sound and the non-target fluctuation sound) becomes difficult due to insufficient suppression of the target sound spectrum becomes particularly serious. In view of the above circumstances, an object of the present invention is to generate a natural sound from which a target sound can be easily heard from a mixed sound of a target sound and a non-target sound (non-target fluctuation sound).

  In order to solve the above problems, a sound processing apparatus according to the present invention includes a target sound spectrum including a target sound frequency component in which a target sound is dominant from a plurality of sound signals generated by a plurality of sound collecting devices. A sound source separation means for generating a non-target sound spectrum composed of components of a non-target sound frequency in which the non-target sound coming from a different direction from the target sound is generated, and the non-target sound spectrum after being separated by the sound source separation means Fluctuation sound suppression means for suppressing non-target fluctuation sound (typically disturbing sound) in the non-target sound spectrum QB1 and non-target sound spectrum QB2 in FIG. 1, for example, and target sound after separation by the sound source separation means Synthesis means for synthesizing a spectrum (for example, target sound spectrum QA1 or target sound spectrum QA2 in FIG. 1) and a non-target sound spectrum (for example, non-target sound spectrum QB3 in FIG. 1) processed by the fluctuation sound suppression means Be prepared. In the above configuration, since the non-target sound spectrum after suppression of the non-target fluctuation sound is synthesized with the target sound spectrum, it is easier to listen to the target sound as compared with the configuration in which the non-target fluctuation sound is not suppressed, and Compared to a configuration in which sound is generated only from the target sound spectrum, it is possible to generate sound that is natural in terms of hearing from a mixed sound of the target sound and the non-target sound (non-target fluctuation sound).

  Since the target of synthesis by the synthesizing means is the “target sound spectrum after separation by the sound source separation means”, the target sound spectrum is executed in the process from separation by the sound source separation means to synthesis by the synthesis means. The presence / absence and contents of processing are unquestioned. Similarly, the presence / absence and contents of the process executed on the non-target sound spectrum in the process from the process by the fluctuating sound suppression means to the synthesis by the synthesis means are not questioned. In addition, the “frequency” in the present invention is a concept including a frequency band having a spread on the frequency axis in addition to a single frequency on the frequency axis.

  The sound processing apparatus according to a preferred aspect of the present invention includes intensity adjustment means for suppressing the non-target sound spectrum after separation by the sound source separation means in accordance with a suppression coefficient (a coefficient common to each frequency). In the above aspect, even when the suppression coefficient is set so that musical noise does not occur due to suppression by the intensity adjustment unit (when suppression by the intensity adjustment unit is relaxed), the fluctuation sound suppression unit is not By suppressing the target fluctuation sound, it is possible to generate sound that allows easy listening of the target sound. Note that there is no problem in the present invention between the processing by the intensity adjusting means and the processing by the fluctuation sound suppressing means.

  In a preferred aspect of the present invention, the fluctuating sound suppression means includes a gain setting means for generating a suppression gain sequence composed of gains set for each frequency, and each frequency of the non-target sound spectrum after separation by the sound source separation means. And a suppression processing unit that suppresses the non-target fluctuation sound by adjusting the intensity according to each gain of the suppression gain series. In the above aspect, since the intensity at each frequency of the non-target sound spectrum is adjusted according to the gain set for each frequency, for example, only one band corresponding to the non-target fluctuation sound in the non-target sound spectrum is included. Compared with a configuration that suppresses noise, there is an advantage that non-target fluctuation sound can be precisely suppressed for each frequency.

  In a preferred aspect of the present invention, the gain setting means includes an emphasis gain sequence configured with a gain set for each frequency so that the non-target fluctuation sound is emphasized in the non-target sound spectrum after separation by the sound source separation means. And a second processing means for generating a suppression gain sequence from the enhancement gain sequence. In the above aspect, a known technique for emphasizing a non-objective fluctuation sound (typically speech) from a mixed sound of a non-objective fluctuation sound and a non-objective steady sound can be applied to generation of an emphasis gain sequence (spectral gain). There is an advantage.

  In a further preferred aspect, the second processing means generates a suppression gain sequence according to the enhancement gain sequence and the variably set adjustment value. For example, a configuration in which the second processing unit calculates a gain at the frequency of the suppression gain sequence by subtracting a multiplication value of the gain at each frequency of the enhancement gain sequence and the adjustment value set variably from a predetermined value (for example, Formula (6a)) or a configuration in which the second processing means calculates the gain at the frequency of the suppression gain sequence by multiplying the difference value between the predetermined value and the gain at each frequency of the enhancement gain sequence by a variable adjustment value. (For example, Formula (6b)) is preferable. According to the above aspect, since the adjustment value is variably set, it is possible to appropriately adjust the degree of suppression of the non-target fluctuation sound. For example, according to the configuration in which the second processing unit variably controls the adjustment value according to the state of the non-target sound (for example, intensity or SN ratio), it is possible to prevent insufficient or excessive suppression of the non-target sound. is there.

  A sound processing apparatus according to a preferred aspect of the present invention includes a noise estimation unit that generates a noise spectrum composed of non-target stationary sounds among non-target sound spectra after separation by a sound source separation unit, and separation by a sound source separation unit. Noise suppression means for suppressing the non-target stationary sound of the noise spectrum from the subsequent target sound spectrum, and the gain setting means includes the non-target sound spectrum after separation by the sound source separation means and the noise spectrum estimated by the noise estimation means. Then, a suppression gain sequence is generated, and the synthesis unit synthesizes the target sound spectrum after processing by the noise suppression unit and the non-target sound spectrum after processing by the fluctuation sound suppression unit. In the above aspect, the noise suppression means suppresses the non-target stationary sound from the target sound spectrum and suppresses the non-target fluctuation sound of the non-target sound spectrum synthesized with the target sound. There is an advantage that it is possible to generate a sound that effectively suppresses the target fluctuation sound. In addition, since a noise spectrum composed of non-target steady sounds (that is, a noise spectrum in which non-target fluctuation sounds are suppressed) is used to generate a suppression gain series, a suppression gain series that can suppress non-target fluctuation sounds with high accuracy. Can be generated. Furthermore, since a common noise spectrum is used for suppression of non-target stationary sound by the noise suppression means and generation of suppression gain sequences by the gain setting means, separate suppression of non-target stationary sounds and generation of suppression gain sequences are required. Compared with a configuration using a noise spectrum, there is also an advantage that the configuration and processing of the sound processing device are simplified.

  In a further preferred aspect, the noise estimation means is configured such that the intensity of one non-target sound frequency component in the first unit interval (for example, the intensity XB1 (n, k) in FIG. 4) is equal to the first unit interval before the start of the first unit interval. When the noise spectrum of two unit intervals is below a threshold value (for example, threshold value XTH in FIG. 4) exceeding the intensity at one non-target sound frequency (for example, intensity μ (n−1, k) in FIG. 4), the first unit interval The intensity at one non-target sound frequency in the noise spectrum (for example, the intensity μ (n, k) in FIG. 4) is calculated from the intensity of one non-target sound frequency component in the first unit section and the second unit section. If the intensity of one non-target sound frequency component in the first unit section exceeds a threshold value, the one in the noise spectrum of the first unit section is set. The intensity at the non-target sound frequency is Without reflecting the intensity of the component of the non-target sound frequency, it is set according to the intensity at one non-target sound frequency in the noise spectrum of the second unit section. In the above configuration, when the intensity of one non-target sound frequency component in the first unit section exceeds a threshold (for example, when a non-target fluctuation sound is generated at one non-target sound frequency), the first unit section Since the intensity of the noise spectrum is set without reflecting the intensity of the component of the non-target sound frequency in, only the non-target stationary sound is extracted with high accuracy (that is, the non-target fluctuation sound is effectively suppressed). Can be generated.

  The intensity of the noise spectrum of the first unit section is the intensity of the noise spectrum of one unit section (second unit section) before the start of the first unit section (for example, immediately before), or before the start of the first unit section. It is set according to the intensity of each noise spectrum of a plurality of unit intervals (second unit intervals). According to a preferred aspect of the present invention, when the intensity of one non-target sound frequency component in the first unit section is below a threshold, the noise estimation means The weighted sum (for example, Equation (2)) with the intensity at one non-target sound frequency in the noise spectrum of the second unit section is set as the intensity at one non-target sound frequency in the noise spectrum of the first unit section. To do. In the above aspect, since the weighted sum of the intensity of the non-target sound frequency in the first unit interval and the intensity of the noise spectrum in the second unit interval is calculated as the intensity of the noise spectrum in the first unit interval, There is an advantage that it is not necessary to hold the noise spectrum over a plurality of past unit intervals when viewed from the interval.

  In a preferred aspect of the present invention, when the intensity of the component of one non-target sound frequency in the first unit section exceeds the threshold, the noise estimation unit is configured to detect the noise at one non-target sound frequency in the noise spectrum of the second unit section. The intensity is set as the intensity at one non-target sound frequency in the noise spectrum of the first unit section (for example, Equation (3)). In the above aspect, since the intensity of the noise spectrum of the second unit interval is applied as the intensity of the noise spectrum of the first unit interval, the process of specifying the intensity of the noise spectrum of the first unit interval is simplified. There are advantages.

  In a preferred aspect of the present invention, when the intensity of the component of one non-target sound frequency in the first unit section exceeds the threshold, the noise estimation unit is configured to detect the noise at one non-target sound frequency in the noise spectrum of the second unit section. A numerical value exceeding the intensity is set as the intensity at one non-target sound frequency in the noise spectrum of the first unit interval (for example, Equation (3a)). In the above aspect, since a numerical value exceeding the noise spectrum intensity of the second unit section is applied as the noise spectrum intensity of the first unit section, the non-target steady sound newly generated during the operation of the sound processing apparatus is detected. It can be appropriately included in the noise spectrum.

  The sound processing apparatus according to each aspect described above is realized by hardware (electronic circuit) such as a DSP (Digital Signal Processor) dedicated to sound processing, and a general-purpose arithmetic processing apparatus such as a CPU (Central Processing Unit). This is also realized through collaboration with programs. The program according to the present invention includes a target sound spectrum composed of a component of a target sound frequency in which a target sound is dominant and a target sound coming from a different direction from a plurality of sound signals generated by a plurality of sound collecting devices. A sound source separation process that generates a non-target sound spectrum composed of components of a non-target sound frequency in which the target sound is dominant, and a variable sound suppression that suppresses non-target fluctuation sound in the non-target sound spectrum after the sound source separation process The computer is caused to execute a process and a synthesis process for synthesizing the target sound spectrum after the sound source separation process and the non-target sound spectrum after the fluctuation sound suppression process. According to the above program, the same operation and effect as the signal processing apparatus according to the present invention are exhibited. The program of the present invention is provided to a user in a form stored in a computer-readable recording medium and installed in the computer, or provided from a server device in a form of distribution via a communication network and installed in the computer. Is done.

1 is a block diagram of a sound processing apparatus according to a first embodiment of the present invention. It is a block diagram of a sound source separation part. It is a graph for demonstrating the process by a signal processing part. It is a flowchart of operation | movement of a noise estimation part. It is a spectrogram of the noise spectrum in a 1st embodiment. It is the spectrogram of the noise spectrum in contrast. It is a block diagram of a fluctuation sound suppression part.

<A: First Embodiment>
FIG. 1 is a block diagram of a sound processing apparatus according to the first embodiment of the present invention. As shown in FIG. 1, a sound collection device M1 and a sound collection device M2 are connected to the sound processing apparatus 100. The sound collection device M1 and the sound collection device M2 are omnidirectional (substantially omnidirectional) microphones that generate a signal representing a surrounding acoustic waveform. The mixed sound of the target sound and the non-target sound reaches the sound collecting device M1 and the sound collecting device M2 from the surroundings. Each of the sound collecting device M1 and the sound collecting device M2 generates an electrical signal representing a waveform of a mixed sound of the target sound and the non-target sound. The sound collecting device M1 generates a sound signal S1, and the sound collecting device M2 generates a sound signal S2.

  The target sound is sound that arrives at the sound collecting device M1 and the sound collecting device M2 from the known direction D0. For example, when the sound processing apparatus 100 is mounted on an electronic device (for example, a mobile phone) to which a user's speech sound is input, the speech sound arrives as a target sound from the front direction D0 with respect to the main body of the electronic device. The sound collection device M1 and the sound collection device M2 are arranged apart from each other along a direction perpendicular to the direction D0 in which the target sound arrives. On the other hand, the non-target sound is sound coming from a direction (DR, DL) different from the direction D0 of the target sound. The non-target sound arrives at the sound collecting device M1 and the sound collecting device M2 from the direction DR of 45 ° clockwise with respect to the direction D0 and the direction DL of 45 ° counterclockwise with respect to the direction D0.

  The sound processing apparatus 100 generates a sound signal SOUT that suppresses the non-target sound of the mixed sound of the target sound and the non-target sound from the sound signal S1 and the sound signal S2. The sound signal SOUT is reproduced as sound by being supplied to a sound emitting device (for example, a speaker or headphones). Note that an A / D converter that converts the sound signal S1 and the sound signal S2 into a digital signal and a D / A converter that converts the sound signal SOUT into an analog signal are omitted for convenience.

  As shown in FIG. 1, the sound processing device 100 is realized by a computer system including an arithmetic processing device 12 and a storage device 14. The storage device 14 stores a program and various data for generating the sound signal SOUT from the sound signal S1 and the sound signal S2. A known recording medium such as a magnetic recording medium or a semiconductor recording medium is arbitrarily employed as the storage device 14. The arithmetic processing unit 12 executes a program stored in the storage device 14 to execute a plurality of elements (frequency analysis unit 20, sound source separation unit 30, noise estimation unit 42, noise suppression unit 44, intensity adjustment unit 52, synthesis unit). 54, an inverse conversion unit 56, and a fluctuating sound suppression unit 60). A configuration in which an electronic circuit (DSP) dedicated to sound processing realizes each element of the arithmetic processing device 12 and a configuration in which each element of the arithmetic processing device 12 is mounted in a plurality of integrated circuits are also adopted. .

  The frequency analysis unit 20 calculates a frequency spectrum P1 for each of a plurality of unit sections (frames) obtained by dividing the sound signal S1 on the time axis. For specifying the frequency spectrum P1, known frequency analysis such as FFT (Fast Fourier Transform) processing is arbitrarily employed. Further, the frequency analysis unit 20 specifies the frequency spectrum P2 for each unit section of the sound signal S2 by the same method as the specification of the frequency spectrum P1.

  The sound source separation unit 30 of FIG. 1 uses the target sound frequency FA and the non-target sound for each of the K (K is a natural number) frequencies (frequency bands) f1 to fK discretely set on the frequency axis. By selecting the frequency FB, the target sound spectrum QA1 and the non-target sound spectrum QB1 are generated for each unit section. The target sound frequency FA is a frequency where the target sound is dominant (typically the frequency where the target sound volume exceeds the volume of the non-target sound), and the non-target sound frequency FB is a frequency where the non-target sound is dominant (typically Specifically, the non-target sound volume is higher than the target sound volume). The target sound spectrum QA1 is composed of components of the target sound frequency FA, and the non-target sound spectrum QB1 is composed of components of the non-target sound frequency FB. In order to select the target sound frequency FA and the non-target sound frequency FB, as illustrated below, a method using the difference between the direction D0 where the target sound arrives and the direction (DR, DL) where the non-target sound arrives (Patent Document 1) is preferably employed.

  FIG. 2 is a block diagram of the sound source separation unit 30. As shown in FIG. 2, the sound source separation unit 30 includes a signal processing unit 32, a frequency selection unit 34, and an intensity specifying unit 36. The signal processing unit 32 compares a plurality of frequency spectra (P0, PR, PL) in which the incoming sound from each of the plurality of directions (D0, DR, DL) is suppressed by comparing with the incoming sound from the other direction. And the frequency spectrum P2. FIG. 3 is a graph for explaining the contents of processing by the signal processing unit 32. The horizontal axis in FIG. 3 means the angle θ with the target sound direction D0 as the reference (0 °), and the vertical axis in FIG. 3 means the signal strength (power).

  As shown in FIG. 2, the signal processing unit 32 includes a first processing unit 321, a second processing unit 322, and a third processing unit 323. The first processing unit 321 generates the frequency spectrum P0 by subtracting the frequency spectrum P2 from the frequency spectrum P1. Since the target sound arriving from the direction D0 reaches the sound collecting device M1 and the sound collecting device M2 with substantially the same phase, the frequency spectrum P0 is the purpose arriving from the direction D0, as indicated by the symbol B0 (solid line) in FIG. This corresponds to a spectrum in which sound is suppressed with respect to incoming sound from another direction. In other words, the first processing unit 321 is a blind spot control type (null) beamformer that forms a blind spot on sound collection in the direction D0.

  The second processing unit 322 generates the frequency spectrum PR by subtracting the frequency spectrum D (P1) of the signal obtained by delaying the sound signal S1 by the delay amount D from the frequency spectrum P2. The delay amount D is set to the time difference between the time when the incoming sound from the direction DR reaches the sound collecting device M1 and the time when it reaches the sound collecting device M2. Therefore, the frequency spectrum PR corresponds to a spectrum in which the non-target sound arriving from the direction DR is suppressed with respect to the incoming sound from another direction, as indicated by a symbol BR (broken line) in FIG. In other words, the second processing unit 322 is a blind spot control type beam former that forms a blind spot on sound collection in the direction DR. Similarly, the third processing unit 323 subtracts the frequency spectrum D (P2) of the signal obtained by delaying the sound signal S2 by the delay amount D from the frequency spectrum P1, as indicated by reference sign BL in FIG. This is a blind spot control type beam former that generates a frequency spectrum PL in which non-target sounds from DL are suppressed.

  2 compares the intensities of the three types of frequency spectra (P0, PR, and PL) generated by the signal processing unit 32 for each of the K frequencies f1 to fK. Each of f1 to fK is sorted into a target sound frequency FA and a non-target sound frequency FB. As shown in FIG. 2, the frequency selection unit 34 includes a first comparison unit 341 and a second comparison unit 342.

  The first comparison unit 341 generates the frequency spectrum PLR by comparing the intensities at each of the K frequencies f1 to fK between the frequency spectrum PR and the frequency spectrum PL. The intensity of the frequency spectrum PLR at the frequency fk is set to the lower one of the intensity at the frequency fk of the frequency spectrum PR and the intensity at the frequency fk of the frequency spectrum PL. Since the frequency spectrum PR is a spectrum in which the non-target sound from the direction DR is suppressed, and the frequency spectrum PL is a spectrum in which the non-target sound from the direction DL is suppressed, the frequency spectrum PLR is the non-purpose of the direction DR and the direction DL. This corresponds to a spectrum in which sound is suppressed (that is, a spectrum in which the target sound from the direction D0 is emphasized).

  The second comparison unit 342 compares the intensities of the K frequencies f1 to fK between the frequency spectrum P0 and the frequency spectrum PLR. The frequency spectrum P0 is a spectrum that emphasizes the non-target sound, and the frequency spectrum PLR is a spectrum that emphasizes the target sound. Accordingly, the second comparison unit 342 selects, as the target sound frequency FA, the frequency fk in which the intensity of the frequency spectrum PLR exceeds the intensity of the frequency spectrum P0 among the K frequencies f1 to fK, and the K frequencies f1 to fK. The frequency fk in which the intensity of the frequency spectrum P0 exceeds the intensity of the frequency spectrum PLR is selected as the non-target sound frequency FB.

  The intensity specifying unit 36 in FIG. 2 generates the target sound spectrum QA1 and the non-target sound spectrum QB1 for each unit section using the result of selection by the frequency selecting unit 34. The target sound spectrum QA1 of the nth (n is a natural number) unit interval is a series of intensity XA1 (n, k) (XA1 (n, 1) to XA1) set for each frequency fk according to the intensity of the target sound. (n, K)), and the non-target sound spectrum QB1 of the nth unit interval is a series of intensity XB1 (n, k) set for each frequency fk according to the intensity of the non-target sound (XB1 ( n, 1) to XB1 (n, K)). The setting of the intensity XA1 (n, k) and the intensity XB1 (n, k) will be described in detail below.

  As shown in FIG. 3, the non-target sound is emphasized in the frequency spectrum P0 (symbol B0), and the target sound is emphasized in the frequency spectrum PLR. Therefore, the intensity specifying unit 36 uses the intensity XA1 (n, k) of each frequency fk selected for the target sound frequency FA in the target sound spectrum QA1 as the intensity (mainly the target sound) of the frequency spectrum PLR. Is set to a value obtained by subtracting the intensity at the frequency fk of the frequency spectrum P0 (mainly the intensity derived from the non-target sound) from the intensity derived from. Since the intensity XA1 (n, k) of each target sound frequency FA is calculated by subtracting the frequency spectrum P0 from the frequency spectrum PLR as described above (spectral subtraction), it exists in the target sound frequency FA of the frequency spectrum PLR. It is possible to generate the target sound spectrum QA1 in which the influence of the non-target sound is effectively reduced. However, a configuration in which the intensity of the frequency spectrum PLR in which the target sound is emphasized is set as the intensity XA1 (n, k) of the target sound spectrum QA1 is also suitable. The intensity XA1 (n, k) of each frequency fk selected as the non-target sound frequency FB in the target sound spectrum QA1 is set to zero.

  In addition, the intensity specifying unit 36 uses the non-target sound spectrum QB1 for the frequency Xk1 (n, k) at each frequency fk selected as the non-target sound frequency FB, and the frequency of the frequency spectrum P1 generated by the frequency analysis unit 20. Set to intensity at fk. It should be noted that the intensity XB1 (n, k) of the non-target sound spectrum QB1 at the non-target sound frequency FB is set to the intensity at the frequency fk of the frequency spectrum P2, or the intensity (mainly at the frequency fk of the frequency spectrum P0). A configuration in which the intensity at the frequency fk of the frequency spectrum PLR (mainly the intensity derived from the target sound) is subtracted from the intensity derived from the non-target sound is also adopted. The intensity XB1 (n, k) of each frequency fk selected as the target sound frequency FA in the non-target sound spectrum QB1 is set to zero.

  As can be understood from the above description, in this embodiment, the target sound frequency FA and the non-target sound frequency FB are obtained by utilizing the difference between the direction D0 of the target sound and the direction of the non-target sound (DR, DL). Selected. Therefore, even when the target sound and the non-target sound have similar acoustic characteristics, the mixed sound of the target sound and the non-target sound is separated into the target sound spectrum QA and the non-target sound spectrum QB with high accuracy. There is an advantage that you can.

  By the way, the component of the non-target sound frequency FB (non-target sound) includes the target sound in addition to the non-target stationary sound that is temporally steady (small change in acoustic characteristics such as volume and pitch). Non-target fluctuation sound coming from another direction is included. Non-target steady sounds are noises such as operating sounds of air-conditioning equipment and crowded noises in crowds, and non-target fluctuation sounds are voices whose utterances change in acoustic characteristics such as volume and pitch (utterances). Sound) and musical sounds. The noise estimation unit 42 in FIG. 1 generates a noise spectrum N (that is, a spectrum of a non-target stationary sound) for each unit section by suppressing the non-target fluctuation sound in the non-target sound spectrum QB1. The noise spectrum N of the nth unit section is a series of intensity (power) μ (n, 1) to μ (n, K) at each of the K frequencies f1 to fK.

  FIG. 4 is a flowchart of an operation in which the noise estimation unit 42 generates the noise spectrum N of the nth unit section. The process of FIG. 4 is sequentially executed for each unit section. When the processing of FIG. 4 is started, the noise estimation unit 42 initializes the variable k to 1 (step S1). The variable k is a number that specifies any of the K frequencies f1 to fK.

  The noise estimation unit 42 determines whether or not the frequency fk is the non-target sound frequency FB (step S2). When the frequency fk is the non-target sound frequency FB, the noise estimation unit 42 has the intensity XB1 (n, k) at the frequency fk (non-target sound frequency FB) in the non-target sound spectrum QB1 of the nth unit section. It is determined whether or not the threshold value XTH is exceeded (step S3).

The threshold value XTH is defined by the following formula (1), and the intensity μ (n− at the frequency fk of the noise spectrum N generated by the noise estimation unit 42 for the immediately preceding ((n−1) th) unit section. 1, k) multiplied by a coefficient τ. The coefficient τ is set to a predetermined value (for example, 2) exceeding 1. Therefore, the threshold value XTH is set to a numerical value (variable value corresponding to the intensity μ (n−1, k)) exceeding the intensity μ (n−1, k). For the first unit section, a predetermined initial value is applied as the intensity μ (n−1, k) in the equation (1).
XTH ＝ τ ・ μ (n-1, k) (1)

  Since the non-target fluctuation sound is more easily changed in intensity compared to the non-target steady sound, the intensity XB1 (n, k) of the frequency fk at which the non-target fluctuation sound is generated in the non-target sound spectrum QB1 varies with time. large. Therefore, the comparison between the intensity XB1 (n, k) and the threshold value XTH in step S3 corresponds to a process for determining whether or not a non-target fluctuation sound is generated at the frequency fk in the non-target sound spectrum QB1. That is, when the intensity XB1 (n, k) exceeds the threshold value XTH, the frequency fk component of the non-target sound spectrum QB1 is estimated to correspond to the non-target fluctuation sound, and the intensity XB1 (n, k) is lower than the threshold value XTH. It is estimated that the component of the frequency fk of the non-target sound spectrum QB1 does not correspond to the non-target fluctuation sound (corresponds to the non-target steady sound). Formula (1) is set so that the intensity XB1 (n, k) exceeds the threshold value XTH when the non-target fluctuation sound is generated and the intensity XB1 (n, k) is lower than the threshold value XTH when only the non-target stationary sound exists. The coefficient τ is selected statistically or experimentally.

When the intensity XB1 (n, k) of the non-target sound spectrum QB1 is lower than the threshold value XTH (that is, when no non-target fluctuation sound is generated at the frequency fk), the noise estimation unit 42 determines the nth unit interval. From the intensity XB1 (n, k) at the frequency fk of the non-target sound spectrum QB1 and the intensity μ (n−1, k) at the frequency fk of the noise spectrum N of the (n−1) th unit section, the nth The intensity μ (n, k) at the frequency fk of the th noise spectrum N is calculated (step S4). The intensity μ (n, k) is defined by, for example, the intensity XB1 (n, k) in the non-target sound spectrum QB1 of the nth unit section and the (n−1) th, as defined by the following formula (2). ) Calculated as a weighted sum (weighted average) with the intensity μ (n−1, k) in the noise spectrum N of the first unit section. The coefficient α in Expression (2) is set to a positive number (for example, 0.9) less than 1. As understood from the equation (2), as the coefficient α increases, the influence of the intensity XB1 (n, k) on the intensity μ (n, k) decreases (the intensity XB1 (n, k in each past unit interval). ) Will increase.
μ (n, k) = α ・ μ (n−1, k) + (1−α) × XB1 (n, k) (2)

On the other hand, when the intensity XB1 (n, k) of the non-target sound spectrum QB1 exceeds the threshold value XTH (S3: YES), the noise estimator 42 is the (n-1) -th as shown in Equation (3). The intensity μ (n−1, k) at the frequency fk of the noise spectrum N is set as the intensity μ (n, k) at the frequency fk (non-target sound frequency FB) of the nth noise spectrum N (step S5). . That is, when the intensity XB1 (n, k) exceeds the threshold value XTH (when the intensity XB1 (n, k) increases due to the occurrence of non-target fluctuation sound at the frequency fk), the intensity XB1 of the non-target sound spectrum QB1 (n, k) is not reflected in the intensity μ (n, k). Therefore, in the noise spectrum N, the non-target fluctuation sound in the non-target sound spectrum QB1 is suppressed (ideally removed).
μ (n, k) ＝ μ (n-1, k) (3)

  When the frequency fk is the target sound frequency FA (S2: NO), the noise estimation unit 42 calculates the intensity μ (n, k) of the (n−1) th noise spectrum N in the same manner as the equation (3). The intensity μ (n, k) at the frequency fk (target sound frequency FA) of the nth noise spectrum N is set (step S6).

  As can be understood from Equation (2) and Equation (3), the intensity μ (n, k) of the noise spectrum N in the nth unit interval is a plurality of past (before the (n−1) th) plurality. This is a numerical value that cumulatively reflects the intensity of the noise spectrum N calculated for the unit section. That is, the intensity μ (n, k) of the noise spectrum N is equal to the non-target sound spectrum QB1 over a plurality of unit intervals where the intensity XB1 (n, k) of the frequency fk selected as the non-target sound frequency FB is lower than the threshold value XTH. This is a numerical value obtained by smoothing (averaging) the intensity XB1 (n, k).

  As described above, when the intensity μ (n, k) is set in each step (S4, S5, S6), the noise estimation unit 42 determines whether or not the variable k has reached the predetermined value K (step S7). . If the variable k has not reached the predetermined value K, the noise estimation unit 42 adds 1 to the variable k (step S8), and then proceeds to step S2. That is, the intensity μ (n, k) is sequentially calculated for each of the K frequencies f1 to fK. When the variable k reaches the numerical value K (that is, when the calculation of the intensity μ (n, 1) to μ (n, K) is completed), the noise estimation unit 42 ends the process of FIG. 4 (S7: YES) ). A series of intensities μ (n, 1) to μ (n, K) for K frequencies f1 to fK corresponds to the noise spectrum N of the nth unit interval.

  The noise suppression unit 44 of FIG. 1 generates the target sound spectrum QA2 by subtracting the noise spectrum N generated by the noise estimation unit 42 from the target sound spectrum QA1 generated by the sound source separation unit 30 (spectral subtraction). Specifically, the noise suppression unit 44 uses the intensity XA1 (n, k) of the frequency fk in the target sound spectrum QA1 of the nth unit section to generate the intensity of the noise spectrum N generated for the unit section at the frequency fk. The target sound spectrum QA2 is generated by subtracting μ (n, k).

That is, the intensity XA2 (n, k) at the frequency fk of the target sound spectrum QA2 for the nth unit interval is expressed by the equation (4a). However, the intensity XA2 (n, k) of the frequency fk at which the right side (XA1 (n, k) −μ (n, k)) of Expression (4a) is lower than a predetermined value (for example, zero) is set to the predetermined value. . In addition, the target sound spectrum QA2 is expressed by Equation (4b). Symbol θa (n, k) in Equation (4b) is the phase at the frequency fk of the target sound spectrum QA1. As can be understood from the equations (4a) and (4b), the target sound spectrum QA2 is an acoustic sound in which the non-target stationary sound (noise spectrum N) is suppressed from the incoming sound from the direction D0 (target sound spectrum QA1). This corresponds to the spectrum of the target sound.
XA2 (n, k) = XA1 (n, k) -μ (n, k) (4a)
QA2 = {XA1 (n, k) −μ (n, k)} e ^{jθa (n, k)} (4b)

  As described above, the intensity XB1 (n, k) is not reflected in the intensity μ (n, k) of the noise spectrum N for the frequency fk where the intensity XB1 (n, k) of the non-target sound spectrum QB1 exceeds the threshold value XTH. As described in detail below, it is possible to generate a noise spectrum N obtained by extracting only non-target stationary sound with high accuracy even in an environment where both non-target stationary sound and non-target fluctuation sound exist.

  5 and 6 are time series (spectrogram) of the noise spectrum N of each unit section. FIG. 5 is a time series of the noise spectrum N in the first embodiment, and FIG. 6 is a time series of the noise spectrum N in the contrast 1 with the first embodiment. In contrast 1, the intensity μ (n, k) of the noise spectrum N is calculated by the equation (2) regardless of the intensity XB1 (n, k) of the non-target sound frequency FB (ie, step S3 in FIG. 4). And step S5 are omitted).

  5 and 6, a straight line in which the frequency fk (peak frequency) having a high intensity in the noise spectrum N is connected along the time axis is illustrated. In the examples of FIGS. 5 and 6, the non-target stationary sound that does not change with time exists on the low frequency side of the noise spectrum N (non-target sound spectrum QB1). 5 and 6 show the time when the non-target fluctuation sound is generated.

  In contrast 1, the intensity μ (n, k) of the noise spectrum N is expressed by the formula (2) regardless of the intensity XB1 (n, k) of the non-target sound spectrum QB1 (that is, regardless of the presence or absence of non-target fluctuation sound). ). Therefore, the noise spectrum N includes both non-target stationary sounds and non-target fluctuation sounds. The past intensity (μ (n-1, k), μ (n-2, k), ...) is cumulatively reflected in the intensity μ (n, k) calculated by Equation (2). Therefore, the intensity μ (n, k) of the frequency fk at which the non-target fluctuation sound is generated at a specific time in the noise spectrum N is, as shown in FIG. 6, even when the non-target fluctuation sound is stopped. , Maintained at a high value over a plurality of subsequent unit intervals. Therefore, the intensity XA1 (n, k) of the target sound spectrum QA1 at the frequency fk where the target sound fluctuation sound is generated is excessively reduced by the processing by the noise suppressing unit 44, which may cause annoying musical noise.

  In contrast to contrast 1, in the first embodiment, the intensity μ (n, k) of the frequency fk at which the intensity XB1 (n, k) exceeds the threshold value XTH has the intensity XB1 (n, k) (that is, the frequency As shown in FIG. 5, a noise spectrum N in which the non-target fluctuation sound is suppressed is generated. Accordingly, there is an advantage that the intensity of the frequency fk at which the non-target fluctuation sound is generated in the target sound spectrum QA1 is prevented from being excessively reduced, and the generation of musical noise is suppressed. Since the non-target fluctuation sound is suppressed in the noise spectrum N, the effect of reducing the non-target fluctuation sound from the target sound spectrum QA1 by the processing by the noise suppression unit 44 is small. However, since the non-target fluctuation sound arriving from the direction DR or the direction DL is excluded from the target sound spectrum QA1 by the selection by the sound source separation section 30, the noise suppression section 44 does not reduce the non-target fluctuation sound. In addition, it is possible to generate a reproduced sound in which both the non-target steady sound and the non-target fluctuation sound are suppressed with high accuracy.

  The intensity specifying unit 36 of the first embodiment generates the target sound spectrum QA1 by subtracting the frequency spectrum P0 in which the non-target sound is emphasized from the frequency spectrum PLR in which the target sound is emphasized. That is, the non-target sound can be suppressed only by the processing by the intensity specifying unit 36. However, for example, when the non-target stationary sound is included in the incoming sound from the direction D0, the non-target stationary sound is not sufficiently suppressed even if the frequency spectrum P0 is subtracted from the frequency spectrum PLR. According to the first embodiment in which the noise spectrum N of the non-target stationary sound is subtracted from the target sound spectrum QA1, the configuration in which the non-target sound is suppressed only by the processing by the intensity specifying unit 36 (that is, the configuration in which the noise suppression unit 44 is omitted). ) Has an advantage that non-target stationary sound is effectively suppressed.

  The intensity adjusting unit 52 in FIG. 1 uses the common X-values XB1 (n, 1) to XB1 (n, K) of the non-target sound spectrum QB1 generated for each unit section by the sound source separation unit 30 (hereinafter referred to as “suppression coefficient”). The non-target sound spectrum QB2 is generated by performing suppression according to p. The non-target sound spectrum QB2 is, for example, an intensity XB2 (n, k) (XB2 (n, k) = p · XB1 (n, k) corresponding to a multiplication value of each intensity XB1 (n, k) and the suppression coefficient p. )) Series. The suppression coefficient p is set to a positive number less than 1. Therefore, the smaller the suppression coefficient p, the greater the degree of suppression of the intensities XB1 (n, 1) to XB1 (n, K). As described above, since the intensity adjustment unit 52 suppresses the entire non-target sound without distinguishing between the non-target stationary sound and the non-target fluctuation sound, the non-target sound spectrum QB2 processed by the intensity adjustment unit 52 is Includes non-target stationary sounds and non-target fluctuation sounds. The fluctuation sound suppression unit 60 of FIG. 1 generates the non-target sound spectrum QB3 by suppressing the non-target fluctuation sound in the non-target sound spectrum QB2.

  FIG. 7 is a block diagram of the fluctuation sound suppression unit 60. As shown in FIG. 7, the fluctuation sound suppression unit 60 includes a gain setting unit 62 and a suppression processing unit 64. The gain setting unit 62 generates a suppression gain sequence H for suppressing non-target fluctuation sound in the non-target sound spectrum QB2 for each unit section. The suppression gain sequence H generated for the nth unit interval is a sequence of K gains (spectral gains) h (n, 1) to h (n, K) corresponding to the frequencies f1 to fK. Each of the gains h (n, 1) to h (n, K) is individually set for each frequency fk within a positive number range less than 1.

The suppression processing unit 64 uses the intensities XB2 (n, 1) to XB2 (n, K) of the non-target sound spectrum QB2 according to the gains h (n, 1) to h (n, K) of the suppression gain sequence H. To adjust the non-target sound spectrum QB3 in which the non-target fluctuation sound of the non-target sound spectrum QB2 is suppressed. The non-target sound spectrum QB3 is a sequence (XB3 (n, 1) to XB3 (n, K)) of intensity XB3 (n, k) set for each frequency fk. The suppression processing unit 64 generates the non-target sound spectrum QB3 by multiplying the non-target sound spectrum QB2 by the suppression gain sequence H. That is, the intensity XB3 (n, k) at the frequency fk in the non-target sound spectrum QB3 is defined by the following formula (5), and the intensity XB2 (n, k) of the non-target sound spectrum QB2 and the suppression gain. This corresponds to a multiplication value of the gain h (n, k) corresponding to the frequency f in the series H.
XB3 (n, k) = h (n, k) ・ XB2 (n, k) (5)

  The gains h (n, 1) to f (n, K) of the suppression gain sequence H are individually set for each frequency fk so that the non-target fluctuation sound is suppressed in the non-target sound spectrum QB3 of the nth unit section. Selected. That is, in the non-target sound spectrum QB2, the gain h (n, k) corresponding to the frequency fk that is likely to be a non-target fluctuation sound is set to a smaller numerical value.

  As shown in FIG. 7, the gain setting unit 62 includes a first processing unit 621 and a second processing unit 622. The first processing unit 621 generates an enhancement gain series G for each unit section. The enhancement gain sequence G generated for the nth unit interval is a sequence of K gains (spectral gains) g (n, 1) to g (n, K) corresponding to the frequencies f1 to fK. The gains g (n, 1) to g (n, K) are such that when the non-target sound spectrum QB2 is multiplied, the non-target fluctuation sound (typically speech) of the non-target sound spectrum QB2 is emphasized. It is set individually for each frequency fk within a positive number range less than 1. That is, in the non-target sound spectrum QB2, the gain g (n, k) corresponding to the frequency fk that is highly likely to be a non-target fluctuation sound is set to a larger value. For generation of the enhancement gain series G, the noise spectrum N generated by the noise estimation unit 42 and the non-target sound spectrum QB2 after adjustment by the intensity adjustment unit 52 are used.

The second processing unit 622 generates a suppression gain sequence H from the enhancement gain sequence G generated by the first processing unit 621. The suppression gain sequence H is generated so as to have an inverse characteristic of the enhancement gain sequence G that emphasizes the non-objective fluctuation sound (that is, a characteristic that suppresses the non-objective fluctuation sound). For example, the second processing unit 622 subtracts the gain g (n, k) of the frequency fk of the enhancement gain sequence G from a predetermined value (1 in this embodiment) as shown in Equation (6), thereby suppressing the suppression gain sequence. The gain h (n, k) of the frequency fk of H is calculated.
h (n, k) = 1-g (n, k) (6)

  A known technique is arbitrarily employed to generate the enhancement gain series G by the first processing unit 621. For example, as illustrated below, Y. Ephraim and D. Malah, “Speech enhancement using a minimum mean-square” error short-time spectral amplitude estimator ", IEEE ASSP, vol.ASSP-32, no.6, p.1109-1121, Dec. 1984, MMSE-STSA method, T. Lotter and P. Vary," Speech enhancement by MAP spectral amplitude estimation using a Super-Gaussian speech model ", EURASIP Journal on Applied Signal Processing, vol.2005, no, 7, p.1110-1126, July 2005 MAP (maximum a posteriori estimation) Estimation is preferably employed.

The non-target sound spectrum QB2 of the n-th unit section is expressed by the following expression (7): non-target fluctuation sound (A (n, k) e ^{jα (n, k)} ) and non-target stationary sound ( B (n, k) e ^{jβ (n, k)} ).
XB2 (n, k) ej [ ^{theta] b (n, k)} = A (n, k) ej [ ^{alpha] (n, k)} + B (n, k) ej [ ^{beta] (n, k)} (7)
In equation (7), θb (n, k) is the phase of the non-target sound spectrum QB2 at the frequency fk. A (n, k) in Expression (7) is the amplitude of the component of the frequency fk in the non-target fluctuation sound in the nth unit section, and α (n, k) is the phase of the component. In addition, B (n, k) in Expression (7) is the amplitude of the component of the frequency fk in the non-target stationary sound in the nth unit section, and β (n, k) is the phase of the component. .

The posterior SN ratio (posteriori SNR) γ (n, k) and the prior SN ratio (priori SNR) ξ (n, k) are the intensity (power) μ (n, k) of the noise spectrum N generated by the noise estimation unit 42. Is expressed by the following equations (8a) and (8b). The function value F [x] of Expression (8b) is set to the variable x when the variable x is a positive number, and is set to zero when the variable x is zero or a negative number. In addition, the coefficient α _SNR in Expression (8b) is a predetermined positive number less than 1.
γ (n, k) = XB2 (n, k) ² / μ (n, k) (8a)
ξ (n, k) = A (n, k) ² / μ (n, k)
= Α _SNR · A ² (n−1, k) / μ (n, k) + (1−α _SNR ) · F [γ (n, k) −1] (8b)

The gain g (n, k) of the emphasized gain series G is obtained by using, for example, the a posteriori SN ratio γ (n, k) in Expression (8a) and the prior SN ratio ξ (n, k) in Expression (8b). It is expressed by the following formula (9) to formula (13). The first processing unit 621 calculates the gains g (n, 1) to g (n, K) of the enhancement gain series G for each unit section by executing any one of the equations (9) to (13). Calculate sequentially.

Expressions (10) and (11) are approximate expressions of Expression (9). However, Equation (11) is established when the prior SN ratio ξ (n, k) is sufficiently smaller than 1. The function F1 (−0.5,1, −ν) of the formulas (9) and (10) is defined by the following formulas. The function I0 means a zero-order modified Bessel function, and the function I1 means a first-order modified Bessel function. In addition, the coefficient q in Expression (10) and Expression (11) is set to a predetermined positive number (for example, 0.20).

The following equation (12) is an arithmetic expression of gain g (n, k) using MAP estimation, and equation (13) is a gain g (n, k) using a Wiener filter. This is an arithmetic expression. Note that the coefficient φ and the coefficient τ in Expression (12) are constants (for example, φ = 0.126, τ = 1,74 or 3.0) that define the shape of the probability distribution of the non-target sound spectrum QB2.

  The synthesizing unit 54 in FIG. 1 synthesizes the non-target sound spectrum QB3 in which the non-target fluctuation sound is suppressed and the target sound spectrum QA2 generated by the noise suppressing unit 44 by the above procedure, thereby outputting the output spectrum R for each unit section. Is generated. The output spectrum R includes the intensity XA2 (n, k) of each frequency fk selected as the target sound frequency FA in the target sound spectrum QA2 and each frequency fk selected as the non-target sound frequency FB out of the non-target sound spectrum QB3. The intensity XB3 (n, k) is arranged along the frequency axis. As described above, since the target sound spectrum QA2 and the non-target sound spectrum QB3 are synthesized, compared with a configuration in which a reproduced sound is generated only from the target sound spectrum QA2, a natural reproduced sound is reproduced in terms of audibility. There is an advantage.

  The inverse conversion unit 56 converts the output spectrum R of each unit section into a time domain signal by inverse FFT processing, and generates a sound signal SOUT by connecting the converted signals of each unit section to each other on the time axis. To do. By supplying the sound signal SOUT to the sound emitting device (not shown), the reproduction sound in which the non-target sound is suppressed and the target sound is emphasized is emitted.

  Now, a configuration in which the non-target sound spectrum QB2 adjusted by the intensity adjusting unit 52 is combined with the target sound spectrum QA2 by the combining unit 54 (that is, a configuration in which the fluctuation sound suppressing unit 60 is omitted) is a pair with the first embodiment. Assumed as proportional 2. In contrast 2 as well, if the suppression coefficient p is set to a small value (for example, 0.01), it is possible to sufficiently suppress both the non-target steady sound and the non-target fluctuation sound. However, if the non-target sound is excessively suppressed, there is a problem that the intensity change on the frequency axis and the time axis in the reproduced sound becomes excessive and musical noise is generated in the reproduced sound. Especially in an environment where misselection between the target sound frequency FA and the non-target sound frequency FB is likely to occur (for example, an environment where the S / N ratio is low), the frequency fk where the target sound is actually dominant is erroneously set as the non-target sound frequency FB. The state that is selected and suppressed by the intensity adjustment unit 52 and the state that is appropriately selected and not suppressed by the target sound frequency FA are alternately switched in a short time, which is caused by a change in intensity on the time axis. Musical noise is particularly easily manifested. Therefore, from the viewpoint of achieving both suppression of non-target sound and reduction of musical noise, a value of about 0.5 is preferable as the suppression coefficient p.

  However, when the suppression coefficient p is set to a large value (for example, 0.5) in the proportionality 2, the degree of suppression of the non-target sound is reduced. Therefore, there is a problem that the non-target sound is reproduced while maintaining a sufficient intensity, and it becomes difficult to listen to the target sound. However, the non-target stationary sound often has an acoustic characteristic different from that of the target sound (typically speech), and is suppressed from the target sound spectrum QA1 by the noise suppression unit 44. Even if it is mixed with the reproduced sound under the eye suppression coefficient p, there is no serious problem. On the other hand, the non-target fluctuation sound (typically voice) often has similar acoustic characteristics to the target sound and is not suppressed by the processing by the noise suppression unit 44. Therefore, if the non-target fluctuation sound is reproduced with sufficient intensity maintained under the high suppression coefficient p, the problem that it becomes difficult to listen to the target sound becomes particularly serious.

  In the first embodiment, since the non-target fluctuation sound is suppressed from the non-target sound spectrum QB2 processed by the intensity adjustment unit 52 and then synthesized with the target sound spectrum QA2, a suppression coefficient is used to reduce musical noise. Even when p is set to a high value, it is possible to generate a reproduced sound in which the target sound can be easily heard while suppressing the non-target fluctuation sound. That is, according to the first embodiment, it is possible to achieve both reduction of musical noise and suppression of non-target fluctuation sound.

  Note that the non-target fluctuation sound is suppressed in the noise spectrum N as well as the non-target sound spectrum QB3. Therefore, even in the configuration in which the noise spectrum N is applied to the synthesis of the target sound spectrum QA2 in the synthesizer 54 instead of the non-target sound spectrum QB3, the above-described effect of achieving both the reduction of the musical noise and the suppression of the non-target fluctuation sound. Is realized. However, the noise spectrum N cumulatively reflects the intensity XB1 (n, k) of the non-target sound spectrum QB1 over a plurality of past unit intervals (smoothed in the direction of the time axis). The reproduced sound obtained by synthesizing N and the target sound spectrum QA2 is unnatural in terms of hearing. On the other hand, in the first embodiment, since the non-target sound spectrum QB3 generated independently for each unit section is applied to the synthesis with the target sound spectrum QA2, it is possible to generate a reproduced sound with a natural impression on hearing. There are advantages.

  In order to specify the suppression gain sequence H that can suppress the non-target fluctuation sound with high accuracy, the noise spectrum N (intensity μ (n, k)) obtained by accurately estimating the non-target stationary sound is used as the gain g of the enhancement gain sequence G. It is important to use it for the calculation of (n, k) (formulas (9) to (13)). That is, if non-target fluctuation sound remains in the noise spectrum N, the accuracy of estimation of the suppression gain sequence H is lowered. In the first embodiment, as illustrated in FIG. 5, the noise spectrum N is generated by removing the non-target fluctuation sound with high accuracy. Therefore, an appropriate suppression gain sequence H that can suppress the non-target fluctuation sound with high accuracy is generated. It can be generated by the gain setting unit 62.

<B: Second Embodiment>
Next, a second embodiment of the present invention will be described. In the following aspects, elements having the same functions and functions as those of the first embodiment are denoted by the same reference numerals as above, and detailed descriptions thereof are omitted as appropriate.

The second processing unit 622 of the gain setting unit 62 in the second embodiment executes the following mathematical formula (6a) instead of the mathematical formula (6) of the first embodiment, so that the enhancement generated by the first processing unit 621 is performed. Each gain h (n, k) of the suppression gain sequence H is generated from each gain g (n, k) of the gain sequence G. That is, the second processing unit 622 subtracts the multiplication value of the gain g (n, k) of the enhancement gain series G and the adjustment value w (n, k) from a predetermined value (1 in this embodiment), thereby suppressing the suppression. The gain h (n, k) of the gain series H is calculated.
h (n, k) = 1-w (n, k) · g (n, k) (6a)

  The adjustment value w (n, k) (w (n, 1) to w (n, K)) is set for each unit interval for each of the K frequencies f1 to fK. As understood from the mathematical formulas (6a) and (5) (calculation by the suppression processing unit 64), the larger the adjustment value w (n, k) (the smaller the gain h (n, k)), the non-purpose. The degree to which the non-objective fluctuation sound in the sound spectrum QB2 is suppressed is large (the degree of suppression of the non-objective fluctuation sound is small as the adjustment value w (n, k) is small).

  If the gain h (n, k) is set to an excessively small value in an environment where there are many non-target sounds (particularly non-target steady sounds), the non-target is caused by excessive suppression of the non-target fluctuation sound in the suppression processing unit 64. Noise may occur in the sound spectrum QB3. Therefore, the second processing unit 622 variably sets the adjustment value w (n, k) according to the state (intensity) of the non-target sound. Specifically, the second processing unit 622 sets the adjustment value w (n, k) to a smaller numerical value as the intensity μ (n, k) of the noise spectrum N is larger. Therefore, the gain h (n, k) is set to a larger value as the intensity μ (n, k) is larger (that is, suppression of non-objective fluctuation sound is alleviated). On the other hand, the degree of suppression of non-target fluctuation sound increases as the intensity μ (n, k) of the noise spectrum N is smaller.

  In the above embodiment, since the gain h (n, k) of the suppression gain sequence H is adjusted according to the adjustment value w (nk), the degree of suppression of non-target fluctuation sound by the suppression processing unit 64 is adjusted. It is possible to set appropriately according to (nk). In particular, since the adjustment value w (nk) is variably set according to the state of the non-target sound, the gain h (n, k) of the suppression gain sequence H is set to an appropriate value according to the state of the non-target sound. There is an advantage that it can be set. For example, in an environment where there are many non-target sounds (particularly non-target steady sounds), it is possible to reduce noise caused by excessive suppression of non-target fluctuation sounds.

<C: Modification>
Various modifications can be made to the embodiments exemplified above. Specific modifications are exemplified below. Two or more aspects may be arbitrarily selected from the following examples and combined.

(1) Modification 1
Even if the target sound spectrum QA1 separated by the sound source separation unit 30 is directly supplied to the synthesis unit 54 (that is, not via the noise suppression unit 44), the non-target variation sound in the variation sound suppression unit 60 is provided. As long as the non-target sound spectrum QB3 after suppression is synthesized with the target sound spectrum QA1, the desired effect of generating natural sound that allows easy listening of the target sound is realized. Therefore, the noise suppression unit 44 in each of the above forms can be omitted. Note that the sound processing apparatus 100 of each of the above forms a configuration in which the intensity specifying unit 36 generates the target sound spectrum QA1 by subtracting the frequency spectrum P0 from the frequency spectrum PLR (the non-target sound is generated by the processing by the intensity specifying unit 36). And a configuration for reducing the intensity XB1 (n, k) of the non-target sound spectrum QB2 by the intensity adjustment unit 52, so that even if the noise suppression unit 44 is omitted, the non-purpose It is possible to generate a reproduced sound in which a stationary sound is suppressed.

(2) Modification 2
Even if the non-target sound spectrum QB1 after separation by the sound source separation unit 30 is directly supplied to the fluctuation sound suppression unit 60 (that is, not via the intensity adjustment unit 52), the non-target sound spectrum QB1 is not detected in the fluctuation sound suppression unit 60. As long as the non-target sound spectrum QB3 after suppression of the target fluctuation sound is synthesized with the target sound spectrum QA1, the desired effect of generating natural sound that is easy to listen to the target sound is realized. Therefore, the intensity adjusting unit 52 in each of the above forms can be omitted. Further, the intensity adjustment unit 52 and the fluctuation sound suppression unit 60 are replaced (that is, the intensity adjustment unit 52 suppresses the suppression coefficient to each intensity XB3 (n, k) of the non-target sound spectrum QB3 generated by the fluctuation sound suppression unit 60. A configuration in which p is multiplied is also employed.

(3) Modification 3
The method by which the noise estimation unit 42 estimates the noise spectrum N is arbitrary. For example, weighted noise estimation and minimum statistics are also preferably used for estimating the noise spectrum N. For weighted noise estimation, Masanori Kato, Akihiko Sugiyama, Masahiro Serizawa, "High-quality noise suppression based on weighted noise estimation and MMSE STSA method", IEICE (A), vol.J87-A, no.7, p. 851-860, July 2004, and minimum statistics are disclosed in R. Martin, “Spectral subtraction based on minimum statistics,” in Proc. Eur. Signal Processing Conf., 1994, p. 1182-1185 Has been. As described above, the non-target sound spectrum QB1 is not an essential element for estimating the noise spectrum N. For example, a configuration in which the noise spectrum N is estimated from the sound signal S1 and the sound signal S2 within a period in which the target sound is stopped (that is, a period in which only the non-target sound exists) is also suitable.

(4) Modification 4
The method by which the gain setting unit 62 sets the suppression gain sequence H (h (n, 1) to h (n, K)) is arbitrary. For example, in each of the above embodiments, the intensity XB2 (n, k) of the non-target sound spectrum QB2 is used to generate the suppression gain sequence H. However, the target sound spectrum processed by the non-target sound spectrum QB2 and the noise suppression unit 44 is used. Applying the spectrum synthesized with QA2 (or the target sound spectrum QA1 after separation by the sound source separation unit 30) to the left side of the equation (7) ( ^{XB2 (n, k)} e ^{jθb (n, k)} ), the suppression gain A configuration for generating the series H (enhanced gain series G) is also suitable. According to the above configuration, it is possible to generate the suppression gain sequence H that can suppress the non-target fluctuation sound with higher accuracy than when only the non-target sound spectrum QB2 is used.

  In each of the above embodiments, the suppression gain sequence H is generated from the enhancement gain sequence G. However, a configuration in which the suppression gain sequence H is generated directly (that is, without using the enhancement gain sequence G) is also employed. However, the specification of the suppression gain series H is merely an example of a method for suppressing the non-target fluctuation sound, and is not an essential element of the present invention. For example, when the frequency of the non-target fluctuation sound is known, a filter that suppresses the component of the frequency is employed as the fluctuation sound suppression unit 60. In other words, the fluctuation sound suppression unit 60 is included as an element for suppressing the intensity of the non-target fluctuation sound in the non-target sound spectrum QB2, and whether or not the suppression gain sequence H is generated and how the non-target fluctuation sound is suppressed. Is unquestionable.

(5) Modification 5
In the second embodiment, the adjustment value w (n, k) is individually set for each of the K frequencies f1 to fK. However, a configuration in which a common adjustment value w is applied to a plurality of frequencies fk is also employed. . For example, the adjustment value w is individually set for each of a plurality of bands defined on the frequency axis. The adjustment value w of one band is variably set according to the average value or total value (that is, the state of the non-target sound in the band) of the intensity μ (n, k) at each frequency fk in the band. The The gain setting unit 62 (second processing unit 622) calculates the gain h (n, k) of each frequency fk belonging to one band from the common adjustment value w set for the band, using Expression (6a). . A configuration in which a common adjustment value w is applied to the calculation of the gains h (n, 1) to h (n, K) of the K frequencies f1 to fK is also adopted. Each of the above configurations has an advantage that the load for calculating the adjustment value w is reduced as compared with the configuration for calculating the adjustment value w (n, k) for each frequency fk.

  In the second embodiment, the gain h (n, k) is variably controlled according to the intensity μ (n, k) of the noise spectrum N. However, the SN ratio of each frequency fk is also adjusted by the adjustment value w (n, k). It is used for control. The SN ratio of the frequency fk is, for example, the intensity XA1 (n, k) of the target sound spectrum QA1 with respect to the intensity μ (n, k) of the noise spectrum N (or the intensity XB1 (n, k) of the non-target sound spectrum QB1). Calculated as a relative ratio. The second processing unit 622 sets the adjustment value w (n, k) to a smaller numerical value as the SN ratio is lower. Even when the S / N ratio is used to control the adjustment value w, the adjustment value w is calculated for each band according to the S / N ratio of each frequency fk, and the entire band is set according to each S / N ratio of the frequencies f1 to fK. A configuration for calculating the common adjustment value w is employed.

  Further, the relationship between the state of the non-target sound (intensity μ (n, k) and SN ratio) and the increase / decrease of the adjustment value w is not limited to the above examples. For example, depending on the characteristics of the non-target sound, a configuration may be adopted in which the adjustment value w (n, k) is set to a larger value as the intensity μ (n, k) is larger (as the SN ratio is lower). However, a configuration in which the adjustment value w is a variable value is not essential in the present invention, and a configuration in which the adjustment value w is fixed to a predetermined value is also employed. Also, a configuration that uses the state of the non-target sound (intensity μ (n, k) or SN ratio) for controlling the adjustment value w is not essential. For example, the adjustment value w can be set according to the sound quality required for the reproduced sound.

Note that the content of the calculation for adjusting the suppression gain sequence H (gain h (n, k)) using the adjustment value w (n, k) is not limited to Equation (6a). For example, a configuration in which the second processing unit 622 executes the following expression (6b) instead of the expression (6a) is also preferable. That is, the second processing unit 622 multiplies the difference value between the predetermined value (1 in this embodiment) and the gain g (n, k) of the enhancement gain series G by the variable adjustment value w (n, k). Then, the gain h (n, k) of the suppression gain series H is calculated.
h (n, k) = w (n, k). {1-g (n, k)} (6b)
When Equation (6b) is used, contrary to the case where Equation (6a) is used, gain h (n, k) increases as adjustment value w (n, k) increases (that is, non-target fluctuation sound). The degree of repression will decrease). The second processing unit 622 sets the adjustment value w (n, k) to a larger value as the intensity μ (n, k) of the noise spectrum N is larger. Therefore, as in the second embodiment, the gain h (n, k) is set to a larger value as the intensity μ (n, k) is larger. In addition, a configuration is adopted in which the adjustment value w (n, k) of Equation (6b) is set to a larger value as the SN ratio of each frequency fk is lower. Of course, the relationship between the adjustment value w (n, k) of Equation (6b) and the state of the non-target sound (intensity μ (n, k) and SN ratio) is arbitrarily selected. As described above, the calculations of the formulas (6a) and (6b) generate the suppression gain series H (gain h (n, k)) according to the enhancement gain series G and the adjustment value w (n, k). It is included as a process.

(6) Modification 6
In the first embodiment, when the intensity XB1 (n, k) of the non-target sound spectrum QB1 exceeds the threshold value XTH, the intensity μ (n-1, k) of the past noise spectrum N is changed to the nth noise spectrum. The intensity of N was set to μ (n, k). According to the above configuration, the influence of the non-objective fluctuation sound can be removed from the noise spectrum N, while the noise XB1 (n, k) exceeding the threshold value XTH is newly generated during the operation of the sound processing apparatus 100 and continues. The target stationary sound (hereinafter, particularly referred to as “new stationary sound”) is also removed from the noise spectrum N. Therefore, there is a possibility that the suppression of the new stationary sound is insufficient. The modification 6 is a structure which eliminates the above problem.

In the modification 6, the process of step S5 of FIG. 4 is different from that of the first embodiment. When the intensity XB1 (n, k) of the non-target sound spectrum QB1 exceeds the threshold value XTH (that is, when a non-target fluctuation sound or a new stationary sound is generated), the noise estimation unit 42 calculates the mathematical expression (3 ), The following equation (3a) is executed. That is, the noise estimator 42 multiplies the intensity μ (n−1, k) of the (n−1) th noise spectrum N by the coefficient β, and the intensity μ (n) of the nth noise spectrum N. , k) (step S5).
μ (n, k) ＝ β ・ μ (n-1, k) …… (3a)

  The coefficient β is set to a predetermined value (for example, 1.01) exceeding 1. Therefore, in a plurality of unit sections in which the intensity XB1 (n, k) exceeds the threshold value XTH, the intensity μ (n, k) of the noise spectrum N increases with time, and the non-target sound (non-target fluctuation sound) Or approach the intensity of a new stationary sound). The intensity μ (n, k) approaches the intensity of the non-target steady sound more rapidly as the coefficient β increases.

  In the above embodiment, since the intensity μ (n, k) of the noise spectrum N approaches the intensity of the new stationary sound over time, the noise spectrum N reflecting the characteristics of the new stationary sound is generated. Therefore, it is possible to effectively suppress non-target sounds including new stationary sounds from the target sound spectrum QA.

  Note that the intensity μ (n, k) of the noise spectrum N increases with time according to the calculation of Equation (3a) not only when a new stationary sound is generated but also when a non-target fluctuation sound is generated. That is, in the modified example 6, the occurrence of the non-target fluctuation sound is reflected in the intensity μ (n, k) of the noise spectrum N. However, since the non-target fluctuation sound is likely to change with time, the possibility of being maintained at a high intensity for a long time is sufficiently low as compared with the new stationary sound. That is, even when a non-target fluctuation sound is generated, before the intensity μ (n, k) of the noise spectrum N is sufficiently close to the non-target fluctuation sound, the non-target fluctuation sound becomes an intensity below the threshold value XTH. Decreasing (the expression (2) is applied to the calculation of the intensity μ (n, k)) suppresses the increase of the intensity μ (n, k). Therefore, although the intensity μ (n, k) of the noise spectrum N increases with time when the intensity XB1 (n, k) exceeds the threshold value XTH, the intensity μ (when the non-target fluctuation sound is generated. The rise in n, k) is small enough. That is, according to the modified example 6, there is an advantage that the noise spectrum N reflecting the new stationary sound can be generated while sufficiently suppressing the influence of the non-target fluctuation sound.

(7) Modification 7
The method of selecting the K frequencies f1 to fK into the target sound frequency FA and the non-target sound frequency FB is appropriately changed. Specifically, the technique (SAFIA) disclosed in Non-Patent Document 1 and Japanese Patent Laid-Open No. 10-313497 is used for selecting the target sound frequency FA and the non-target sound frequency FB. For example, it is assumed that the sound collection device M1 is closer to the target sound source than the sound collection device M2, and the sound collection device M2 is closer to the non-target sound source than the sound collection device M1. The sound source separation unit 30 compares the intensities of the K frequencies f1 to fK between the frequency spectrum P1 and the frequency spectrum P2, and selects the frequency fk having a high intensity of the frequency spectrum P1 as the target sound frequency FA. A frequency fk having a high intensity of the frequency spectrum P2 is selected as a non-target sound frequency FB. According to the above configuration, since the signal processing unit 32 of FIG. 2 is not required, there is an advantage that the processing and configuration of the sound processing device 100 are simplified.

  Instead of the blind spot control type beamformer, the following configuration in which a delay addition type beamformer is adopted in the signal processing unit 32 (first processing unit 321, second processing unit 322, third processing unit 323) is also suitable. . The first processing unit 321 generates the frequency spectrum P0 in which the target sound in the direction D0 is emphasized by adding the frequency spectrum P1 and the frequency spectrum P2. The second processing unit 322 generates the frequency spectrum PR in which the non-target sound in the direction DR is emphasized by adding the frequency spectrum P2 and the frequency spectrum P1 to which the delay amount D is added. Similarly, the third processing unit 323 generates a frequency spectrum PL in which the non-target sound in the direction DL is emphasized. The first comparison unit 341 sets the intensity at the frequency fk of the frequency spectrum PLR to the higher intensity of the intensity at the frequency fk of the frequency spectrum PR and the intensity at the frequency fk of the frequency spectrum PL. Therefore, the frequency spectrum PLR is a spectrum in which the non-target sound in the direction DR and the direction DL is emphasized. Then, the second comparison unit 342 selects, as the non-target sound frequency FB, the frequency of which the intensity of the frequency spectrum PLR exceeds the intensity of the frequency spectrum P0 among the K frequencies, and the frequency spectrum P0 of the K frequencies. A frequency whose intensity exceeds the intensity of the frequency spectrum PLR is selected as the target sound frequency FA.

  A configuration in which the signal processing unit 32 processes the sound signal S1 and the sound signal S2 in the time domain is also employed. That is, the signal processing unit 32 subtracts the sound signal S2 from the sound signal S1, the signal SR obtained by subtracting the sound signal S1 provided with the delay amount D from the sound signal S2, and the sound signal provided with the delay amount D. A signal SL obtained by subtracting S2 from the sound signal S1 is generated. The frequency analysis unit 20 arranged at the subsequent stage of the signal processing unit 32 converts the signal S0 into the frequency spectrum P0, converts the signal SR into the frequency spectrum PR, and converts the signal SL into the frequency spectrum PL.

(8) Modification 8
The method of calculating the intensity μ (n, k) when the intensity XB1 (n, k) of the non-target sound spectrum QB1 is lower than the threshold value XTH (S3: NO) is not limited to the formula (2). For example, the average (moving average) of the intensities XB1 (n, k) over a predetermined number of unit sections including the nth unit section is calculated as the intensity μ (n, k). That is, the number of non-target sound spectra QB1 (number of unit sections) used for calculating the intensity μ (n, k) is arbitrarily changed.

  In the second embodiment, when the intensity XB1 (n, k) exceeds the threshold value XTH (S3: YES), the method for calculating the intensity μ (n, k) is the past intensity μ (n−1, k). ) And the coefficient β (formula (3a)). For example, a configuration in which an addition value of the past intensity μ (n−1, k) and a predetermined positive number is calculated as the intensity μ (n, k) is also employed. That is, when the intensity XB1 (n, k) exceeds the threshold value XTH, a configuration in which a numerical value exceeding the intensity μ (n−1, k) of the past noise spectrum N is set as the intensity μ (n, k) is preferable. is there.

(9) Modification 9
In each of the above embodiments, the non-target sound spectrum QB3 (QB1, QB2) is generated for each unit section. However, considering the tendency that the non-target sound spectrum QB3 composed of non-target steady sounds has little temporal change, a configuration in which one non-target sound spectrum QB3 is generated in units of a plurality of unit sections is also employed. The The same applies to the noise spectrum N, and a configuration in which one noise spectrum N is generated in units of a plurality of unit sections is also employed. In each of the above embodiments, the suppression gain series H is generated for each unit section. However, a configuration in which one suppression gain series H is generated using a plurality of unit sections as a unit is also employed.

DESCRIPTION OF SYMBOLS 100 ... Sound processing device, 12 ... Arithmetic processing device, 14 ... Memory | storage device, 20 ... Frequency analysis part, 30 ... Sound source separation part, 32 ... Signal processing part, 34 ... Frequency selection part, 36 ... ... Intensity specifying unit 42... Noise estimating unit 42 and 44... Noise suppressing unit 44 and 52... Intensity adjusting unit 54. ... gain setting section, 621 ... first processing section, 622 ... second processing section, 64 ... suppression processing section.

Claims (8)

From a plurality of sound signals generated by a plurality of sound collecting devices, a target sound spectrum composed of components of a target sound frequency where the target sound is dominant, and a non-target sound coming from a different direction from the target sound is dominant. Sound source separation means for generating a non-target sound spectrum composed of components of the target sound frequency;
Fluctuation sound suppression means for suppressing non-target fluctuation sound in the non-target sound spectrum after separation by the sound source separation means;
A sound processing apparatus comprising: synthesis means for synthesizing the target sound spectrum after separation by the sound source separation means and the non-target sound spectrum after processing by the fluctuation sound suppression means. The sound processing apparatus according to claim 1, further comprising: an intensity adjustment unit that suppresses the non-target sound spectrum after separation by the sound source separation unit according to a suppression coefficient. The fluctuating sound suppression means includes
Gain setting means for generating a suppression gain sequence composed of gains set for each frequency;
2. A suppression processing unit that suppresses the non-target fluctuation sound by adjusting the intensity at each frequency of the non-target sound spectrum after separation by the sound source separation unit according to each gain of the suppression gain series. The sound processing apparatus according to claim 2. The gain setting means includes
First processing means for generating an emphasis gain sequence composed of gains set for each frequency so that non-target fluctuation sound is emphasized in the non-target sound spectrum after separation by the sound source separation means;
The sound processing apparatus according to claim 3, further comprising: a second processing unit that generates the suppression gain sequence from the enhancement gain sequence. The sound processing apparatus according to claim 4, wherein the second processing unit generates the suppression gain series according to the enhancement gain series and a variably set adjustment value. Noise estimation means for generating a noise spectrum composed of non-target stationary sounds out of the non-target sound spectrums separated by the sound source separation means;
Noise suppression means for suppressing non-target stationary sound of the noise spectrum from the target sound spectrum after separation by the sound source separation means,
The gain setting means generates the suppression gain sequence from the non-target sound spectrum after separation by the sound source separation means and the noise spectrum estimated by the noise estimation means,
The sound processing apparatus according to claim 3, wherein the synthesizing unit synthesizes the target sound spectrum processed by the noise suppressing unit and the non-target sound spectrum processed by the fluctuating sound suppressing unit. The noise estimation means includes
The intensity of the component of one non-target sound frequency in the first unit section is below a threshold value exceeding the intensity at the one non-target sound frequency in the noise spectrum of the second unit section before the start of the first unit section. , The intensity at the one non-target sound frequency in the noise spectrum of the first unit section, the intensity of the component of the one non-target sound frequency in the first unit section, and the noise spectrum of the second unit section. Set according to the intensity at the one non-target sound frequency,
When the intensity of the component of the one non-target sound frequency in the first unit section exceeds the threshold, the intensity at the one non-target sound frequency in the noise spectrum of the first unit section is determined as the first unit section. The sound processing device according to claim 6, wherein the sound processing device is set in accordance with the intensity at the one non-target sound frequency in the noise spectrum of the second unit section without reflecting the intensity of the component of the one non-target sound frequency. From a plurality of sound signals generated by a plurality of sound collecting devices, a target sound spectrum composed of components of a target sound frequency where the target sound is dominant, and a non-target sound coming from a different direction from the target sound is dominant. Sound source separation processing for generating a non-target sound spectrum composed of components of the target sound frequency;
Fluctuation sound suppression processing for suppressing non-target fluctuation sound in the non-target sound spectrum after the sound source separation processing;
A program for causing a computer to execute a synthesis process for synthesizing a target sound spectrum after the sound source separation process and a non-target sound spectrum after the fluctuating sound suppression process.

Images