JP2005173055A - Device, method and program for removing acoustic signal - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To exclude a stationary sound to be a cause of erroneous estimation, to predict change of the sound which is going to be removed automatically and with high precision and to appropriately remove it in the case of removal processing in a mixed sound. <P>SOLUTION: This device for removing an acoustic signal is provided with a stationary sound detection part 18 which detects the stationary sound in a mixed acoustic signal constituted by mixing a known acoustic signal which is going to be removed with other acoustic signals, an acoustic amplitude extraction part 200 which extracts acoustic amplitude spectrums from the acoustic signal and a removal processing part 104 which selects a mixed acoustic amplitude spectrum with frequency which does not coincide with frequency of the stationary sound among the extracted mixed acoustic amplitude spectrums and removes the known acoustic amplitude spectrum from the mixed acoustic amplitude spectrum with the selected frequency. <P>COPYRIGHT: (C)2005,JPO&NCIPI

Description

  The present invention relates to an acoustic signal removing device, an acoustic signal removing method, and an acoustic signal for removing sound such as BGM and voice mixed in content when reusing the content such as rebroadcasting a program that has already been broadcast. Relates to the removal program.

In the broadcasting industry in recent years, content may be reused, for example, by rebroadcasting a program that has already been broadcast. When the content is reused, it may be used as a new material by deleting only the music being used from the video material in which the already broadcast audio and music are mixed. As described above, as a technique for subtracting specific sound from existing mixed sound, for example, there is a technique disclosed in Patent Document 1.
JP 2000-31395A

  However, when creating audio for a broadcast program, frequency characteristics and volume are often adjusted according to the production intention. For this reason, since the phase of the sound changes in an unpredictable manner, it cannot be appropriately erased simply by performing an electronic subtraction process.

  In detail, even if the sound to be removed is known music such as BGM of a program, for example, BGM bass and treble are emphasized or attenuated for program effects when recording, In the process of repeating playback, the frequency characteristics of BGM may change, and simple subtraction cannot be performed.

  In addition, steady noise (steady sound) of a specific frequency such as hiss and various beat sounds from analog tape recorders may be included in the video data. There was a problem that. In other words, the stationary sound is a sound that is not included at all on the known acoustic side, and thus causes an erroneous estimation of the frequency characteristics.

  Furthermore, the process of removing the known sound is performed only at the places where the known sound is mixed. However, if only a part of the audio file is deleted, the volume changes between the deleted part and the other part. There is a problem that occurs.

  The present invention has been made to solve the above-described problems, and eliminates stationary sound that causes erroneous estimation during removal processing in mixed sound, and automatically and accurately changes the sound to be removed. An object of the present invention is to provide an acoustic signal removal device, an acoustic signal removal method, and an acoustic signal removal program that can be predicted and removed appropriately.

  In order to solve the above-described problems, the present invention detects a stationary sound in a mixed acoustic signal obtained by mixing a known acoustic signal to be removed and another acoustic signal, and detects a known acoustic amplitude spectrum from the known acoustic signal. And a mixed acoustic amplitude spectrum for each frequency is extracted from the mixed acoustic signal, and a mixed acoustic amplitude spectrum having a frequency that does not match the frequency of the stationary sound is selected from the extracted mixed acoustic amplitude spectra. Remove the known acoustic amplitude spectrum from the mixed acoustic amplitude spectrum of the frequency.

  In the above invention, in the mixed acoustic signal, the frequency that becomes the minimum amplitude, the frequency that becomes the nth value sorted in order of values, or the frequency that becomes the maximum value of n that does not exceed a certain value from the standard deviation up to the nth. Is preferably detected as a stationary sound.

  In another invention, when the known acoustic signal is removed from the mixed acoustic signal obtained by mixing the known acoustic signal to be removed and the other acoustic signal, the acoustic signal having a constant amplitude as the mixed acoustic Perform a pseudo subtraction process with the known acoustic signal set to 0, measure the difference in volume between the acoustic signal with a constant amplitude and the acoustic signal after the pseudo subtraction process, and based on the measurement result, signal for each time of the known acoustic signal The intensity is set, the known acoustic amplitude spectrum is extracted from the known acoustic signal, the mixed acoustic amplitude spectrum is extracted from the mixed acoustic signal, the known acoustic amplitude spectrum is converted based on the setting, and the mixed acoustic amplitude spectrum is extracted. Remove the known acoustic amplitude spectrum. This pseudo subtraction process can be a subtraction of an amplitude spectrum, similarly to the removal process that is finally performed.

  According to the present invention as described above, the amplitude spectrum of the acoustic signal does not depend on the phase, so that even if the phase changes, the frequency characteristics and volume changes of the known acoustic signal in the mixed acoustic signal are not affected by the change. Can be estimated appropriately. As a result, according to the present invention, for example, only the music can be erased accurately from the audio signal of the program in which audio and music are mixed, using the sound data such as a music CD used at the time of creating the program. In the present invention, not only music but also background noise mixed at the time of creating a program can be deleted using sound data in which only noise is recorded at the same time.

  In addition, according to the present invention, stationary noise of a specific frequency such as hiss and various beat sounds by an analog tape recorder is detected, and the removal process is performed by ignoring the frequency channel including the noise. The frequency characteristics can be estimated more appropriately.

  Furthermore, according to another invention, a single sound having a constant amplitude (for example, 480 Hz) is input as a mixed sound, a pseudo-subtraction process is performed with the amount of known sound drawn as zero, and a difference in output volume is measured. Since the values are set to match, even when only a part of a specific audio file is deleted, the respective volumes can be kept the same.

[Configuration of known sound removal system]
Embodiments of the present invention will be described with reference to the drawings. FIG. 1 is a block diagram showing the overall configuration of a known sound removal system according to this embodiment.

  As shown in FIG. 1, the known sound removal system according to the present embodiment includes an input I / F 1 and a DV capture 2 for inputting mixed sound and known sound. Files (for example, AVI files and WAV files) input from these input I / F 1 and DV capture 2 are stored in the storage device 5. The input I / F 1 is an interface that takes in an acoustic signal from a playback device such as a CD player or an MD player. The DV capture 2 is an interface for extracting MIX audio for erasure, which is a mixed acoustic signal in which video and audio are mixed.

  Further, the known sound removal system includes a voice conversion unit (PreWav / PostWav) 3 and a voice data extraction unit (DVReMix) 4 that perform voice data extraction processing and voice conversion processing on various data stored in the storage device 5. I have. The audio conversion unit (PreWav / PostWav) 3 and the audio data extraction unit (DVReMix) 4 read the designated file (AVI file or WAV file) from the storage device 5, perform a predetermined process, The file (WAV file) is stored in the storage device 5.

  The audio conversion unit (PreWav) 3 performs frequency conversion and separation of monaural from stereo. That is, the WAV file is separated into two left and right channels so as to conform to the format of the sound removal engine program 100, and the sampling rate is converted to 48 kHz, and two WAV files (output file name: left channel is MIX-L.WAV). , The right channel is generated as MIX-R.WAV) and stored in the storage device 5.

  The audio data extraction unit (DVReMix) 3 is a module that extracts only audio data from content composed of video data and audio data. In this embodiment, audio data is extracted from an AVI file in the WAV format. The WAV file here is in a stereo format, and its sampling rate is 32 kHz or 48 kHz, which is the same as DV audio. The extracted WAV file is stored in the storage device 5.

  The known sound removal system includes a sound removal engine program 100 that removes the known sound signal from the mixed sound signal. The sound removal engine program 100 reads each audio file (WAV file) stored in the storage device 5 and stores the removed data and various data related to the removal process in the storage device 5 via the temporary memory 7. Or output from the monitor 10 or the speaker 11 through the output I / F 8. The monitor 10 displays an editor 400 that is a GUI for displaying the operation and processing results by the user interface 6, and the speaker 11 outputs mixed sound, known sound, and post-removal sound based on the user operation by the user interface 6. .

  The sound removal engine program 100 acquires an operation signal based on a user operation by an input device such as a keyboard 6a or a mouse 6b through the user interface 6, and performs various processes based on the operation signal. The known sound removal processing by the sound removal engine program 100 will be described later.

  Further, the known sound removal system includes a synchronization control unit 9, thereby reading out data from the storage device 5, removal processing by the sound removal engine program 100, and data input / output by the memory 7 and the output I / F 8. Synchronize. Thereby, the video displayed on the editor 400 and the sound output from the speaker 11 can be synchronized with the processing by the sound removal engine program 100 and the user operation on the user interface 6.

  Furthermore, the sound removal engine program 100 includes a simulation unit 14 that sets the default value by simulation when setting parameters, and supports the user's work.

  Specifically, the simulation unit 14 inputs a single tone (480 Hz) having a constant amplitude as a mixed sound, performs a removal process with a known sound as zero, compares the output volume with the mixed sound before the processing, and performs a difference. The amount is measured, and a default value of the removal intensity in the user interface 6 is set so that the difference amount becomes zero.

  Further, the simulation unit 14 performs the simulation by assuming that the phase difference between the mixed sound and the known sound is uniformly distributed from 0 degrees to 360 degrees by a “phase-independent subtraction function” described later. Specifically, the simulation unit 14 sets the amplitude of the other acoustic signal to a predetermined value, and changes the phase difference of the amplitude of the known acoustic signal with respect to the amplitude in a range of 0 to 360 degrees. By calculating the average value of the amplitude, calculating the approximate value of the ratio of the amplitude of the other acoustic signal to the average value from the ratio of the amplitude of the known acoustic signal to the average value, and adding the average value to this approximate value Based on the amplitude of the obtained other acoustic signal, the removal strength of the known signal is set. Hereinafter, the derivation of the phase difference independent subtraction function will be described.

First, in the present embodiment, when calculation is performed for each frequency channel and the frequency is f (Hz), the relationship between the mixed sound, the known sound, and the signal of the sound output after erasure (other sound) is
"Mixed sound" = "Sound output after erasure" + "Known sound"
It is expressed. And each of the above sounds

It is expressed. Here, from the relationship between mixed sound, known sound and sound output after erasure,

It becomes. Since the above equation (Equation 2) holds at any time t, the coefficients of Sin (ωt) and cos (ωt) are equal. Therefore,

Is established. From this relationship, M can be represented by A and B, and the phase difference (θa · θb) between known sound and audio output as follows.

Here, when the phase difference (θa · θb) between the known sound and the audio output is replaced with the phase difference θδ,

It becomes.

In the present embodiment, it is assumed that this phase difference θδ occurs with a uniform probability, and the average value of actually measured M values is calculated by integrating θδ from 0 radians to 2π radians as shown in the following equation. To do.

For example, assuming that A = 1.0, the following table is obtained by simulating the case where b takes various values.

を正規化し、

here,

Normalize

で割っても（

の逆数倍しても）、下表のように、上記関係は維持される。

And As is clear from the above equation 6, if A and B are multiplied by a constant, N also becomes a constant multiple.

Divide by (

The above relationship is maintained as shown in the table below.

  In the removal process of the present embodiment, the amplitude (M) of the mixed sound and the amplitude (B) of the known sound are acquired, so that Rb, which is a value obtained by dividing B by M, can be obtained. Therefore, Ra can be obtained from the table data corresponding to Table 2 or an approximate expression thereof.

next,

Thus, the amplitude A of the other sound can be obtained.

からＡを算出する。

As an actual application, B is changed from 0 to 100 in increments of 0.01, and the following approximate expression is generated for the values in the table, and the average value of M

A is calculated from

  Theoretically, the amplitude of the mixed sound does not exceed the amplitude of the known sound. However, in actuality, it occurs due to an estimation error or an erasure intensity greater than 1, and in such a case (Rb is 1). Ra is set to 0 in the case of larger).

  The sound removal engine program 100 also includes a stationary sound detection unit 18. As shown in FIG. 6B, the stationary sound detection unit 18 detects a stationary sound from the mixed acoustic signal related to the removal processing in the removal engine program 100, and removes the detected steady sound from the removal engine program 100. It is a module to notify to.

  Based on this notification, the removal engine program 100 selects a mixed acoustic amplitude spectrum having a frequency that does not match the frequency of the stationary sound from the mixed acoustic amplitude spectrum, and selects a known acoustic amplitude spectrum from the mixed acoustic amplitude spectrum of the selected frequency. Remove.

  In detecting the stationary sound, the amplitude for each frequency channel in the mixed sound signal is detected in association with the time, and the amplitude is sorted and the minimum value or the representative value is calculated. There are the following three methods for calculating the stationary sound.

(1) Minimum value method The amplitude data for each frequency channel is extracted from the mixed sound in association with the time by FFT processing. Here, for example, a channel of 15.7 kHz
Time 0.00 seconds 0.01 seconds 0.02 seconds …… 1456.345 seconds Amplitude 1.5 2.5 1.5 4.5
It is assumed that the value of is detected.

  In this minimum value method, the minimum value for all time values is determined as a stationary sound from the above data.

(2) Ranking method In the above-described minimum value method, the amplitude may be small when the noise is not “steady” or depending on the phase of other speech or synthesized music. The minimum value may be lower than the original steady value. In this ranking method, the following values are used instead of the minimum values.

  The above amplitude values are sorted from the smallest.

Rank 1 2 3 4 5… 345 346
Amplitude 0.34 0.035 0.36 0.40 0.40… 18.9 19.5
For example, if it is 3 seconds, it will be 300 samples because it is every 10 milliseconds. A number from 1 to 300 is given to this. Then, based on the prediction that there is at least 10% of sections without noise, the 10% value of 300, that is, the 30th value is adopted as the stationary sound.

(3) Ranking improvement method In the ranking method, there are too many or too few sections of noise alone, and it is necessary to set a threshold of 10%, so the calculation result may be affected by the threshold setting. There is. Therefore, as an improvement of the ranking method, the standard deviation σ (n) is obtained by taking statistics for the noises from the smallest to the nth of the above amplitude values, and this σ (n) does not exceed a certain value. The maximum value of n is obtained, and the amplitude value at the time of n is set as a stationary sound. This constant value is preferably determined empirically.

[Operation of known sound removal system]
The known sound removal system having the above-described configuration operates as follows. FIG. 2 is a flowchart showing the operation of the known sound removal system. In the present embodiment, a video file (DV) in which video and audio are recorded in stereo is referred to as mixed sound (MIX audio), an audio file in which the original music is recorded is referred to as known sound, and is included as BGM in the video file. An example of removing the original music that has been performed will be described. The processing in this embodiment is roughly divided into (1) preprocessing, (2) music erasure processing, and (3) postprocessing. Hereinafter, each process is explained in full detail.

(1) Pre-processing In the pre-processing, MIX sound for erasure is extracted from DV and BGM sound (original music) is prepared. Specifically, a video is captured from the DV capture 2 using DV video editing software (S101), and the captured file is stored as a type 1 AVI file (output file name: MIX.AVI). Accumulate in device 5.

  Next, the audio data extraction unit (DVReMix) 3 extracts audio data from the AVI file in the WAV format (output file name: MIX.WAV) (S102). The WAV file here is in a stereo format, and its sampling rate is 32 kHz or 48 kHz, which is the same as DV audio. The extracted WAV file is stored in the storage device 5.

  Thereafter, the audio conversion unit (PreWav) 4 performs frequency conversion and separation of monaural from stereo (S103). That is, the WAV file is separated into two left and right channels so as to conform to the format of the sound removal engine program 100, and the sampling rate is converted to 48 kHz, and two WAV files (output file name: left channel is MIX-L.WAV). , The right channel is generated as MIX-R.WAV) and stored in the storage device 5. In step S103, the offset of the start time of the video is output to the setting file (file name: MIX.time) simultaneously with the audio conversion, and is stored in the storage device 5.

  In parallel with these steps S101 to S103, the original music is taken in (S104). Specifically, the original music (BGM music) is taken from a CD or the like and stored in the storage device 5 as a 44.1 kHz stereo WAV file (output file name: BGM.WAV). Next, the audio conversion unit (PreWav) 3 performs frequency conversion and separation of stereo from monaural (S105). That is, to match the format of the sound removal engine program 100, the WAV file captured in step S104 is separated into two left and right channels, the sampling rate is converted to 48 kHz, and two WAV files (output file name: left channel is BGM) -L.WAV, the right channel is BGM-R.WAV) and stored in the storage device 5.

(2) Music Erasing Process In the music erasing process, the removal engine program (GEQ) 100 erases the BGM sound from the MIX sound (S106). The audio file output after erasure is a mono 48 kHz WAV file (output file name: ERASE-L.WAV for the left channel and ERASE-R.WAV for the right channel) for both the left and right channels in the memory 7 or the storage device 5. Accumulated.

(3) Post-processing In the post-processing, the voice erased by the removal engine program is converted into a voice for DV and restored to DV (AVI file). First, the audio conversion unit (PostWav) 3 performs frequency conversion and monaural to stereo conversion (S107). That is, the left and right two-channel WAV file output from the sound removal engine program 100 is synthesized into stereo, and if necessary, converted to the same sampling rate as the original DV sound, as a WAV file (file name: ERASE.WAV). And stored in the storage device 5. Next, in the audio data extraction unit (DVReMix) 4, the audio of the captured AVI file (BGM.AVI) is replaced with the audio after deletion (ERASE.WAV), and as an after-removal audio file (file name: ERASE.AVI) And stored in the storage device 5.

  For example, when working on the left audio (L) of stereo broadcasting in the first operation, various parameter setting data for the left audio is stored, and this is used for the removal process for the next right audio. And set as a default value in the user interface 6.

[Theory of sound removal processing]
Next, the sound removal engine program 100 described above will be described in detail. First, the theory of sound removal processing by the sound removal engine program 100 will be described.

ここでは、ｂ（ｔ）の元となる音源の音響信号ｂ’（ｔ）が既知という条件下で、ｍ（ｔ）が与えられたときに、未知のｓ（ｔ）を求める。例えば、人間の声や物音と共にＢＧＭが鳴っているテレビ番組等の音響信号ｍ（ｔ）を入力とし、そのＢＧＭの楽曲が既知でその音響信号ｂ’（ｔ）が別途用意できるときに、そのＢＧＭの音楽音響信号を用いて番組中のＢＧＭを除去し、人間の声や物音だけの音響信号ｓ（ｔ）を得る処理を実現する。 (Basic concept)
A mixed acoustic signal m (t) in which a non-stationary acoustic signal b (t) such as BGM is mixed with an acoustic signal s (t) (t is a time axis) such as a desired voice or a physical sound is observed. To do.

Here, when m (t) is given under the condition that the acoustic signal b ′ (t) of the sound source that is the source of b (t) is known, the unknown s (t) is obtained. For example, when an acoustic signal m (t) of a TV program or the like in which a BGM is sounding together with a human voice or a sound is input, the music of the BGM is known and the acoustic signal b ′ (t) can be prepared separately. The BGM in the program is removed using the BGM music sound signal, and the process of obtaining the sound signal s (t) of only human voice or sound is realized.

Here, since b (t) and b ′ (t) do not completely match,

In the process corresponding to the subtraction, it is necessary to estimate the component corresponding to b (t) from b ′ (t) and obtain s (t). Specifically, the component corresponding to b (t) of the known acoustic signal b ′ (t) is estimated by correcting the following deformation in the mixed sound m (t).

-Temporal position shift The position where the known acoustic signal b '(t) is sounding in the mixed sound m (t) is not necessarily from the beginning. Therefore, the known acoustic signal b ′ (t) is shifted in the time axis direction, the relative positions of the two are matched, and the known acoustic signal is subtracted from the mixed sound.

-Time change of frequency characteristic When a known acoustic signal b '(t) is sounded in the mixed sound m (t), the frequency characteristic often changes due to the influence of a graphic equalizer or the like. For example, the low range and high range may be emphasized and attenuated. Therefore, the frequency characteristic of b ′ (t) is similarly changed and corrected, and the known acoustic signal is subtracted from the mixed sound.

・ Change in volume over time When a known acoustic signal b '(t) is produced in the mixed sound m (t), the mixing ratio is changed by operating the fader of the mixer at the time of creating the mixed sound, and the volume changes over time. Often done. Therefore, the volume of b ′ (t) is similarly corrected by changing the time, and the known acoustic signal is subtracted from the mixed sound.

で定義されるとき、それらの振幅スペクトルＭ(ω，ｔ)，Ｂ’(ω，ｔ)は、

で求まる。 The processing flow of this program is shown in FIG. In this program, the subtraction process is performed on the amplitude spectrum in the time frequency domain without subtracting the waveform in the time domain. Short-time Fourier transform (STFT) X _m (ω, t), X _b (ω, t) at time t using the window function h (t) for the acoustic signals m (t), b ′ (t)

And their amplitude spectra M (ω, t), B ′ (ω, t) are

It is obtained by.

  In this embodiment, an acoustic signal is subjected to A / D conversion at a sampling frequency of 48 kHz and a quantization bit number of 16 bits, and an STFT using a Hanning window having a window width of 8192 as a window function h (t) is converted into a fast Fourier transform (FFT). ) To calculate. At that time, since the FFT frame is shifted by 480 points, the frame shift time (one frame shift) is 10 ms. This frame shift is used as a processing time unit. This program can easily handle other sampling frequencies (16kHz, 44kHz, etc.), window widths, and frame shifts.

上記の式における各種パラメータ関数ａ(ｔ)，ｇ(ω，ｔ)，ｒ(ｔ) ，ｃ(ω，ｔ)を順に説明する。 The amplitude spectrum S (ω, t) of the desired acoustic signal s (t) after removal of the known acoustic signal is obtained from the amplitude spectra M (ω, t) and B ′ (ω, t) as follows.

Various parameter functions a (t), g (ω, t), r (t), c (ω, t) in the above formula will be described in order.

A (t) is a function of an arbitrary shape for finally adjusting the amount by which the component corresponding to the amplitude spectrum of the known acoustic signal is subtracted from the amplitude spectrum of the mixed sound, and usually a (t) ≧ 1 And The larger the value, the larger the subtraction amount.

により求める。ここで、ｇω(ω，ｔ)は、周波数特性の時間変化を表し、周波数特性の変化がないときはｇω(ω，ｔ)＝１となる。一方、Ｍ(ω，ｔ)，Ｂ’(ω，ｔ)は、音量の時間変化を表し、音量の変化がないときは定数となる。Ｍ(ω，ｔ)とＢ’(ω，ｔ)との音量差は、基本的にｇｔ(ｔ)で補正される。ｇｒ(ｔ)は、主にｇ(ω，ｔ)の値を全体的に持ち上げるための関数で、補正時の微調整に使用される。使用しない場合には、ｇｒ(ｔ)＝０とする。 G (ω, t) is a function for correcting the time change of the frequency characteristic and the time change of the volume,

Ask for. Here, gω (ω, t) represents a time change of the frequency characteristic, and gω (ω, t) = 1 when there is no change of the frequency characteristic. On the other hand, M (ω, t), B ′ (ω, t) represents a change in volume over time, and is a constant when there is no change in volume. The volume difference between M (ω, t) and B ′ (ω, t) is basically corrected by gt (t). gr (t) is a function mainly for raising the value of g (ω, t) as a whole, and is used for fine adjustment during correction. If not used, gr (t) = 0.

・ R (t) is a function for correcting the positional deviation in time. Normally, a constant is set to correct a certain deviation (in this program, only the constant is supported). ).

C (ω, t) is a function of an arbitrary shape for equalizing processing and fader operation processing on the amplitude spectrum. Depending on the shape in the ω direction, the frequency characteristic after removal of the known acoustic signal can be adjusted like a graphic equalizer. Further, the change in volume after removal of the known acoustic signal can be adjusted by the shape in the t direction, like the volume fader operation of the mixer. When not used, c (ω, t) = 1. In this program, only c (ω, t) = 1 is supported, but a process of c (ω, t) may be added.

この単位波形を、Overlap Add法によって配置することにより、既知音響信号除去後の所望の音響信号ｓ（ｔ）を合成する。 By using the amplitude spectrum S (ω, t) thus obtained and the phase θm (ω, t) of the mixed sound m (t), Xs (ω, t) is obtained and subjected to inverse Fourier transform (IFFT). Get the unit waveform.

By arranging this unit waveform by the overlap addition method, a desired acoustic signal s (t) after the known acoustic signal is removed is synthesized.

(Setting of various parameter functions)
When executing the above processing, the various parameter functions a (t), g (ω, t) (gω (ω, t), gt (t), gr (t)), r The shapes of (t) and c (ω, t) may be set manually by the user, or possible ones may be automatically estimated. Alternatively, the user may correct after automatic estimation. In this program, some shapes of various parameter functions g (ω, t) (gω (ω, t), gt (t)), r (t) of Equation (11), Equation (12), and Equation (13) Since only automatic estimation is supported, the specific automatic estimation method implemented is described below.

In the estimation of g (ω, t), first, the time change gω (ω, t) of the frequency characteristic is estimated, and then the time change gt (t) of the volume is estimated. However, r (t) needs to be determined prior to estimating g (ω, t). Here, for convenience, B ′ (ω, t + r (t)) is described as B ′ (ω, t).

により推定する。ただし、ψは一つのＢＧＭ区間(時問軸上の領域)を表し、Ψは、ψの集合とする。一方、周波数特性が時間変化していくときには、ｇω(ω，ｔ)の時刻ｔに近いＢＧＭ区間ψから

を求め、補間(内挿あるいは外挿)することによりｇω(ω，ｔ)を推定する。最後に、ｇω(ω，ｔ)を周波数軸方向に平滑化する。なお、平滑化幅は任意に設定でき、平滑化をしなくてもよい。 In the estimation of the time variation gω (ω, t) of the frequency characteristic, as a rule, a section (hereinafter referred to as a BGM section) in which the acoustic signal s (t) of only a human voice or a sound is hardly included is used. A plurality of BGM sections may be used. In the BGM section, the amplitude spectrum M (ω, t) of the mixed sound m (t) is almost entirely derived from the amplitude spectrum B ′ (ω, t) corresponding to BGM by the known acoustic signal b ′ (t). It becomes. Therefore, when the frequency characteristic is steady without changing with time, that is, when gω (ω, t) = g′ω (ω) can be assumed, g′ω (ω) is

Estimated by Here, ψ represents one BGM section (region on the time axis), and ψ is a set of ψ. On the other hand, when the frequency characteristic changes with time, from the BGM section ψ near the time t of gω (ω, t).

Gω (ω, t) is estimated by interpolation (interpolation or extrapolation). Finally, gω (ω, t) is smoothed in the frequency axis direction. Note that the smoothing width can be arbitrarily set, and smoothing may not be performed.

を求める(Φはφの集合を表す)。Φとして任意の分割が適用できるが、例えば、音楽で用いる平均律の１オクターブ毎に分割(対数周波数軸上で等間隔に分割)するとよい。そして、ｇｔ(ｔ)は、ｍｉｎ（ｇ’ｔ(φ，ｔ)）あるいは、数１１により推定する。

In the estimation of the volume change gt (t) over time, the amplitudes of M (ω, t) and gω (ω, t) B ′ (ω, t) after frequency characteristic correction are compared at each time. However, M (ω, t) includes components derived from s (t) in addition to components derived from B ′ (ω, t). Therefore, the frequency axis ω is divided into a plurality of frequency bands Φ, and for each band φ (φ∈Φ)

(Φ represents a set of φ). Arbitrary division can be applied as Φ, but, for example, it is preferable to divide every one octave of equal temperament used in music (equal intervals on the logarithmic frequency axis). Then, gt (t) is estimated by min (g′t (φ, t)) or Equation 11.

Finally, gt (t) is smoothed in the time axis direction. Note that the smoothing width can be arbitrarily set, and smoothing may not be performed.

In the estimation of r (t), in principle, a set Ψ of BGM intervals ψ is used so that the time axis of the correspondence relationship between M (ω, t) and B (ω, t) in those intervals is matched. R (t). In this program, the element of the set Ψ of BGM intervals for estimating r (t) is limited to one, and r (t) corresponds to only a constant.

が有効である。なお、最初のψ₁は、ユーザーが手作業で指定する。 In the above estimation of g (ω, t), r (t), etc., a set Ψ of BGM sections ψ is used. This may be designated manually by the user, or may be automatically estimated as follows. In the automatic estimation of Ψ, basically, a set of remaining BGM sections is obtained using one BGM section ψ1 as a clue. First, various parameter functions of B (ω, t) are estimated and temporarily determined based on ψ1, and the amplitude spectrum of M (ω, t) and B (ω, t) in the interval of ψ1 at that time. The distance between them is obtained, and a constant multiple of the maximum value (the maximum value is used in this program, but an implementation using an average value is also possible) is set as a BGM section determination threshold. Then, the distance between the amplitude spectra of M (ω, t) and B (ω, t) is obtained for all the sections, a section below the BGM section determination threshold is detected, and newly added to Ψ. Although this estimation is not repeated in this program, Ψ may be obtained by repeating this estimation. Here, as the distance between M (ω, t) and B (ω, t), for example, the root mean square logarithmic spectral distance

Is effective. The first ψ ₁ is manually specified by the user.

(Estimated expansion and contraction of known sound)
In the present embodiment, the following modifications can be accommodated by extending the frameworks of the above formulas 5 and 6.

  When a known acoustic signal b ′ (t) is sounded in the expansion / contraction sound m (t) in the time axis or frequency axis direction, the sound signal is expanded or contracted in the time axis or frequency axis direction due to the difference in the rotational speed of the record or the like. Sometimes. Therefore, it is necessary to correct b ′ (t) by expanding and contracting in the time axis or frequency axis direction, and subtracting a known acoustic signal from the mixed sound.

上記の式におけるパラメータ関数ｐ（ω）、ｑ（ｔ) を説明する。 In order to cope with this, the second equation in Equation 5 is defined as follows.

The parameter functions p (ω) and q (t) in the above equation will be described.

P (ω) is a function for correcting expansion / contraction in the frequency axis direction, and linear / non-linear expansion / contraction in the frequency axis direction by converting the frequency axis ω of the amplitude spectrum B ′ (ω, t). Enable. B ′ (ω, t) is 0 outside the original definition range of ω, and is interpolated as appropriate when discretized and implemented.

Q (t) is a function for correcting expansion / contraction in the time axis direction, and linear / nonlinear expansion / contraction in the time axis direction by converting the time axis t of the amplitude spectrum B ′ (ω, t). Enable. Note that B ′ (ω, t) takes 0 outside the original definition range of t, and is interpolated as appropriate when discretized and implemented. Although it is possible to express q (t) and r (t) by a single function, q (t) is set for the purpose of continuous expansion and contraction, and r (t) is discontinuous. It is set for the purpose of representing the position shift.

A method for automatically estimating the shapes of the parameter functions p (ω) and q (t) newly introduced in the above equation 13 will be described below.

In the estimation of p (ω) and q (t), p (ω) is set so that the distance between M (ω, t) and B (ω, t) (for example, logarithmic spectral distance, etc.) is minimized. Change q (t). At that time, B (ω, t) = a (t) g (ω, t) B ′ (p (ω), q (t) + r (t)) of the right side, a (t) = 1,
1. 1. Estimate g (ω, t) and r (t) after temporarily fixing p (ω) and q (t) (during estimation) (Estimated) g (ω, t) and r (t) are temporarily fixed, and p (ω) and q (t) are estimated repeatedly. , Q (t). This may be performed in a segmented manner by dividing the time axis without performing it for all sections of the acoustic signal at once. The initial value is determined in consideration of the continuity of the preceding and following sections. Further, by using the set Ψ of BGM sections ψ, p (ω), q () so that the time axes of the correspondence relationship between M (ω, t) and B (ω, t) in the plurality of sections are matched. t) may be estimated. When a part of the known acoustic signal b ′ (t) is not used and is mixed by skipping, r (t) is made to be a discontinuous function so as to skip that part.

Here, when automatically estimating the set Ψ of BGM sections ψ, the user needs to manually specify the first ψ ₁ , but the time axis of the acoustic signal is divided finely, There is also a method for investigating and finding the correspondence between short divided sections.

のようにＳ(ω，ｔ)を求める処理へ拡張できる。その際には、Ｂｎ (ω，ｔ)の各種パラメータ関数を順に設定するか、全体のバランスを取りながら、複数のＢｎ (ω，ｔ)の各種パラメータ関数を平行して設定する。 (Corresponding to multiple known acoustic signals, etc.)
This program corresponds to the case where one type of known acoustic signal b ′ (t) is included in the mixed acoustic signal m (t). b ' ₁ (t), b' ₂ (t)..., b ′ _N (t), when there are a plurality of amplitude spectra B ′ ₁ (ω, t), B ′ ₂ (ω , t) ... from _{B 'N (ω, t)} , B was determined by the above expression 5 second equation _{1 (ω, t), B} 2 (ω, t) ... B N (ω, t) with,

Thus, the process can be expanded to obtain S (ω, t). At that time, various parameter functions of Bn (ω, t) are set in order, or various parameter functions of a plurality of Bn (ω, t) are set in parallel while maintaining the overall balance.

Although this program is intended for monaural signals, a stereo signal may be applied by mixing the left and right to convert to a monaural signal, or applied to the left and right signals of the stereo signal. Good. Moreover, you may extend so that it may apply using the sound source direction in a stereo signal.

(Experimental result)
The experimental results of the sound removal engine program according to this embodiment are shown below. Here, when an acoustic signal m (t) is observed in a mixture in which an acoustic signal b (t) such as BGM is added to an acoustic signal s (t) such as a voice or a sound, a b (t) Under the condition that the acoustic signal b ′ (t) of the original sound source is known, the unknown s (t) was obtained. If an audio file in which m (t) and b ′ (t) are recorded is given, an audio file of s (t) can be obtained.

  As a result of experiments on mixed sound in which background music (BGM) is added to human voice, the BGM in the mixed sound is removed using the sound signal of the original music of the BGM, and human voice and sound are It was confirmed that it was obtained.

  As an example of the experimental results, FIGS. 4A to 4F show results of actually processing a mixed sound in which classical music is played in the BGM of dialogue between two men and women. Using the mixed sound m (t) shown in FIGS. 4 (a) and 4 (b) as an input, the BGM component is removed using the known acoustic signal b ′ (t) of the original sound source shown in FIGS. 4 (c) and 4 (d). As a result, the acoustic signal s (t) after the known acoustic signal removal shown in FIGS. 4E and 4F was obtained.

  From the above, when an audio signal such as a TV program or a movie with BGM sounding in the background of a human voice or sound is input, BGM in the program is removed using a BGM music audio signal prepared separately, It is possible to obtain an audio signal only of voice and sound. In addition, you may make it provide another music as BGM with respect to the acoustic signal after BGM removal.

[Configuration of sound removal engine]
The configuration of the sound removal engine program 100 based on the theory described above will be described. FIG. 5 is a block diagram illustrating functions of the sound removal engine program 100.

  As shown in FIG. 5, the sound removal engine program 100 includes, as signal input means, a mixed sound input unit 101 to which a mixed sound signal is input, and a known sound signal input unit 102 to which a known sound signal to be removed is input. And a post-removal acoustic signal output unit 107 as an output means of the acoustic signal subjected to the removal process.

  The sound removal engine program 100 also includes an amplitude spectrum extraction unit 200 that extracts an amplitude spectrum from the input sound signal. Specifically, the amplitude spectrum extraction unit 200 includes a data division unit 201, a window function processing unit 202, and a Fourier transform unit 203.

  The data dividing unit 201 divides the mixed acoustic signal into sections having a specific length (window size). In general speech recognition and the like, the length of one section is about 20 milliseconds, but the same sound continues longer in music than in speech, so in this embodiment, about 10 times longer than this. It is 8192 samples (8192 ÷ 48,000 = 0.170 approximately 170 milliseconds), which is a power of 2.

  The window function processing unit 202 multiplies the audio signal data in the window size section (170 msec) divided by the data dividing unit 201 by a Hanning function so as to smoothly converge to zero in the first and last portions of the data. Convert to signal waveform.

  The Fourier transform unit 203 performs Fourier transform on the data of each of the mixed acoustic signal and the known acoustic signal, and separates and outputs the phase and amplitude spectrum for each frequency channel. Data consisting only of the amplitude spectrum is output as “time frequency data”.

  More specifically, the Fourier transform unit 203 performs fast Fourier transform (FFT) on the audio data that has been subjected to the Hanning function process. Note that the input speech data is only real numbers and includes an imaginary part.In this FFT, the input and output are calculated as complex numbers, so two window transformations are performed on the real part and imaginary part of the input data, respectively. Fast Fourier transform is performed, and after conversion, separation is performed using a conjugate relationship, realizing a double speed improvement. This system uses SSE2 instructions that can be used with Intel's Pentium 4 (registered trademark) processor to speed up the processing.

  Then, the amplitude spectrum extraction unit 200 moves the section to be Fourier-transformed in units of 480 samples (480 ÷ 48,000 = 0.01: 10 milliseconds), the multiplication of the Hanning window function by the window function processing unit 202 and the Fourier transform unit 203, and the Fourier Repeat the conversion process. From the data obtained every 10 milliseconds in this way, “time frequency data” that is data representing “amplitude only” of the audio signal is obtained for each frequency channel. The frequency channels thus obtained are 4096 channels from 0 Hz (direct current) to about 24 kHz every about 5.86 Hz, such as 0 Hz, 5.86 Hz, 11, 72 Hz, 17.57 Hz... 23,994.14 Hz.

  When the input signal is a mixed sound signal, the amplitude spectrum extraction unit 200 functions as a mixed sound amplitude extraction unit that extracts a mixed sound amplitude spectrum from the mixed sound signal, and the input signal tries to be removed. When it is a known acoustic signal, it functions as a known acoustic amplitude extracting section that removes this and extracts the acoustic amplitude spectrum.

  Further, the sound removal engine program 100 automatically estimates the change of the known sound in the mixed sound based on the amplitude spectrum of the known sound extracted from the amplitude spectrum extraction unit 200, and corrects the automatic estimation result by a user operation. A parameter estimation unit 300 is provided.

  The parameter estimation unit 300, based on the data for each frequency channel of the “time frequency data” extracted from the amplitude spectrum extraction unit 200, all the parameter functions a (t) and g (5) described above. ω, t) (gω (ω, t), gt (t), gr (t)), p (ω), q (ω), r (t), c (ω, t) are automatically estimated. Or set by user operation. The user may draw and specify an arbitrary function shape from the beginning, or may first perform automatic estimation and correct the result.

  The parameter estimation unit 300 includes a calibration unit 304 that performs calibration of mixed sound and known sound. The calibration unit 304 uses the section (BGM section) in which the acoustic signal s (t) of only human voice or sound is not included (BGM section), and the various parameter functions g (ω, t ), (Gω (ω, t), gt (t)) and r (t) are partially estimated automatically.

  Specifically, by a user's manual operation, a section from 1 second to several seconds in which only known sound is output is selected from the mixed sound, and substantially the same part is selected for known sound. And the calibration part 304 totals the volume for every frequency channel in this selected area, and compares the value obtained from a mixed sound and a removal sound. In this section, since the mixed sound contains only known sound, the ratio becomes the frequency characteristic.

  Further, the parameter estimation unit 300 includes a frequency characteristic change correction unit 301, a sound volume change correction unit 302, and a time position correction unit 303, whereby (1) a temporal positional shift between mixed sound and known sound, ( 2) Estimate the frequency characteristics of the known sound, and (3) change over time of the volume of the known sound.

  The frequency characteristic change correction unit 301 is a module that estimates a frequency distribution. When the frequency distribution is estimated, c (ω, t), which is a function of an arbitrary shape for equalizing processing and fader operation processing on the amplitude spectrum. For, the frequency characteristic after removal of the known acoustic signal is adjusted like a graphic equalizer by changing the shape in the ω direction.

  Further, the frequency characteristic change correction unit 301 smoothes the frequency characteristic because a value obtained by noise or the like becomes unstable in a portion where the volume of the BGM is low in the audio channel.

  In detail, since it is impossible to estimate if there is no sound of a known sound in each frequency channel to be estimated, when estimating the frequency distribution, the so-called “rich” part that is included in the known sound from low to high is included. It is desirable to use it.

  However, in the case of very subdivided channels such as 4096 channels, it is considered almost impossible to include sound in all channels, and the sound of known sounds is small, whereas the mixed sound If there is noise on the side, the result of division may be an insane estimate.

  On the other hand, in this embodiment, the frequency distribution is smoothed. This smoothing is realized by taking the average value of the previous and subsequent channels and smoothing them. In this embodiment, the number of channels can be made smoother as this value is increased by the slider “SmoothingFreq.Weight” of the editor 400.

  In this embodiment, another smoothing mechanism (Blur_freq_mode) is implemented by the button “BAFAM” of the editor 400. The frequency channel without sound cannot be estimated, and may be extremely depressed compared to the adjacent frequency channel. By raising the value to the same value as the adjacent frequency channel, it is possible to avoid such an unpredictable case. it can.

  Furthermore, in this embodiment, the frequency channel that is completely zero is implemented by the slider “ShiftFreqWeight” of the editor 400 in order to avoid affecting the estimation process.

  Further, the frequency characteristic change correction unit 301 according to the present embodiment has a steady sound processing function for performing processing while ignoring stationary noise of a specific frequency such as hiss and various beat sounds by an analog tape recorder. When the mixed sound contains a stationary sound such as a horizontal frequency (15.75 kHz) included in the video data, there is a problem that the estimation is largely wrong in the surrounding frequency channels. That is, the stationary sound is a sound that is not included at all on the known acoustic side, and thus causes a frequency characteristic estimation error.

  The sound volume change correction unit 302 performs estimation and smoothing of the time change of the sound volume. When estimating the temporal change in volume, the volume change after removal of the known acoustic signal can be adjusted by correcting the shape of c (ω, t) in the t direction, as in the volume fader operation of the mixer. .

  More specifically, in the time direction sound volume estimation, when the mixed sound side includes all frequency regions depending on the time, there is a tendency to estimate that the known sound is larger than the actual sound. In this case, if it is simply drawn, it will draw even a sound that should not be erased, and there is a risk that the sound will be “skinned”.

  In the present embodiment, the sound volume change correction unit 302 detects the time change of the sound volume of the known sound over the entire mixed sound time range. Since the mixed sound includes sounds such as voice in addition to the known sound, the frequency channels of the known sound corrected by the mixed sound and the frequency characteristics are summed up for every octave (every twice the frequency). Compared at the same time, the one with the smallest ratio of the mixed sound to the known sound is selected. Thereby, when compared with every octave, it is possible to reflect a possibility that only one known sound is present in any one section. This is the volume ratio between the known sound and the mixed sound at that time.

  In the present embodiment, the user recognizes that the sound volume is clearly increased from the graph display, and manually corrects the problem. In this estimation, an automatic determination method such as a robust statistical method may be employed.

  Further, the volume change correction unit 302 performs smoothing even when estimating the time change, and smoothes the average value of the volume of the known sound before and after the time. This value is implemented as a slider “SmoothingTimeWeight” in the editor 400, and can be smoothed by increasing this value.

  In the present embodiment, another smoothing mechanism (Blur_time_mode) is implemented by a “BATAM” button. When it is impossible to estimate the time when there is no sound and the time is extremely low compared to the adjacent time, the unpredictable case is avoided by raising the value to the same value as the adjacent time.

  Note that the slider “ShiftTimeWeight” in the editor 400 is implemented to prevent this because the sound volume is estimated to be zero at a time when there is no known sound.

  Furthermore, regarding the adjustment of the volume of the mixed sound and the sound after erasure, when only a part of a specific audio file is erased, it is necessary to maintain the same volume. For this reason, in the present embodiment, a single tone (480 Hz) with a constant amplitude is synthesized and input as a mixed sound, a removal process is performed with the amount of known sound drawn as zero, a difference in output volume is measured, and the value is calculated. Set to match.

  The time position correcting unit 303 estimates a time positional shift between the start point of the mixed sound and the start point of the known sound. By setting the constant of the function r (t) described above, a constant shift is performed. Correct the width. More specifically, when the specified mixed sound and the known sound are displaced in time, the respective correctors are shifted by up to 100 milliseconds before and after the known sound with respect to the mixed sound. The processing by 301 and 302 is repeated. The difference in volume between the mixed sound and the known sound in the designated section is obtained, and it is determined that the difference is the best at the time when the difference is the minimum, and the time difference between the mixed sound and the known sound is determined.

  In this system, it is possible to perform automatic estimation by executing the above-described procedure or to specify that a predetermined time lag is specified by the user and not to perform automatic estimation. Alternatively, the mixed sound and the known sound at the user's ear may be output from separate left and right speakers, listened while comparing these sounds, and may be aligned by the user's hearing.

  Further, the sound removal engine program 100 includes a removal processing unit 104 that removes a known sound amplitude spectrum from the mixed sound amplitude spectrum extracted by the amplitude spectrum extraction unit 200, and an inverse Fourier transform to restore the sound after removal. A Fourier transform unit 105 and an arrangement processing unit 106 are provided.

  The removal processing unit 104 converts the known sound according to the estimation data generated by the parameter estimation unit 300, and deletes the converted signal from the “time frequency data” of the mixed sound. In this embodiment, a “phase-independent subtraction algorithm” by the simulation unit 14 is also implemented in this embodiment. In other words, in the present embodiment, the above-described “phase-independent subtraction function” is used to perform a simulation on the assumption that the phase is uniformly distributed from 0 degrees to 360 degrees, and the erasing intensity is matched with this simulation result. Is set automatically.

  The inverse Fourier transform unit 105 restores only the sound-only data from which the known sound is eliminated from the “time-frequency data” obtained by the subtraction calculation and the phase data in the mixed sound signal by the inverse Fourier transform. Specifically, the inverse Fourier transform unit 105 uses the amplitude spectrum S (ω, t) obtained by the sound removal engine program 100 and the phase θm (ω, t) of the mixed sound m (t) to use Xs (ω, t). By obtaining t) and performing inverse Fourier transform (IFFT) on it, a unit waveform is obtained.

  Here, the frequency channel data after the subtraction at each time is subjected to inverse Fourier transform, but the phase of the sound of each channel at this time is assumed to have the same value as the phase of the known sound or the mixed sound before being erased. To. By this operation, the phase of the sound before erasure can be maintained, and it is possible to prevent the occurrence of noise “buchibuchi” for each section. Note that the same technique used to create time-frequency data is used for speeding up IFFT.

  The placement processing unit 106 uses the OverlapAdd method to superimpose the audio of each time having a Hanning window width of 170 milliseconds, and finally restores the audio from which the music has been erased. To do.

[Operation of the sound removal engine]
The sound removal engine program 100 having the above-described configuration operates as follows. FIG. 6A is a flowchart showing this operation.

  As shown in the figure, when known sound (original music) and mixed sound data are input (step S301), first, the data dividing unit 201 converts the mixed sound signal into a specific length (window size). Divide into sections. Here, 8192 samples (8192 ÷ 48,000 = 0.170 approximately 170 milliseconds), which is a power of 2.

  Next, in step S302, time-frequency data is acquired. Specifically, the window function processing unit 202 multiplies the audio signal data in the window size section (170 msec) by a Hanning function (S302a), and the Fourier transform unit 203 performs fast Fourier transform (FFT) (S302b). ). Then, the section for Fourier transform is moved in units of 480 samples (480 ÷ 48,000 = 0.01: 10 milliseconds) (S302c), and these steps S302a to c are repeated by loop processing.

  Various parameters are estimated for the “time frequency data” obtained every 10 milliseconds in this way. Specifically, the parameter estimation unit 300 selects, based on a user operation, a section from 1 second to several seconds (the above-described “BGM section”) in which only music (BGM) is sounded in the mixed sound. Calibration (S304) is performed. Select almost the same part for the original song.

  Next, the frequency channels are summed for each frequency channel in this section, the values obtained from the mixed sound and the removed sound (original music) are compared, and the frequency characteristic is acquired from the ratio (S305), and smoothing is performed ( S306).

  After that, the time change of the volume of the BGM is detected over the entire time range of the mixed sound, and the volume ratio of the BGM to the mixed sound for each frequency is acquired (S307). Based on this, the BGM in the mixed sound and the original music are obtained. It is determined whether or not the position is shifted in time (S309).

  If the BGM in the mixed sound specified in step S01 and the original music are shifted in time, the original music is shifted up and down by 100 milliseconds every 1 millisecond with respect to the mixed sound (S310). ), The processing from step S305 to step S308 is repeated. A difference in volume in this section is obtained, and it is considered that the difference coincides with the time when the difference becomes the smallest, so this is a time positional shift between the BGM in the mixed sound and the original music.

  After the positional deviation is corrected, the removal processing unit 104 performs a subtraction operation of the known sound estimated from the mixed sound (S311), and the Fourier transform unit 203 performs inverse Fourier transform on the frequency channel data after the subtraction at each time. (S312). At this time, the sound phase of each channel takes the same value as the phase of BGM in the mixed sound before erasure. Then, the data of each window size subjected to the inverse Fourier transform is overlapped by the OverlapAdd method in the placement processing unit (S313), and finally the sound from which the music is erased is restored and the data is output (S314). .

[editor]
(Editor configuration)
Next, the editor 400 that is a GUI of the known sound removal system will be described. FIG. 7 is an explanatory diagram showing the screen configuration of the editor 400.

  As shown in the drawing, the editor 400 displays a window 401 for displaying the waveform of the MIX file, a window 402 for displaying the waveform of the BGM file, a window 403 for displaying the spectrum of the MIX file, and a spectrum of the BGM file. A window 404, a window 405 for displaying a spectrum of voice after erasure and frequency-time characteristics, and a window 406 for displaying / manipulating erasure intensity and the like are provided. Each of these windows functions as a range setting unit that sets the range of the mixed acoustic signal to be subjected to the removal process.

  In the window 401 and the window 402, the waveform of the mix file and the BGM file is displayed, and can be enlarged / reduced (down / up) by moving the mouse up / down while holding down the right button of the mouse. By doing so, it is possible to scroll left and right. Furthermore, on this window, a section can be selected by moving left and right while pressing the left button. The selected section changes color, and the first and last times of the selection time are displayed. These operations can also be performed by using the direction keys on the keyboard.

  Further, by operating the operation button 401a or 402a at the right end, scrolling in the time axis direction, reproduction / reproduction stop of the selected section, and spectrum display of the selected section can be executed.

  In the windows 403 and 404, the spectrum display of the MIX file or the BGM file in the section selected in the window 401 or 402 is performed, and almost the same operation as the windows 401 and 402 can be performed. In addition, in the spectrum display here, the spectrum is displayed in 16 levels of intensity, the horizontal axis indicates time, and the vertical axis indicates frequency.

  In particular, in the window 403, when estimating the frequency distribution in the calibration process, a section in which only BGM can be heard is selected on this window, and the “SR” button is pressed to start and end positions of the selected section. Are input in the fields of CalibrationStartTime (BGM only section start position) and CalibrationEndTime (BGM only section end position) in the text box portion 407b. The input to the text box portion 407b includes direct numerical input.

  The window 405 displays the frequency distribution (frequency characteristics and amplitude) of the acoustic signal for each time as a line or figure, and the signal strength (volume) of the frequency is thermographed by color-coding the line or figure stepwise. When the radio button 407e is checked, the spectrum display after erasure and the spectrum display of the frequency-time characteristic are switched as shown in FIG. FIG. 10A is a spectrum display after erasure, and FIG. 10B is a spectrum display in which time-frequency characteristics are normalized. In this window 405, the same mouse and keyboard operations as those in the windows 401 to 404 described above can be performed.

  In the window 406, by checking the radio button part 407d, the erasing intensity curve (FIG. 9A), the frequency characteristic curve (FIG. 9B), the time characteristic curve (FIG. c)) and time-frequency characteristics ((d) in the figure) are switched and displayed. This window 406 functions as a setting unit for setting the signal intensity of a known acoustic amplitude spectrum to be removed at a corresponding time by deforming the curve, and moves to the right while pressing the left button. The function curve can be finely and arbitrarily adjusted. Note that even if the mouse is moved to the left, it cannot be changed.

  In each of these windows, a time code is described as a time axis. In this time code, 15 times from the acoustic signal end time of a predetermined frequency (here, 1 kHz of the color bar) detected from the mixed acoustic signal. It is automatically adjusted to zero after 2 seconds.

  In addition, the editor 400 includes two left and right operation panels 407 and 408 for displaying a file and setting various parameters.

  The left operation panel 407 is provided with a window 407a for displaying a MIX file, a text box section 407b for setting parameters, a check box section 407c, radio button sections 407d and 407e, and a delete execution button 407d. It has been. On the other hand, the right operation panel 408 includes a window 408a for displaying a BGM file and a slider unit 408b for setting parameters.

  In windows 407a and 408a, by dragging a MIX file or BGM file here, the waveform of the MIX file or BGM file is displayed in windows 401 and 402. If the dragged file is a video file, the video is displayed in this window. Is displayed. Below these windows 407a and 408a, a slide bar is arranged, and by sliding it, the playback start position of the MIX file can be changed. In this embodiment, the waveform display section is set to 5 minutes from the playback start position. Further, below the slide bar, a play button, a play pause button, a play stop button, a volume adjustment bar, and a text box for numerical input of the play start position are arranged.

  As shown in FIG. 8A, the text box portion 407b includes “EraseRatio” which is an input field for the erasing intensity of the BGM to be erased, and the start position and the end of the BGM only section of the MIX file for estimating the frequency distribution. “Calibration Start Time” and “Calibration Start Time” that are fields for inputting the position, and “Offset Between Target and BGM” that is a field for inputting the deviation of the start position of the selected section of the MIX file and the BGM file. Yes. In “EraseRatio”, the basic value is 1, and can be set to 0.1 to 5. In “Offset Between Target and BGM”, when −1 is set, a mode for calculating automatic alignment is set.

  In the check box unit 407c, as shown in FIG. 8B, in order to improve the estimation accuracy of the frequency distribution, in addition to the section specified by the user, other BGM only parts are searched for automatically. Re-estimation mode “PEM” to be added to the data, frequency blurring mode “BAFAM” that corrects by estimating from the value of the adjacent channel for the frequency channel whose estimation is erroneously small value, and the estimation is erroneously small value The time axis blurring mode “BATAM” that corrects the estimated time from the previous and next time values can be selected.

  In the radio button unit 407d, as shown in FIG. 8 (c), by selectively checking, the EraseRatio (intensity curve), FreqWeight in the window 406 as shown in FIGS. 9 (a) to 9 (d). (Frequency characteristic curve), TimeWeight (time characteristic curve), BGMWeight (time-frequency characteristic curve) display can be switched. In this radio button section 407d, when “e” is checked, the intensity curve is checked, when “f” is checked, the frequency characteristic curve is checked, when “t” is checked, the time characteristic curve is checked, and when “BGM” is checked, the time-frequency is checked. A characteristic curve is displayed in window 406, and parameters can be reset on each of these displays.

  In the radio button unit 407e, as shown in FIG. 8 (d), the spectrum and time after erasure in the window 405 as shown in FIGS. 10 (a) and 10 (b) can be selectively checked. -The display with the spectrum of the frequency characteristic can be switched. That is, in this radio button section 407e, if “Res” is checked, the spectrum of the data after deletion is displayed in the window 405, and if “BGM” is checked, the spectrum of the time-frequency characteristic is displayed.

  In the slider unit 408b, by sliding the bar left and right, the shift amount of the frequency characteristic curve is set by “Shift Freq Weight”, the bias amount of the time characteristic curve is set by “Shift Time Weight”, and the time − The smoothing width in the frequency direction can be changed with “Smoothing Time Weight” as the bias amount of the frequency characteristic. In the present embodiment, when these parameters are changed, the setting of TimeWeight is reset.

  In the button section 407f, music erasure processing is executed by clicking the “DEL_Music” button, and music erasure processing processing with readjusted characteristics is executed after erasing music by clicking the “RDEL_Music” button. To do.

(Editor change example)
The editor 400 which is the GUI described above can be configured as follows. FIG. 11 is a configuration diagram illustrating a modification example of the editor 400.

  Similarly to the editor 400 described above, the editor according to this modified example also calls a part of the function of the sound removal engine program 100 by the user's operation, so that all the parameter functions a (t), g (ω, t) (gω (ω, t), gt (t), gr (t)), p (ω), q (ω), r (t), c (ω, t), User can set manually. The user of the editor may draw and specify an arbitrary function shape from the beginning, or may first perform automatic estimation and modify the result.

  The editor is roughly divided into a sub-window W1 for operating the mixed acoustic signal m (t), a sub-window W2 for operating the known acoustic signal b ′ (t), and a desired acoustic signal s (t) after the known acoustic signal is removed. It consists of three sub-windows W3 for operation. When there are a plurality of known acoustic signals b '(t), the known acoustic signal b' (t) operated in the sub window W2 can be switched by the changeover switch W2S.

  First, functions common to all sub-windows will be described. The operation range slider P1 indicates where in the acoustic signal is currently displayed. The cursor P2 is an iconized (folding) button P3 indicating the position of the current operation target on the time axis. When the button P3 is pressed, the subwindow to which the button belongs is temporarily folded, and the remaining non-current operation target is reduced. The sub-window of use can be hidden to make effective use of a narrow screen. When the float (enlarge) button P4 is pressed, the sub-window to which the button belongs is temporarily separated from the parent window (float), and further expanded to facilitate operation / editing. ) When only the button P4 is drawn, when this button is pressed, the sub-window associated therewith is floated and appears. On the reproduction control panel P5, there are arranged a group of buttons that can reproduce, stop, fast-forward, and fast-reverse acoustic signals for humans to hear and confirm.

  In the sub-windows W1, W2, and W3, a graph E1 of the power of the mixed acoustic signal m (t), a graph E2 of the amplitude spectrum M (ω, t), a graph E3 of the power of the known acoustic signal b ′ (t) A graph E4 of the amplitude spectrum B ′ (ω, t), a graph E5 of the power of the acoustic signal s (t) after removal of the known acoustic signal, and a graph E6 of the amplitude spectrum S (ω, t) are displayed. In each amplitude spectrum, the amplitude is drawn with shading on the left side (the horizontal axis is the time axis, the vertical axis is the frequency axis), and the amplitude at the cursor position is drawn on the right side (the horizontal axis is power, the vertical axis is the frequency axis) ).

  The sub-window W2 for operating the known acoustic signal b ′ (t) is a window that is the center of the operation, and all the parameter functions a (t), g (ω, t) (gω (ω, t), gt (t), gr (t)), p (ω), q (t), and r (t) can be freely set. Hereinafter, description of each operation panel will be described.

1. Operation panel C1 for correcting the time variation of the frequency characteristics (right side of E7)
On the panel for displaying and operating gω (ω, t), gω (ω, t) at time t at the cursor position is drawn (the horizontal axis is the size and the vertical axis is the frequency axis). The setting operation result is immediately reflected on the display panel E7 of g (ω, t). In E7, the magnitude of the value of g (ω, t) is depicted in shading (the horizontal axis is the time axis, and the vertical axis is the frequency axis).

2. Operation panel C2 for correcting changes in volume over time (lower side of E7)
A panel for displaying and operating gt (t), and the setting operation result is immediately reflected on the display panel E7 for g (ω, t).

3. Operation panel C3 (lower side of E7) for raising the value of g (ω, t) as a whole
The setting operation result is immediately reflected on the display panel E7 of g (ω, t) on the panel for displaying and operating gr (t).

4). Operation panel C4 for finally adjusting the amount of subtraction of the component corresponding to the amplitude spectrum of the known acoustic signal from the amplitude spectrum of the mixed sound
It is a panel for displaying and operating a (t).

5). Operation panel C5 for correcting expansion and contraction in the frequency axis direction
This is a panel for displaying and operating p (ω).

6). Operation panel C6 for correcting expansion and contraction in the time axis direction
This is a panel for displaying and operating q (t).

7). Operation panel C7 for correcting a positional shift in time
It is a panel for displaying and operating r (t).

  In the sub-window W3 for operating the acoustic signal s (t) after removing the known acoustic signal, the shape of the parameter function c (ω, t) in Equation 5 can be freely set. Hereinafter, description of each operation panel will be described.

1. Graphic equalizer (GEQ) operation panel C8 (right side of E8)
This is a panel for displaying and operating the shape of c (ω, t) in the ω direction, and c (ω, t) at time t at the cursor position is drawn (the horizontal axis is the size, the vertical axis is the frequency axis) ). The setting operation result is immediately reflected on the display panel E8 of c (ω, t). In E8, the magnitude of the value of c (ω, t) is depicted in shading (the horizontal axis is the time axis, and the vertical axis is the frequency axis).

2. Volume fader operation panel C9 (below E8)
This is a panel for displaying / manipulating the shape of c (ω, t) in the t direction, and the setting operation result is immediately reflected on the display panel E8 of c (ω, t).

  In this editor, not only audio files can be input and output, but also the shape of various parameter functions can be input and output to the file. As a result, the removal operation can be interrupted / resumed. In order to achieve a high-speed response to the user, GUI, signal processing, and sound reproduction are implemented as different threads based on multi-threads, and waiting time such as time-consuming signal processing may be hidden.

[Computer-readable recording medium recording the program]
In addition, in the known sound removal program according to the above-described embodiment and its modification example, it is possible to easily construct a device or system having each of the above-described functions by being installed in a computer such as a user terminal or a Web server or an IC chip. Can do. This program can be distributed through a communication line, for example, and can be transferred as a package application that operates on a stand-alone computer.

  Such a program can be recorded on recording media 116 to 119 readable by the general-purpose computer 120 as shown in FIG. Specifically, as shown in the figure, recording is performed on various recording media such as a RAM card 118 in addition to a magnetic recording medium such as a flexible disk 116 and a cassette tape 119, or an optical disk such as a CD-ROM and a DVD-ROM 117. can do.

  According to the computer-readable recording medium in which the program is recorded, the above-described content display system and method can be implemented using a general-purpose computer or a dedicated computer, and the program can be stored, transported, and Easy installation.

[Operations and effects in the embodiment]
According to the present embodiment described above, the amplitude data does not change even if the phase changes, so that processing independent of the phase is possible. Therefore, for example, only music can be erased from the audio signal of a program in which audio and music are mixed, using sound data such as a music CD used at the time of program creation.

  Especially when creating program audio, even if the frequency characteristics and volume are adjusted according to the production intention, and the phase of the sound changes unpredictably, the change of music in the mixed sound is automatically and Predict with high accuracy and erase without depending on the phase.

  In the present embodiment, when setting the range in the sound related to the removal process, the synchronization control unit 9 synchronizes the video and the audio and outputs them from the monitor 10 and the speaker 11. The operation can be performed while visually confirming the sound against the image, and the work efficiency can be improved.

  In this embodiment, since the time change graph is displayed and can be corrected by an intuitive operation that the user draws with a mouse, music can be changed based on the user's intention such as considering each scene of the program and the reuse method. The effect of erasing can be adjusted.

[Example of change]
Note that, for example, the following functions can be added to the system according to the above-described embodiment.

(Modification 1)
In the standard format of commercial broadcasting, the audio start position is determined, and it is necessary to proceed based on information such as cue sheets that describe the time and songs used in the program. A function is necessary.

  Therefore, in this modified example, as shown in FIG. 13A, an automatic time adjustment function 12 is provided in the system. As shown in FIG. 13B, the automatic time adjustment function 12 uses a window of the editor 400 or the like to perform a range of the mixed sound signal to be removed by the window of the editor 400 or the like when performing processing from the on-air tape of the TV. , The detection unit detects a predetermined frequency (in this case, 1 kHz of the color bar) of the mixed acoustic data related to the setting of the range, and measures with a timer 15 seconds after the end of the acoustic signal at the predetermined frequency. This is a module for setting the time code to 0 by the setting unit. Thus, the video and audio of the target portion can be extracted simply by inputting the time of the music to be erased described on the Q sheet.

  According to such an automatic time adjustment function 12, it is possible to adjust the time code from the color bar signal of the standard format for commercial use in the processing from the on-air tape of the TV, which is described in the Q sheet. Just by inputting the time of the music to be erased, the video and audio of the target part can be extracted.

(Modification 2)
As described in the above-described embodiments, some broadcast programs have a stereo system including left and right sounds. However, since the sound removal process must be performed for each of the left and right in monaural, there is a problem that the work of the removal process that requires setting of many parameters is doubled for stereo broadcasting.

  Therefore, in this modified example, as shown in FIG. 14A, a parameter storage unit 13 is provided in the system. As shown in FIG. 14B, the parameter storage unit 13 stores various parameter setting data for the left audio when, for example, the left audio (L) of stereo broadcasting is operated in the first operation. Remember. That is, when various parameters are set by the user interface 6 for the left audio separated and extracted by the audio data extraction unit 3, the setting data for the left audio is stored in the parameter storage unit 13, and this is stored in the next right Called during the voice removal process and set as a default value in the user interface 6.

  According to the parameter storage unit 13, when processing content data including left and right audio data, the settings for one of the left and right audios are stored and stored when the work for the other audio is performed. Therefore, work efficiency can be improved.

(Modification 3)
There is a time shift of the BGM in the mixed sound with the known sound source, and it is difficult to accurately match the time of the mixed sound and the BGM.

  Therefore, in this modified example, as shown in FIG. 15, a range setting unit 15 is provided in the system. The range setting unit 15 performs a process of narrowing down the range step by step when setting a time position shift between the BGM in the mixed sound and the known sound to be removed. The range setting unit 15 is linked to the user interface 6 and operates when the temporal positional deviation is automatically estimated. As a result, when setting the range, first, the position adjustment is performed in a wider range, and the range is sequentially closed up, so that fine adjustment is possible.

  In addition, the range setting unit 15 has a function for detecting a melody, whereby the frequency distribution can be estimated. That is, the range setting unit 15 detects the frequency distribution according to the melody included in the music when the known acoustic signal is a signal related to music, and based on the detected signal start position. Set the relative time position. According to such a range setting unit 15, when the known sound to be removed is music or the like, relative positioning can be made easier by detecting the melody.

  Further, in the present modification example, the headphones 16 are connected to the output I / F 8. The headphone 16 outputs a BGM sound source that is a known sound from the left speaker 16a, and outputs a MIX sound source that is a mixed sound from the left speaker 16b. As a result, the user's ear listens while comparing these sounds and aligns with the user's hearing.

  According to such a headphone 16, the mixed sound and the known sound are output from the left and right speakers of the headphone, for example, so that the user can determine the time lag using the auditory sense and improve the work efficiency. Can be made.

(Modification 4)
In order to erase the known sound, it is necessary to perform calibration to detect the characteristics of the known sound in the mixed sound. However, the known sound in the mixed sound is emphasized or attenuated at the time of program creation, And its characteristics have changed. Therefore, even if a sample at only one place is acquired during the calibration, there is a possibility that the removal process cannot be performed properly. In addition, when the known sound is music or the like, it may be partially expanded or contracted in the time axis or frequency axis direction due to the difference in the rotation speed of the record or the like. Can't get the right sample.

  Therefore, in this modified example, as shown in FIG. 16A, a calibration setting unit 17 is provided in the system. As shown in FIG. 16B, the calibration setting unit 17 is a module for acquiring samples for calibration in the above-described removal engine program 100 from a plurality of locations arbitrarily selected by the user. The plurality of samples acquired by the calibration setting unit 17 is also used for the above-described estimation of the known sound. Using this sample, the known sound is expanded and contracted in the time axis or frequency axis direction to correct the mixed sound. Subtract known acoustic signal.

  According to such a calibration setting unit 17, in order to obtain calibration samples from a plurality of locations in the mixed sound, the average, standard deviation, and representative value are obtained from the data obtained from the obtained plurality of locations. Etc. can be performed with higher accuracy.

(Modification 5)
When creating audio for a broadcast program, frequency characteristics and volume are often adjusted according to the production intention. For this reason, an unpredictable change in the phase of the sound occurs, and it cannot be erased appropriately by simply performing electronic subtraction.

  Therefore, in this modified example, as shown in FIG. 17A, the sound collection microphone 19 can be connected to the input I / F 1, and the video camera 20 can be connected to the DV capture 2. ing. The sound collection microphone 19 has directivity and is a first sound collection device that acquires a known sound signal to be removed, and the video camera 20 is a second sound collection device that acquires a mixed sound signal. . The sound collecting microphone 19 can be carried away from the video camera 20, and is set near the sound source of the noise to be erased while the video camera 20 is shooting, and only the noise is recorded.

  Note that the sound collecting microphone 19 and the video camera 20 described above may have an integral structure as indicated by 21 in FIG. In this case, the second sound collecting device 21a that acquires the mixed sound signal is disposed at the tip of the integrated microphone 21, and the first sound collecting device 21b that acquires the known sound signal is located behind the integrated microphone 21. To place. The sound collecting devices 21a and 21b are spaced apart from each other, the sound collecting device 21a has directivity in front of the device for normal photographing, and the sound collecting device 21b has directivity other than the front. Constitute.

  According to this modified example, for example, only noise is accurately detected using sound data such as noise recorded by the sound collector 21b from a sound signal mixed with noise recorded by the sound collector 21a. Can be erased.

It is a block diagram which shows the structure of the known sound removal system which concerns on embodiment. It is a flowchart which shows operation | movement of the known sound removal system which concerns on embodiment. It is a flowchart which shows the basic theory of the known sound removal method which concerns on embodiment. It is explanatory drawing which shows the effect by the known sound removal method which concerns on embodiment. It is a functional block diagram of the sound removal engine program which concerns on embodiment. It is a flowchart which shows operation | movement of the sound removal engine program which concerns on embodiment. It is explanatory drawing which shows the structure of the editor which concerns on embodiment. It is explanatory drawing which shows the operation panel of the editor which concerns on embodiment. It is explanatory drawing which shows the window of the editor which concerns on embodiment. It is explanatory drawing which shows the window of the editor which concerns on embodiment. It is explanatory drawing which shows the structure of the editor which concerns on the example of a change. It is a perspective view which shows the computer-readable recording medium which recorded the program which concerns on embodiment. It is explanatory drawing which shows a structure and operation | movement of the known sound removal system which concerns on the example 1 of a change. It is explanatory drawing which shows a structure and operation | movement of the known sound removal system which concerns on the example 2 of a change. It is explanatory drawing which shows a structure and operation | movement of the known sound removal system which concerns on the example 3 of a change. It is explanatory drawing which shows the structure and operation | movement of the known sound removal system which concerns on the example 4 of a change. It is explanatory drawing which shows a structure and operation | movement of the known sound removal system which concerns on the example 5 of a change.

Explanation of symbols

  DESCRIPTION OF SYMBOLS 1 ... Input I / F, 2 ... DV capture, 3 ... Audio | voice data extraction part, 4 ... Audio | voice conversion part, 5 ... Storage device, 6 ... User interface, 6a ... Keyboard, 6b ... Mouse, 7 ... Memory, 8 ... Output I / F, 9 ... Synchronization control unit, 10 ... Monitor, 11 ... Speaker, 12 ... Time adjustment unit, 13 ... Parameter storage unit, 14 ... Simulation unit, 15 ... Range setting unit, 16 ... Headphone, 17 ... Calibration setting , 18 ... stationary sound detection unit, 19 ... sound collecting microphone, 20 ... video camera, 21 ... integrated microphone, 100 ... sound removal engine program, 101 ... mixed sound input unit, 102 ... known sound signal input unit, 103 ... Control unit 104 ... removal processing unit 105 ... inverse Fourier transform unit 106 ... placement processing unit 107 ... acoustic signal output unit after removal 116-119 ... recording medium 120 General-purpose computer, 200 ... amplitude spectrum extraction unit, 201 ... data division unit, 202 ... window function processing unit, 203 ... Fourier transform unit, 300 ... parameter estimation unit, 301 ... frequency characteristic change correction unit, 302 ... volume change correction unit, 303: time position correction unit, 304: calibration unit, 400: editor

Claims (15)

A stationary sound detection unit for detecting stationary sound in a mixed acoustic signal obtained by mixing a known acoustic signal to be removed and another acoustic signal;
A known acoustic amplitude extraction unit for extracting a known acoustic amplitude spectrum from the known acoustic signal;
A mixed acoustic amplitude extraction unit that extracts a mixed acoustic amplitude spectrum for each frequency from the mixed acoustic signal;
Of the extracted mixed acoustic amplitude spectrum, a frequency selection unit that selects a mixed acoustic amplitude spectrum having a frequency that does not match the frequency of the stationary sound;
A removal processing unit for removing the known acoustic amplitude spectrum from the mixed acoustic amplitude spectrum of the selected frequency;
An acoustic signal removing device comprising:   The acoustic signal removing apparatus according to claim 1, wherein the stationary sound detection unit detects a frequency having a minimum amplitude in the mixed acoustic signal as a stationary sound.   The said stationary sound detection part acquires the amplitude for every time in the said mixed acoustic signal, sorts in order of the value, and detects the frequency used as the nth value as a stationary sound. The acoustic signal removal apparatus as described.   The stationary sound detection unit acquires amplitudes for each time in the mixed acoustic signal, sorts the amplitudes in order of the values, obtains a maximum value of n not exceeding a certain value from the nth standard deviation, and determines the nth The acoustic signal removing device according to claim 1, wherein a frequency having a value of is detected as a stationary sound. An acoustic signal removing device that removes a known acoustic signal from a mixed acoustic signal obtained by mixing a known acoustic signal to be removed and another acoustic signal,
A simulation for performing a pseudo subtraction process on the sound signal having a constant amplitude as the mixed sound with the known sound signal set to 0, and measuring a difference in volume between the sound signal having the constant amplitude and the sound signal after the pseudo subtraction process. And
Based on the measurement result by the simulation unit, a removal intensity setting unit that sets a signal intensity for each time of the known acoustic signal,
A known acoustic amplitude extraction unit for extracting a known acoustic amplitude spectrum from the known acoustic signal;
A mixed acoustic amplitude extraction unit that extracts a mixed acoustic amplitude spectrum from the mixed acoustic signal;
Based on the setting by the removal intensity setting unit, the removal processing unit that converts the known acoustic amplitude spectrum and removes the known acoustic amplitude spectrum from the mixed acoustic amplitude spectrum;
An acoustic signal removing device comprising: Detecting a stationary sound in a mixed acoustic signal obtained by mixing a known acoustic signal to be removed and another acoustic signal;
Extracting a known acoustic amplitude spectrum from the known acoustic signal and extracting a mixed acoustic amplitude spectrum for each frequency from the mixed acoustic signal;
Selecting a mixed acoustic amplitude spectrum having a frequency that does not match the frequency of the stationary sound from the extracted mixed acoustic amplitude spectrum;
Removing the known acoustic amplitude spectrum from a mixed acoustic amplitude spectrum at a selected frequency;
An acoustic signal removing method comprising:   The acoustic signal removal method according to claim 6, wherein a frequency having a minimum amplitude in the mixed acoustic signal is detected as a stationary sound.   The acoustic signal removal method according to claim 6, wherein amplitudes for each time in the mixed acoustic signal are acquired, sorted in the order of the values, and the frequency that becomes the n-th value is detected as a stationary sound.   In the mixed acoustic signal, the amplitude for each time is acquired, sorted in the order of the values, the maximum value of n not exceeding a certain value is obtained from the standard deviation up to the nth, and the frequency that becomes the nth value is steady. The acoustic signal removing method according to claim 6, wherein the acoustic signal is detected as sound. An acoustic signal removal method for removing a known acoustic signal from a mixed acoustic signal obtained by mixing a known acoustic signal to be removed and another acoustic signal,
Performing a pseudo-subtraction process on the acoustic signal having a constant amplitude as the mixed sound with the known acoustic signal set to 0, and measuring a difference in volume between the acoustic signal having the constant amplitude and the acoustic signal after the pseudo-subtraction process When,
Based on the measurement result, setting a signal intensity for each time of the known acoustic signal;
Extracting a known acoustic amplitude spectrum from the known acoustic signal and extracting a mixed acoustic amplitude spectrum from the mixed acoustic signal;
Converting the known acoustic amplitude spectrum based on the setting and removing the known acoustic amplitude spectrum from the mixed acoustic amplitude spectrum;
An acoustic signal removing method comprising: On the computer,
Detecting a stationary sound in a mixed acoustic signal obtained by mixing a known acoustic signal to be removed and another acoustic signal;
Extracting a known acoustic amplitude spectrum from the known acoustic signal and extracting a mixed acoustic amplitude spectrum for each frequency from the mixed acoustic signal;
Selecting a mixed acoustic amplitude spectrum having a frequency that does not match the frequency of the stationary sound from the extracted mixed acoustic amplitude spectrum;
Removing the known acoustic amplitude spectrum from a mixed acoustic amplitude spectrum at a selected frequency;
An acoustic signal removal program characterized by causing the processing to be executed.   The acoustic signal removal program according to claim 11, wherein a frequency having a minimum amplitude in the mixed acoustic signal is detected as a stationary sound.   The acoustic signal removal program according to claim 11, wherein amplitudes for each time in the mixed acoustic signal are acquired, sorted in order of values, and a frequency that becomes an nth value is detected as a stationary sound.   In the mixed acoustic signal, the amplitude for each time is acquired, sorted in the order of the values, the maximum value of n not exceeding a certain value is obtained from the standard deviation up to the nth, and the frequency that becomes the nth value is steady. The sound signal removal program according to claim 11, wherein the sound signal removal program is detected as sound. An acoustic signal removal program for removing a known acoustic signal from a mixed acoustic signal obtained by mixing a known acoustic signal to be removed and another acoustic signal,
On the computer,
Performing a pseudo-subtraction process on the acoustic signal of constant amplitude as the mixed sound with the known acoustic signal set to 0, and measuring a difference in volume between the acoustic signal of constant amplitude and the acoustic signal after the pseudo-subtraction process When,
Based on the measurement result, setting a signal intensity for each time of the known acoustic signal;
Extracting a known acoustic amplitude spectrum from the known acoustic signal and extracting a mixed acoustic amplitude spectrum from the mixed acoustic signal;
Converting the known acoustic amplitude spectrum based on the setting and removing the known acoustic amplitude spectrum from the mixed acoustic amplitude spectrum;
An acoustic signal removal program characterized by causing a process comprising:

Images