JP2005266797A - Method and apparatus for separating sound-source signal and method and device for detecting pitch - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To pick up signals from a plurality of sound sources by stereo microphones and to separate a desired sound source signal. <P>SOLUTION: A stereo speech from a terminal 11 is inputted to a pitch detection part 12 to performs pitch detection based upon a 2-wavelength part of a pitch as a detection unit, thereby detecting a pitch and a stationary part where the same pitches are successive. A delay correcting addition part 13 delays and corrects the stereo speech so that the speech is in phase with a speech from a desired sound source, and adds them together to output a signal whose desired sound source signal is emphasized. A separation coefficient generation part 14 in a sound source signal separation part 19 generates a filter coefficient of a filter arithmetic circuit 15 in the sound source signal separation part 19 according to the pitch detected by the pitch detection part 12. The filter coefficient generated by the separation coefficient generation part 14 is sent to the filter arithmetic part 15, which filters the output signal from the delay correcting addition part 13 with a filter coefficient updated by detected pitches to obtain a waveform output after the desired sound source signal is separated. <P>COPYRIGHT: (C)2005,JPO&NCIPI

Description

  The present invention relates to a sound source signal separation device and method, and a pitch detection device and method, for example, a sound source signal separation device and method for separating sound signals from a plurality of sound sources with a stereo microphone, and sound source signal separation. The present invention relates to a pitch detection apparatus and method for performing suitable pitch detection.

  It is known to separate a desired sound source signal from an acoustic signal in which a plurality of types of sound source signals are mixed. For example, as shown in FIG. 26, sound generated from a plurality of persons, for example, three persons SPA, SPB, SPC is collected by an acoustic-electric conversion means, for example, left and right stereo microphones MCL, MCR. The technique is to separate the audio signal from a desired single person from the obtained acoustic signal.

  As a conventional technique of such sound source signal separation, there is an acoustic signal separation circuit disclosed in Patent Document 1 and a microphone device using the same. In these acoustic signal separation circuits and microphone devices using the acoustic signal separation circuits, a plurality of mixed signals obtained by linearly adding a plurality of linearly independent sound source signals are divided into frames, and a plurality of signals separated by a separation circuit for each frame. The original speech signal is separated from the mixed signal by multiplying the inverse matrix of the mixing matrix that minimizes the correlation with zero lag time between them.

  Patent Document 2 discloses a sound source signal estimation apparatus for estimating a desired sound source, which is used when a desired sound signal is extracted in an environment where there is a lot of noise in the surroundings.

  Further, it is considered that the pitch of the target sound is obtained for the separation of the sound source signal. As a technique for detecting the pitch, an acoustic signal analysis method and apparatus and an audio signal processing method and apparatus disclosed in Patent Document 3 are proposed. There is. In these devices and methods, an input signal is cut out for each frame having a predetermined time length, frequency analysis is performed for each frame, and harmonic characteristics are evaluated in each frame from the frequency analysis result of each frame. In addition, harmonic evaluation is performed on the difference between the amplitudes of the frequency analysis results of each frame, and the pitch of the input signal is detected using the results of the harmonic evaluation.

JP 2001-222289 A Japanese Patent Laid-Open No. 7-28492 JP 2000-181499 A

  Generally, in order to separate a plurality of sound sources, more microphones than the number of sound sources are required, and studies using such a plurality of microphones are being conducted. For example, Patent Document 1 described above discloses that only two sound sources can be separated from two microphones. Patent Document 2 discloses a technique for extracting an audio signal from a target sound source using a plurality of microphones (microphone arrays). In these techniques, in order to separate a desired sound source signal from a mixed signal obtained by mixing a plurality of sound source signals, it is necessary to use as many microphones (multi-microphones) as the number of sound sources.

  Therefore, according to such a conventional technique, it is difficult to separate sound source signals of three or more sound sources in the case of a stereo microphone used in a portable AV device such as a camera-integrated VTR (so-called video camera). is there.

  In addition, when obtaining the pitch of the target sound prior to separating the sound source signal, it is desired to detect the pitch suitable for separating the sound source signal.

  The present invention has been proposed in view of such conventional circumstances, and uses a small number of sound collecting means such as stereo microphones to generate audio signals (generally acoustic signals) from a plurality of sound sources. ), And a sound source signal separation device and method, and a pitch detection device and method that can effectively separate a sound signal from a desired desired sound source.

  In order to solve the above-described problem, a sound source signal separation device according to the present invention is a desired sound source signal among input sound signals obtained by mixing sound signals from a plurality of sound sources and collected by a plurality of sound collecting means. Sound source signal emphasizing means for emphasizing the sound, pitch detecting means for detecting the pitch of the desired sound source signal in the input sound signal, and the detected sound source signal and the sound source signal emphasized by the sound source signal emphasizing means. And a sound source signal separating means for separating the desired sound source signal from the input sound signal.

  As an example of the sound source signal separating means, a filter means for separating the desired sound source signal from an output signal from the sound source signal enhancing means, and a filter coefficient of the filter means based on detection information from the pitch detecting means. And a filter coefficient output means for outputting.

  Here, it is preferable that the filter coefficient output means outputs a filter coefficient having a characteristic that allows the frequency characteristic of the filter means to pass a frequency component that is an integral multiple of the frequency of the pitch detected by the pitch detection means. The filter coefficient output means includes storage means in which filter coefficients corresponding to several types of pitches are stored in advance, and corresponds to the pitch from the storage means according to the pitch detected by the pitch detection means. It is preferable to read out and output the filter coefficients.

  Further, the high frequency processing means for processing the consonant band of the output signal from the sound source signal enhancing means, and the consonant band of the output signal from the sound source signal enhancing means are extracted and sent to the high frequency processing means, and the sound source signal enhancement is performed. It is preferable to further include filter bank means for taking out a band other than the consonant of the output signal from the means and sending it to the filter means and taking out the vowel band of the output signal from the sound source signal emphasizing means and sending it to the pitch detection means. .

  The plurality of sound collecting means may be left and right stereo microphones. Further, the sound source signal emphasizing unit corrects and adds a delay time difference of sound propagation from the desired sound source to the plurality of sound collecting units to the sound signals from the plurality of sound collecting units. It is preferable to emphasize only the acoustic signal from the desired sound source. Furthermore, it is preferable that the pitch detection means performs pitch detection using two wavelengths of the pitch of the desired sound source signal as detection units.

  Further, as another example of the sound source signal separating means, a continuity portion having the same or substantially the same pitch in the output signal from the sound source signal emphasizing means is used, and based on detection information from the pitch detecting means, Basic waveform creation means for creating a waveform, and basic waveform replacement means for replacing at least a part of the signal based on the input acoustic signal with a repeated waveform of the basic waveform created by the basic waveform creation means and outputting the waveform It is characterized by.

  Here, it is preferable that the pitch detection means performs pitch detection using two wavelengths of the pitch of the desired sound source signal as detection units. The plurality of sound collecting means may be left and right stereo microphones. Further, the sound source signal emphasizing unit corrects and adds a delay time difference of sound propagation from the desired sound source to the plurality of sound collecting units to the sound signals from the plurality of sound collecting units. It is preferable to emphasize only the acoustic signal from the desired sound source. Further, it is preferable that the basic waveform creating means creates a basic waveform by adding and averaging the two portions of the pitch for the stationary portion where the pitch of the desired sound source signal is continuous.

  Next, in order to achieve the above object, the audio signal separation method according to the present invention mixes acoustic signals from a plurality of sound sources and collects a desired sound source among the input sound signals collected by a plurality of sound collecting means. The input based on the step of enhancing the signal, the step of detecting the pitch of the desired sound source signal in the input acoustic signal, and the sound source signal emphasized in the step of enhancing and the detected pitch. Separating the desired sound source signal from the acoustic signal.

  Next, in order to achieve the above object, the pitch detection apparatus according to the present invention emphasizes a desired sound source signal of an input sound signal that is obtained by mixing sound signals from a plurality of sound sources and collected by a plurality of sound collecting means. Sound source signal emphasizing means, period detecting means for detecting a two-wavelength period with two wavelengths of a pitch in the output signal from the sound source emphasizing means as a detection unit, and a change in the two-wavelength period detected by the period detecting means And a continuous determination means for determining whether or not the same or substantially the same pitch is continuous, and outputting pitch information in accordance with the determination result.

  Here, the plurality of sound collecting means may be left and right stereo microphones. Further, the sound source signal emphasizing unit corrects and adds a delay time difference of sound propagation from the desired sound source to the plurality of sound collecting units to the sound signals from the plurality of sound collecting units. It is preferable to emphasize only the acoustic signal from the desired sound source.

  In addition, in order to achieve the above object, the pitch detection method according to the present invention emphasizes a desired sound source signal of an input sound signal obtained by mixing sound signals from a plurality of sound sources and collected by a plurality of sound collecting means. A sound source signal emphasizing step, a period detecting step for detecting a two-wavelength period using two wavelengths of pitch in the output signal obtained by the sound source emphasizing step as a detection unit, and a change in two wavelength periods detected by the period detecting step And a continuous determination step of determining whether or not the same or substantially the same pitch is continuous, and outputting pitch information according to the determination result.

  Next, in order to achieve the above object, the sound source signal separation device according to the present invention has a wavelength component that is a multiple of two times the pitch of a desired sound source signal of an input acoustic signal obtained by mixing acoustic signals from a plurality of sound sources. Pitch detection means for performing pitch detection as a detection unit and sound source signal separation means for separating a desired sound source signal based on the detected pitch are provided.

  Furthermore, the sound source signal separation method according to the present invention detects the wavelength component of a multiple of 2 of the pitch of the desired sound source signal of the input sound signal obtained by mixing the sound signals from a plurality of sound sources in order to achieve the above object. The method includes a step of performing pitch detection as a unit and a step of separating a desired sound source signal based on the detected pitch.

  According to the present invention, with respect to a filter for separating a desired sound source signal from among input sound signals collected by a plurality of sound collecting means by mixing sound signals from a plurality of sound sources, the pitch of the input sound signal Since the filter coefficient is updated according to the detected pitch, the sound from the target sound source can be separated satisfactorily.

  Further, according to the present invention, since a basic waveform based on the stationary part of the pitch of the input signal is created and replaced, a good sound source signal approximating the sound from the target sound source can be separated.

  Furthermore, by performing pitch detection using the two wavelengths of the pitch as a detection unit, reliable and stable pitch detection can be performed.

  Hereinafter, specific embodiments to which the present invention is applied will be described in detail with reference to the drawings.

  A schematic configuration of a specific example of the sound source signal separation device used in the embodiment of the present invention is shown in FIG.

  In FIG. 1, an acoustic signal collected by a microphone or the like, specifically, for example, a stereo audio signal collected by a stereo microphone, is input to an input terminal 11, and a pitch detector 12 and a desired sound source signal are input to the input terminal 11. The signal is sent to a delay correction adding unit 13 as a sound source signal emphasizing means to be emphasized. The output from the pitch detection unit 12 is sent to the separation coefficient creation unit 14 in the sound source signal separation unit 19, and the output from the delay correction addition unit 13 is a filter that outputs a frequency band equal to or lower than the middle range as necessary ( The signal is sent to the filter arithmetic circuit 15 in the sound source signal separation unit 19 via the low-pass filter) 20A. The filter calculation circuit 15 is a filter that separates a desired target voice. Every time the pitch detected by the pitch detection unit 12 is updated, the separation coefficient creating unit 14 that is a separation coefficient output unit detects the detected pitch. The filter coefficient corresponding to is generated and sent to the filter arithmetic circuit 15. The output from the delay correction addition unit 13 is sent to the high frequency processing unit 17 via a filter (high pass filter) 20B that passes a high frequency band as necessary, and is output to an unsteady waveform such as a consonant. Processing is performed. The output from the filter arithmetic circuit 15 and the output from the high frequency processing unit 17 are added by the adder 16 and are taken out from the output terminal 18 as a separated waveform output signal.

  In a specific example of the sound source signal separation device having such a configuration, the pitch detection unit 12 is configured such that the pitch (sound pitch) of a stationary part, which is a part where the same or substantially the same pitch is continued, such as a vowel in an audio signal. ) Is detected, and the detected pitch is output from the pitch detector 12 and, if necessary, information indicating the stationary part (for example, coordinate information on a time axis indicating a continuous section). Is output. The delay correction adding unit 13 is used as an example of a sound source signal emphasizing unit that emphasizes a desired sound source signal, and a propagation delay time corresponding to the distance from the sound source to a plurality of (two in the case of stereo) microphones. Depending on the difference, the signals from the microphones are added with a time delay to strengthen the signal from the desired sound source and weaken the other signals. Details will be described later. The separation coefficient creation unit 14 creates a filter coefficient for separating a signal from a desired sound source in accordance with the pitch of the stationary part detected by the pitch detection unit 12, and details will be described later. The filter arithmetic circuit 15 performs a filtering process on the output from the delay correction addition unit 13 (the output via the filter (low-pass filter) 20A as necessary) using the filter coefficient from the separation coefficient creation unit 14 to obtain a desired value. The signal from the sound source is separated. The high-frequency processing unit 17 applies an unsteady waveform such as a consonant of a signal via a filter (high-pass filter) 20B that passes a high-frequency to the output from the delay correction adding unit 13 as necessary. Predetermined processing is performed and output to the adder 16. The adder 16 adds the output from the filter arithmetic circuit 15 and the output from the high frequency processing unit 17 and sends the result to the output terminal 18 as a separated waveform output signal of the target sound.

  Next, a schematic configuration of a specific example of the pitch detector 12 is shown in FIG. The input terminal 21 in FIG. 2 corresponds to the input terminal 11 in FIG. 1 and receives, for example, a stereo sound signal collected by a stereo microphone. For example, a signal from a desired sound source is emphasized as will be described later, which is sent to a delay correction adding unit 23 via a low pass filter (LPF) 22 for passing a vowel band, for example, in which the pitch constantly appears. Directivity control processing is performed. The output from the delay correction addition unit 23 is sent to the maximum value pitch detection unit 26 via the maximum value detection unit 24 and via the maximum value between zero crosses detection unit 25 of the maximum value. The output from the maximum value pitch detection unit 26 is sent to the continuation determination unit 27, and the representative pitch output is taken out from the terminal 28, and the coordinate (time) output indicating the section of the stationary part is taken out from the terminal 29.

  Here, a principle configuration example of the delay correction addition unit 13 of FIG. 1 or the delay correction addition unit 23 of FIG. 2 will be described with reference to FIG. In FIG. 3, signals from left and right stereo microphones MCL and MCR are sent to delay circuits 32L and 32R using a buffer memory or the like for delaying the left and right stereo signals, respectively. In the case of the delay correction adding unit 23 of FIG. 2 above, in order to improve the quality of pitch detection, the left and right stereo signals are passed through a low pass filter (LPF) 22 for passing a band such as a vowel in the audio signal. It may be sent to the delay circuits 32L and 32R of the delay correction adder later. The delay signals from these delay circuits 32L and 32R are added by an adder 34 and taken out from an output terminal 35 as a delay correction addition signal. If necessary, the delay signals from the delay circuits 32L and 32R may be subtracted by the subtracter 36 and taken out from the output terminal 37 as a delay correction subtraction signal.

  The delay correction adder having the principle configuration as shown in FIG. 3 performs directivity control processing that enhances only the audio signal from the target sound source to be separated and attenuates other signal components. Is. In the example of FIG. 3, when the sound source SL on the left side, the sound source SC on the center, and the sound source SR on the right side are arranged with respect to the stereo microphones MCL and MCR, for example, when the right sound source SR is used as the target sound source, The sound emitted from the sound source SR has a time delay required for propagating in the air, so that the time (physical delay amount) τ is greater in the microphone MCL far from the sound source than in the microphone MCR closer to the sound source. Only after a delay is collected. At this time, the delay amount from the delay circuits 32L and 32R is corrected by setting the delay amount of the delay circuit 32L longer by the time τ than the delay circuit 32R with respect to the delay circuits 32L and 32R using the buffer memory or the like. As shown in FIG. 4, the output signal thus obtained has a high correlation coefficient between the left and right signals for the target sound from the target sound source SR (phases are more consistent), and the correlation coefficient for the other sounds. Lower (phases are more inconsistent). When the central sound source SC is the target sound source, the sound emitted from the sound source SC is simultaneously collected (without a delay time difference) by the stereo microphones MCL and MCR, and therefore the delay amounts of the delay circuits 32L and 32R are collected. By making these equal, the correlation of the target sound from the sound source SC can be increased, and the correlation of other sounds can be decreased. In this way, by adjusting the delay amounts of the delay circuits 32L and 32R, it is possible to increase the correlation only for the sound from the target sound source.

  Therefore, by adding the delayed output signals from the delay circuits 32L and 32R by the adder 34, only highly correlated speech is enhanced. In particular, in a repetitive waveform portion such as a vowel portion, a portion having a uniform phase is emphasized by adding waveforms having a uniform phase, and a portion having a non-uniform phase is attenuated. From the output terminal 35, a signal in which only the target sound is enhanced or emphasized is taken out. In addition, when the delayed output signals from the delay circuits 32L and 32R are subtracted by the subtractor 36, only the sound from the target sound source is attenuated because the portion having the same phase is subtracted. A signal in which only the target sound is attenuated is taken out from the terminal 37.

  Explaining the correlation coefficient, the waveform whose delay amount is corrected as described above with respect to the sound input to the two microphones has a high degree of coincidence of the waveform, and conversely, as with other sounds, the phase The shifted waveform has a low degree of coincidence. The correlation coefficient cor representing the degree of coincidence can be obtained by the following equation (1). In the equation (1), m1 and m2 indicate time samples of the stereo microphones MCL and MCR, and n pairs of sample values (m11, m21), (m12, m22),..., (M1n, m2n) ) For the correlation coefficient cor. S1 and S2 are standard deviations.

  Next, the pitch detection operation in the pitch detection unit 12 will be described. A specific configuration example of the pitch detector 12 is as shown in FIG. First, as shown in FIG. 5, for example, the signal from the microphone is a mixture of target sound and other sounds. In FIG. 5, the solid line indicates the actually obtained signal waveform, and the broken line indicates the signal waveform of the target speech. Even if the target speech is emphasized by performing the directivity control processing by delay correction and addition as described above, other speech remains and a signal waveform in which these speeches are mixed is obtained. Here, the signal waveform indicated by the broken line of the target voice in FIG. 5 is regular with little fluctuation in the amplitude direction (level direction), whereas the mixed signal waveform indicated by the solid line also varies in the level direction. I understand that However, when the mixed signal waveform is compared with the waveform of the target speech, it can be confirmed that the peak interval is preserved in the time direction, although there is no correlation in the level direction.

  If the spectrum of the signal waveform as shown in FIG. 5 is taken, it becomes as shown in FIG. 6, for example, and it can be seen that it has a multiple structure of a certain fundamental frequency Fx. This fundamental frequency Fx generally corresponds to a pitch representing the pitch of the sound, and is also referred to as a pitch frequency. A period between adjacent peaks in the signal waveform of FIG. 5 is defined as one period Tx (one wavelength λx). This corresponds to the reciprocal of the cycle (pitch cycle). That is, Fx = 1 / Tx. In the example of FIG. 6, for example, a peak appears at a position of frequency 2Fx that is twice the pitch frequency Fx, and generally a peak appears at a position that is an integral multiple of frequency Fx.

  By the way, with respect to the pitch period Tx (pitch wavelength λx) corresponding to the adjacent peaks in the signal waveform, the actual waveform signal includes a component having a wavelength longer than this pitch period, and in particular, a pitch twice as large. It can be seen that in the component of the cycle Ty (= 2Tx), that is, in the spectrum of FIG. 6, the component of the frequency Fy (= Fx / 2) that is ½ of the pitch frequency Fx appears relatively influential. The fact that the component of the ½ pitch frequency Fy (= Fx / 2) appears relatively large as described above is generally applicable to an ordinary audio signal. For example, the pitches shown in FIGS. Similarly, in the example of the audio signal having the frequency Fx of about 650 Hz and the example of the audio signal having the pitch frequency Fx of about 580 Hz shown in FIGS. 9 and 10, the component of the frequency Fy (= Fx / 2) that is ½ the pitch. Can be clearly seen. 7 and 9 show audio signal waveforms on the time axis, and FIGS. 8 and 10 show spectra on the frequency axis.

  FIG. 11 is an explanatory diagram showing an example in the case of synthesizing the component of the pitch frequency Fx as described above and the component of the frequency Fy that is a half thereof. FIG. 11A shows a basic waveform (for example, a sine wave) of the pitch frequency Fx, and FIG. 11B shows a basic waveform of a wavelength twice the pitch wavelength, that is, a frequency Fy of 1/2 (= Fx / 2). Is shown. When these components are synthesized as shown in FIG. 11 (c), the same fluctuation occurs alternately every other wavelength, and for example, as shown in FIG. 11 (d), the shape becomes similar every other wavelength. More cases. For this reason, if the period between adjacent peaks is taken, variations appear alternately, so that stable pitch detection cannot be performed.

  Therefore, in the embodiment of the present invention, pitch detection is performed in units of a cycle Ty (= 2Tx) which is twice the cycle Tx (pitch wavelength λx) between peaks. Thus, if a peak is detected for every two wavelengths, it can be detected for each peak when the shape of the signal waveform is similar, so there is a tendency for errors to be smaller. At this time, the same result can be statistically obtained as the detection start timing even if the phase is shifted by one wavelength. In addition to the two wavelengths, the peak detection interval can in principle be an even multiple of 4 wavelengths, 6 wavelengths, 8 wavelengths,... However, for example, when a peak is detected every four wavelengths, the error is reduced, but there is a demerit that the number of samples is required.

  Next, a specific example of the pitch detection operation will be described with reference to FIG. In FIG. 12, a stereo audio signal is input in the first step S41, low-pass filter processing is performed in step S42, and directivity processing by the delay correction addition processing described above is performed in step S43. These correspond to the input from the input terminal 21 (11) of FIG. 2, the processing in the LPF (low-pass filter) 22, and the processing in the delay correction adding unit 23, respectively.

In the next step S44, a local maximum calculation process is performed by the local maximum detector 24 shown in FIG. This is to obtain a local peak as shown by the x mark in the waveform of FIG. 13, and has a positive peak (maximum point) and a negative peak (minimum point). Uses a local peak (maximum point) on the positive side, and can be obtained by detecting the point where the sample value of the signal waveform in the time axis direction changes from increasing to decreasing. Specifically, when the coordinate (position) on the time axis of each sample point of the signal waveform is represented by a sample number, the sample value of the sample point at the position n (ie, sample number n) is d (n), When the threshold value of the difference between sample values is th,
d (n) -d (n-1)> th and d (n + 1) -d (n) <-th (2)
In this case, the point n is the maximum point, and the sample value at that time is the maximum value.

  In the next step S45, the maximum value between the zero crosses in the range in which the value is positive among the maximum values obtained in step S44 by the maximum value between zero crosses detection unit 25 in FIG. 2 above. Is detected. That is, the maximum value among the local maximum values existing from the zero cross point where the sample value starts from negative to positive and from the next positive to negative zero cross point is detected. The coordinates (sample point position, sample number) on the time axis of the point of the maximum value between the zero crosses are recorded.

In the next step S46, the interval between the first and third maximum values of the maximum value obtained in step S45 in the maximum value pitch detection unit 26 in FIG. 2, that is, every other maximum value is obtained. The pitch is detected between values (for two wavelengths). That is, pitch detection is performed using two wavelengths as a detection unit. The pitch detection in this case corresponds to detection of a period Ty (= 2Tx) for two wavelengths, and this detected period Ty (or frequency Fy = 1 / Ty) is used as the original pitch period Tx (or It is used instead of the pitch frequency Fx). Here, when the coordinates on the time axis of each sample point of the signal waveform are represented by sample numbers, the period Ty obtained by the pitch detection can be represented by the number of samples (difference in sample numbers), and the first maximum. When the coordinate on the time axis of the maximum value (sample number) is max1, and the coordinate on the time axis of the maximum value of the third maximum value is max3,
Ty = max3−max1 (3)
It becomes.

The subsequent step S47 and subsequent steps correspond to the processing in the continuity determination unit 27 in FIG. 2, and in step S47, the pitches before and after the pitch detection unit section are compared. As the pitch in this case, the pitch period Tx may be obtained from Ty / 2 and used, but the period Ty for two wavelengths detected during the pitch detection may be used as it is. At this time, the ratio r of the pitch (or period Ty) for each adjacent pitch detection unit is obtained. For example, when the period Ty for the two wavelengths is used, the period for the two wavelengths of the current pitch detection unit n is calculated. When Ty (n) is given, the pitch ratio (ratio of the period Ty in this embodiment) r is
r (n) = Ty (n) / Ty (n-1) (4)
It becomes.

  Here, FIG. 14 shows an example of specific numerical values of the pitch detection result in the case of the signal waveform shown in FIG. In FIG. 14, periods for two wavelengths are sequentially detected from the first pitch detection unit, and these are indicated as Ty (1), Ty (2), Ty (3),. A value indicating the period Ty for two wavelengths detected in the pitch detection unit, the ratio r, and a continuity determination flag described later are illustrated.

  In the next step S48, in order to detect a section where the pitch ratio (ratio of period Ty) r obtained in step S47 is substantially stable (the stationary part), a change Δr (= It is determined whether or not the absolute value | Δr | (= | 1-r |) of 1-r) is smaller than a predetermined threshold th_r, and when it is determined that the absolute value | Δr | Then, the continuity determination flag is set (flag is set to 1), or a counter for measuring a section where the pitch is continuous (stationary part) is counted up. If it is determined in step S48 that the absolute value | Δr | of the ratio change is equal to or greater than the predetermined threshold th_r (NO), the process proceeds to step S50, and the continuity determination flag is reset (flag is set to 0). The predetermined threshold th_r is a value such as 0.05, for example. In the example of FIG. 14, r is 1.00 and | Δr | is 0 in the unit interval in which Ty (2) is detected. In the unit interval where the flag is 1 and Ty (3) is detected, r is 0.97 and | Δr | is 0.03, so the flag becomes 1, and so on, and Ty (n) is detected. In the unit interval, r is 0.7 and | Δr | is 0.3, so the flag is 0.

  In the next step S51, it is determined whether or not the detected pitch (or cycle Ty) has continuity. Here, for example, when the continuity determination flag set in step S49 is continuously counted five times or more, it is determined that there is continuity, and the detected pitch (or cycle Ty) is valid. to decide. For example, as in the example of FIG. 14, it is effective when the flag is continuous with 1 from the cycle Ty (2) to Ty (6), and the representative pitch, for example, Ty (2) to Ty (6) is effective. ) Average value is output.

  That is, if it is determined in step S51 that there is continuity (YES), the process proceeds to step S52, and the coordinates (time) on the time axis of a section (stationary part) where substantially the same pitch continues are represented by the sample number. After the output, the representative pitch (for example, the average value of the period Ty of the continuous section) is output in the next step S53, and the process ends. If it is determined in step S51 that there is no continuity (NO), the process ends. By repeatedly executing the process shown in FIG. 12, the pitch detection for the input signal waveform is continuously performed.

  To summarize the pitch detection operation in the above embodiment, the sound of two or more sound sources for a stereo microphone is targeted, and the target person's voice is separated, so that the pitch of a stationary part such as a mixed waveform vowel is detected. doing. At this time, it doesn't matter whether the voice is loud or male. At that time, if the waveform is pure, the level direction is preserved because there is no mixing, so the period can be determined by autocorrelation or the like. However, in the case of a mixed waveform, the level direction is not preserved, so the same method is difficult to use. However, it can be confirmed that the pitch in the time direction is preserved. Therefore, in the embodiment of the present invention, instead of obtaining the adjacent pitch by looking at the peak-to-peak from the characteristics of the voice waveform, the pitch detection is performed for two wavelengths, thereby improving the reliability. Accurate pitch detection can be performed, and an effect that facilitates subsequent voice separation processing can be obtained.

  Next, a specific example of the operation of the sound source signal separation device in FIG. 1 will be described.

  As the pitch detection unit 12 in FIG. 1, a unit that performs pitch detection from a period of two wavelengths as in the above-described embodiment can be used, but is not limited thereto, and a period of one wavelength is detected. Or a device that detects a period corresponding to an even number of wavelengths of four or more wavelengths may be used.

  The pitch detection unit 12 obtains a pitch for each pitch detection unit, obtains a coordinate (sample number) of a continuous section or a stationary part where the pitch continues, and an audio signal separation device using the stereo microphone of FIG. Is to separate signal waveforms of two or more sound sources from these pieces of information.

  The pitch obtained by the pitch detection unit 12 is sent to the separation coefficient creation unit 14, and a filter coefficient (separation coefficient) of a separation filter (filter operation circuit 15) for separating desired target speech is created. In the separation coefficient creation unit 14, when the representative pitch obtained by the pitch detection unit 12 is a fundamental frequency, the filter coefficient (separation) of the separation filter is obtained by a bandpass filter coefficient creation formula as shown in the following formula (5). Coefficient). In this equation (5), the filter coefficient at the tap position i is h [i], the number of filter taps is FIRLEN, HLFLEN is (FIRLEN-1) / 2, Pi is the circular ratio π, m is the number of harmonics, sampling The frequency FS, for example 48KHz, is 48000. Lo [n] and Hi [n] mean the bandwidth at the frequency of each harmonic order. Lo [n] is the lower frequency, and Hi [n] is the higher frequency. The bandwidth is arbitrary and matches the separation performance. Although m is the number of overtones, the number of overtones may be merely a fixed number. For example, when the maximum frequency is max_freq and the fundamental frequency is f [1], the integer value m = max_freq / f [1] Also good. However, when m = 0, f [0] = f [1] / 2 is applied. Further, the fundamental frequency may be f [0].

  FIG. 15 shows a specific example of the frequency characteristic of the separation filter (filter operation circuit 15) using the filter coefficient created by the separation coefficient creation unit 14. The filter having the frequency characteristics shown in FIG. 15 is a so-called comb-shaped bandpass filter. In this bandpass filter, the peaks and valleys become steeper as the number of taps increases, and the valley region decreases as the band width decreases. Since it increases, the probability of separation increases. Further, the bandpass filter coefficient created in the above equation (5) is actually expressed as shown in FIG. 16 by the tap position on the tap axis. At this time, it is necessary to select a window function in order to further increase the separation power.

  In the filter operation circuit 15, the above-described detected pitch and filter are filtered by an FIR filter that represents the product-sum operation, using the filter coefficient created by the separation coefficient creating unit 14 for the middle range and below. The target speech including the harmonic component is separated.

  Further, an unsteady waveform such as a consonant is input to the high frequency processing unit 17. The reason for dividing the high range and the mid range and below is because the principle of sound generation is different as follows, so it is better to change the processing in the band so that the vowel part concentrates below the mid range and the consonant part concentrates on the high range, This is because it is easier to determine continuity.

  In the sound generation principle, the vowel part is a stationary signal because it is generated using the periodic movement of the vocal cords as a vibration source. However, some consonant parts do not accompany vocal cord vibrations such as friction sounds and plosive sounds, and the consonant waveforms tend to be random. Therefore, if random waveforms are mixed in the vowel part, the random waveforms become noise components, which adversely affects pitch detection. Further, when sampling is performed with the same number of samples, the high frequency signal is less reproducible than the low frequency signal, which may cause the waveform to be corrupted, and thus the pitch may be detected incorrectly.

  Therefore, the accuracy of the determination can be improved by performing the process of determining the continuity in the middle range or less by dividing the high range and the middle range or less.

  In the high frequency processing unit 17, for example, in the stationary part of the target speech, that is, the vowel part, a process of removing a random high-frequency waveform due to consonants that do not normally appear such as a frictional sound or a plosive sound is performed.

  In speech, consonants with a high level usually do not exist in the vowel part. Therefore, even if the target voice can be separated from the vowel part of the voice signal composed of a plurality of sound sources, if a random high-frequency waveform is added to the vowel part, it may sound different from the actual target voice. Therefore, the high frequency processing unit 17 performs a process of reducing the gain of the high-frequency waveform in the stationary part which is a vowel part, and the adder 16 prevents the addition as much as possible, thereby obtaining an output closer to the target voice. it can.

  The output from the filter arithmetic circuit 15 and the output from the high frequency processing unit 17 are added by the adder 16 and are taken out from the output terminal 18 as a separated waveform output signal of the target sound.

  Here, the relationship between a stereo microphone and a sound source (such as a person) will be described. The distance between the stereo microphones is not particularly specified, but is generally within a few centimeters to several tens of centimeters for a portable device. For example, when collecting sound using a stereo microphone attached to a portable device such as a camera-integrated VTR (so-called video camera), the person who is a sound source is divided into three sections (center, left, right). If the division is several tens of degrees, the target sound source can be separated regardless of the position of the person. In consideration of the distance between the microphones, if the distance between the two microphones is taken into consideration, it is possible to divide into more regions if the distance is wide, and there is a disadvantage that the number of separation sections increases, but it is inconvenient to carry. Conversely, when the microphone interval is narrowed, the number of sections is reduced to three, but there is an advantage that it is convenient to carry.

  In the embodiment of the present invention as described above, the low-pass filter (LPF) 22 of FIG. 1 and the filters 20A and 20B of FIG. 1 of the pitch detector 12 may be combined into one filter bank. In this case, the delay correction addition unit 23 in FIG. 2 is shared with the delay correction addition unit 13 in FIG. 1 and sends the output from the delay correction addition unit 13 to the filter bank to separate the low frequency for pitch detection and separation. What is necessary is just to make it isolate | separate into the below-mid range for a filter, and the high region for a high region process.

  FIG. 17 is a block diagram showing a specific example of a sound source signal separation device using the filter bank unit 73 as described above.

  In FIG. 17, a stereo sound signal collected by a stereo microphone is input to an input terminal 71 and sent to a delay correction adding unit 72 as sound source signal enhancing means for enhancing a desired target sound source signal. As the delay correction addition unit 72, the configuration described with reference to FIG. 3 can be used. The output from the delay correction adding unit 72 is sent to the filter bank unit 73. The filter bank unit 73 is a part that performs band division, and prepares a high-pass filter that outputs a high frequency, a low-pass filter that outputs a mid-frequency, and a low-pass filter that outputs a low frequency. For example, the high band is a band that passes the consonant band, the middle band or lower is a band other than the consonant band, and the low band is a frequency band lower than the middle band. Among the signals of each band divided by the filter bank unit 73, the low frequency signal is sent to the pitch detector 75 via the continuity determination unit 74, and the signal in the middle frequency or lower is sent to the filter arithmetic circuit 77, and the high frequency signal Is sent to the high frequency processing section 79.

  Here, the pitch detection unit described with FIG. 2 includes a low-pass filter that outputs a low frequency in the filter bank unit 73 of FIG. 17, a continuity determination unit 74, and a pitch detector 75. Further, the delay correction adding unit 23 in FIG. 2 is moved to the preceding stage of the low pass filter (LPF) 22 and corresponds to the delay correction adding unit 72 in FIG. That is, as described above, the continuity determination unit 74 in FIG. 17 determines a portion (stationary portion) in which each successive pitch continues within, for example, a few percent of an error, and this continuity portion exceeds a predetermined time. When continuous (for example, the continuity determination flag in the detection unit for two wavelengths continues five times or more), it is determined that the pitch is valid, and the representative pitch at that time is output from the pitch detector 75.

  The separation coefficient creation unit 76 in the sound source signal separation unit 191 creates a filter coefficient (separation coefficient) of a separation filter (filter operation circuit 77) for separating a desired target sound, for example, according to the above equation (5). This is the same as the separation coefficient creation unit 14 of FIG. 1 described above. The created filter coefficients are sent to the filter arithmetic circuit 77 in the sound source signal separation unit 191. The filter arithmetic circuit 77 inputs components in the middle range or lower from the filter bank unit 73, and the filter arithmetic circuit of FIG. As in 15, the audio signal from the desired target sound source is separated. The high frequency processing unit 79 performs processing on unsteady waveforms such as consonants, and is the same as the high frequency processing unit 17 of FIG. 1 described above. The output from the filter arithmetic circuit 77 and the output from the high-frequency processing unit 79 are added by an adder 78 and taken out from the output terminal 80 as a separated waveform output.

  In such an embodiment, the pitch is detected in the stationary part, but due to the characteristics of speech that is actually spoken by one person, there is a region on the time axis beyond the part determined to be stationary in the mixed waveform. . In the above-described embodiment, the separation filter coefficient is created every time the pitch is detected. However, if the filter is actually applied only to the continuity determination portion, the processing is insufficient. Therefore, it is preferable to further increase the separation force in the time direction by using a coefficient around the continuity determination.

  For example, FIG. 18 shows two continuity parts detected in the vowel part with time on the horizontal axis, RA being the first stationarity determination part and RB being the second stationarity determination part. Then, the filter coefficients obtained at that time are different. At this time, the filter coefficient of the stationary part RA is applied before and after the time axis of the stationary part RA, and the coefficient of the stationary part RB is applied before and after the time axis of the stationary part RB. At this time, the areas to be applied before and after can be determined in advance using statistical data. For example, if a high frequency is detected as a pitch, the time is lengthened or shortened. If a low frequency is detected as a pitch, the time is shortened or lengthened.

  FIG. 19 shows a specific example of a signal waveform on the actual time axis. FIG. 19A shows a waveform before the filter is applied. In the range Rp of the arrow, the stationarity determination portion and the representative pitch are detected, that is, the fundamental frequency is detected. FIG. 19B shows a waveform that has passed through a band-pass filter created based on the pitch, and the region is further expanded by using the same coefficient at the arrow portion Rq.

  Furthermore, in order to improve the separation characteristics of the target sound, if all the harmonic component bands of the pitch frequency are passed, the sound other than the target may not be attenuated, but by using statistical data in advance, a certain harmonic order It is possible not to add the bandwidth.

  Next, still another specific example of the embodiment of the present invention will be described with reference to FIG. The sound source signal separation device shown in FIG. 20 is obtained by adding a configuration relating to speaker determination and area designation to the configuration of the sound source signal separation device described with reference to FIG. 17. A coefficient memory / coefficient selection unit 86 is used in the sound source signal separation unit 192 instead of the separation coefficient creation unit 76 in the sound source signal separation unit 191.

  The coefficient memory / coefficient selection unit 86 as the separation coefficient output means in FIG. 20 accumulates separation filter coefficients created in advance according to several types of pitches in the memory, and according to the detected pitches. Corresponding separation filter coefficients are read out. This is because, for example, the pitch value is divided into a plurality of sections, separation filter coefficients are created in advance for the representative pitch in the section, and the separation filter coefficients for each section are stored in the memory. The separation filter coefficient of the corresponding section may be read from the memory in accordance with which of the plurality of sections the pitch obtained by detection falls within. Thereby, in the sound source signal separation device, it is not necessary to create a separation filter coefficient for each detected pitch by calculation, and the separation filter coefficient can be obtained at high speed by memory access, and the processing speed can be increased.

  The speaker determination is to determine whether or not the voice (target voice) is from a target person among a plurality of sound sources (a plurality of persons), and the speaker determination unit 82 in this embodiment. In FIG. 4, a signal waveform that basically passes through an LPF (low-pass filter) 81 is used. The low-frequency signal via the LPF 81 may be a signal in the same band as the low-frequency signal extracted for pitch detection from the filter bank unit 73. In the speaker determination according to the present embodiment, the degree of coincidence is obtained using the value of the correlation coefficient cor as described together with the above equation (1) using the output of the delay correction addition in FIGS. By seeing, it can be determined whether or not the target person is speaking. As a specific example of the determination method, as shown in FIG. 21 (a), a determination method using the threshold value of the correlation value itself of the entire section of the continuity determination region as the continuity portion described above, or (b) of FIG. As shown in FIG. 21), the continuity determination area is divided into fine parts and determined with an appearance probability equal to or higher than a predetermined threshold, or as shown in FIG. And a method of determining with an appearance probability equal to or higher than a threshold value of the correlation value, and the like. In addition to this, determination may be made including the correlation of the characteristic data of the waveform. In addition, by adjusting the delay amount in the delay correction addition, it can be applied to each direction of a plurality of sound sources (a plurality of people), and it is also possible to determine who is speaking.

  The output from the speaker determination unit 82 is sent to the continuity determination unit 74 and the region designation unit 83. When the stationarity determination unit 74 determines a portion that is stationarity, time-axis coordinate data is obtained and the coordinate data is sent to the region designation unit 83. When the speaker is determined, the region designating unit 83 performs a process of taking a certain interval wider than the region of the continuity determining unit, and notifies the buffers 84 and 85 of the timing, thereby adjusting the region. do. The buffer 84 is inserted between the filter bank unit 73 and the filter arithmetic circuit 77 in the sound source signal separation unit 192, and the buffer 85 is inserted between the filter bank unit 73 and the high frequency processing unit 79. For the time (section) determined to be out of the area by the area designating unit 83, it is only necessary to lower the gain. For adjusting the gain, for example, a tap similar to the filter arithmetic circuit 77 is prepared, taps other than the center are set to zero, and only the center tap is set to a coefficient other than 1. Moreover, when it is set to 1/10, it is sufficient to use a coefficient of 0.1 for only the center tap.

  The other configuration in FIG. 20 is the same as the configuration in FIG. 17 described above.

  Summarizing the operation of the embodiment of the sound source signal separation device of the present invention described above, since the target person's voice is separated for two or more sound sources for a stereo microphone, a steady state such as a mixed waveform vowel is obtained. The pitch of the sex part is detected. At this time, it doesn't matter whether the voice is loud or male. By obtaining a bandpass coefficient (separation filter coefficient) for obtaining the pass characteristics of the target sound with reference to this pitch, the sound other than the target sound in a band other than the peak portion on the frequency axis related to the target sound. Is attenuated. Also, by preparing a coefficient memory in advance in order to increase the calculation speed, it is possible to save time and effort for calculating the coefficient.

  Next, FIG. 22 shows a schematic configuration of a specific example of a sound source signal separation device used in another embodiment of the present invention.

  In FIG. 22, an acoustic signal collected by a microphone or the like, specifically, for example, a stereo audio signal collected by a stereo microphone is input to the input terminal 110, and the pitch detector 12 and a desired sound source signal are input to the input terminal 110. The signal is sent to a delay correction adding unit 13 as a sound source signal emphasizing means to be emphasized. The output from the delay correction addition unit 13 is sent to the basic waveform creation unit 140 and the basic waveform replacement unit 150 in the sound source signal separation unit 190, and the basic waveform creation unit 14 is based on the pitch detected by the pitch detection unit 12. To create a basic waveform. The basic waveform from the basic waveform creation unit 140 is sent to the basic waveform replacement unit 150, and at least a part (for example, a stationary part described later) of the audio signal from the delay correction addition unit 13 is replaced with the basic waveform and output. It is taken out from the terminal 160 as a separated waveform output signal.

  In the specific example of the sound source signal separation device having such a configuration, the pitch detection unit 12 and the delay correction addition unit 13 are the same as those in the above-described configuration of FIG. Is omitted.

  As the pitch detection unit 12 in FIG. 22, a unit that detects pitch from the period of two wavelengths as in the above-described embodiment can be used, but is not limited to this, and the period of one wavelength is detected. Or a device that detects a period corresponding to an even number of wavelengths of four or more wavelengths may be used. Increasing the number of wavelengths for pitch detection increases the number of samples to be processed, but has the advantage of reducing errors. Further, such a pitch detector is widely used not only for the sound source signal separation device as shown in FIG. 22 but also for various sound source signal separation devices that perform sound source signal separation by detecting the pitch. it can.

  The basic waveform creation unit 140 creates a basic waveform based on the pitch of the stationary part detected by the pitch detection unit 12. As this basic waveform, a waveform that is an integral multiple of the pitch wavelength is generally used, but in the present embodiment, a waveform having a wavelength that is twice the pitch wavelength is used, as will be described later. Next, the basic waveform replacement unit 150 replaces, for example, the above-mentioned stationary portion of the audio signal from the delay correction addition unit 13 (or the input terminal 110) with a repetitive waveform of the basic waveform generated by the basic waveform generation unit 140. Thus, the separated waveform output signal in which only the audio signal from the desired sound source is emphasized is sent to the output terminal 160.

  Next, a specific example of the operation of the sound source signal separation device in FIG. 22 will be described.

  The pitch detection unit 12 obtains a pitch for each pitch detection unit, obtains a coordinate (sample number) of a continuous section or a stationary part where the pitch continues, and an audio signal separation device using the stereo microphone of FIG. Is to separate signal waveforms of two or more sound sources from these pieces of information.

  Here, as described above, the delay amount is corrected for the target sound for each microphone, the phases are matched, and these are added to enhance the target sound, and other sounds are relatively attenuated. In consideration of this point, the basic waveform of the stationary part can be created by adding the signal waveform of the stationary part as the period of the pitch detection unit.

  That is, the delay correction adding unit 13 in FIG. 22 performs delay amount correction so as to eliminate the difference in the propagation delay time of the sound from the target sound source to each microphone as described with reference to FIG. Output. The basic waveform creation unit 140 creates the basic waveform by processing the output signal waveform from the delay correction addition unit 13 based on the information from the pitch detection unit 12, and specifically, the pitch continuous section. Alternatively, the basic waveform is created by adding the signal waveform of the stationary part with the pitch detection unit as a period. The solid line waveform a in FIG. 23 shows an example of the basic waveform created in this way, and six waveforms for two wavelengths as shown in FIG. 5 (for example, period Ty (1) ˜ The waveform is averaged by adding (corresponding to Ty (6)). Moreover, the broken line waveform b in FIG. 23 shows the waveform of the original target speech for reference. As is apparent from FIG. 23, the basic waveform a created by adding the signal waveform of the pitch continuous section or the stationary part with two wavelengths as pitch detection units as a period is the waveform b of the original target speech. It can be seen that a very close approximation is obtained. This basic waveform is preserved or emphasized because it is added without phase shift with respect to the target sound, but the sound with phase shift is added with respect to other sounds, and thus shows a damping effect. At this time, it is preferable that the pitch detection is performed in units of two wavelengths and the basic waveform is also generated in units of two wavelengths. The generated basic waveform also stores a Ty component having a period longer than the pitch period Tx. This is because that.

  In the next basic waveform replacement unit 150, the pitch continuous section or the stationary part of the output signal waveform from the delay correction addition unit 13 is replaced with a repetitive waveform of the basic waveform generated by the basic waveform generation unit 140. Yes. A solid line waveform a in FIG. 24 shows an example of a repetitive waveform of the basic waveform replaced by the basic waveform replacement unit 150, and a broken line waveform b in FIG. 24 shows the waveform of the original target speech for reference. .

  In this way, the output waveform signal from the basic waveform replacement unit 150 in which the pitch continuous section or the stationary part is replaced with the basic waveform is taken out from the output terminal 160 as the separated output waveform signal of the target voice.

  FIG. 25 is a flowchart schematically showing the operation of such an audio signal separation device. In FIG. 25, in the first step S61, for example, pitch detection using the two wavelengths as described above as a detection unit is performed, and in the next step S62, it is determined whether or not there is continuity. Returning to step S61 of detection, if YES, the process proceeds to step S63 and subsequent steps. In step S63, the coordinates of the start point and end point of each pitch detection unit obtained by the pitch detection are input, and in step S64, the signal waveform of each pitch detection unit is added and averaged to obtain a basic waveform. In step S65, the basic waveform replacement process as described above is performed.

  Note that the relationship between the stereo microphone and the sound source (such as a person) is the same as described above, and a description thereof will be omitted.

  Summarizing the operation of the embodiment of the sound source signal separation device of the present invention described above, since the target person's voice is separated for two or more sound sources for a stereo microphone, a steady state such as a mixed waveform vowel is obtained. The pitch of the sex part is detected. At this time, it doesn't matter whether the voice is loud or male. When there is little error from the previous pitch, it is determined as continuity, and the continuous portions are added and averaged, and the completed waveform is used as a basic waveform to replace the original waveform. As the replacement waveform is added, the mixed waveform is attenuated, and only the target sound is emphasized to achieve separation.

  The present invention is not limited to the above-described embodiment. For example, the above-described pitch detection may be performed using not only two-wavelength periods but also multiple wavelengths of 2 such as four wavelengths as a period. In this case, if the number of wavelengths is four or more, the error is further reduced, but the pitch detection period may be set as appropriate in consideration of an increase in the number of samples to be processed. Further, such a pitch detection configuration can be widely used not only for the sound source signal separation device of the above-described embodiment, but also for various devices for separating sound source signals by detecting the pitch. Of course, various modifications can be made without departing from the scope of the present invention.

It is a block diagram which shows schematic structure of the sound source signal separation apparatus used as embodiment of this invention. It is a block diagram which shows the structural example of the pitch detection apparatus used for embodiment of this invention. It is a block diagram which shows the structural example of the delay correction | amendment addition part used for embodiment of this invention. It is a figure which shows the audio | voice signal waveform for demonstrating operation | movement of the delay correction addition part used for embodiment of this invention. It is a wave form diagram which shows the waveform on the time-axis of the audio | voice signal used for embodiment of this invention. It is a figure which shows the spectrum on the frequency axis of the audio | voice signal shown in FIG. It is a wave form diagram which shows the waveform on the time-axis of the audio | voice signal whose pitch frequency is about 650 Hz. It is a figure which shows the spectrum on the frequency axis of the audio | voice signal shown in FIG. It is a wave form diagram which shows the waveform on the time-axis of the audio | voice signal whose pitch frequency is about 580 Hz. It is a figure which shows the spectrum on the frequency axis of the audio | voice signal shown in FIG. It is a figure which shows the audio | voice signal waveform for demonstrating the reason for performing pitch detection by making 2 wavelengths into a detection unit in embodiment of this invention. It is a flowchart for demonstrating an example of the operation | movement of the pitch detection process in embodiment of this invention. It is a wave form diagram for demonstrating the maximum value and minimum value of an audio | voice signal waveform. It is a figure which shows the specific example of the information detected for every pitch detection unit for 2 wavelengths. It is a figure which shows the specific example of the frequency characteristic of the separation filter using the filter coefficient produced in the separation coefficient preparation part. It is a figure which shows the specific example of the filter coefficient produced in the separation coefficient production part. It is a block diagram which shows the other specific example of the sound source signal separation apparatus in embodiment of this invention. It is a figure for demonstrating the expansion on the time axis of the filter coefficient of a stationary part. It is a wave form diagram which shows the specific example of the signal waveform on a time-axis. It is a block diagram which shows the other specific example of the sound source signal separation apparatus in embodiment of this invention. It is a figure for demonstrating the relationship between a continuity determination area | region and a speaker determination. It is a block diagram which shows schematic structure of the sound source signal separation apparatus used as embodiment of this invention. It is a wave form diagram which shows an example of the basic waveform produced by the basic waveform production part. It is a wave form diagram which shows an example of the repetition waveform of the basic waveform replaced by the basic waveform replacement part. It is a flowchart for demonstrating an example of the sound source signal separation process in embodiment of this invention. It is a figure which shows the specific example of the sound collection with a stereo microphone when using three persons as a sound source.

Explanation of symbols

  12 pitch detector, 13, 23, 72 delay correction adder, 14, 76 separation factor generator, 15, 77 filter operation circuit, 17, 79 high frequency processor, 19, 190, 191, 192 sound source signal separator, 24 maximum value detection unit, 25 maximum value detection unit between zero crosses of maximum value, 26 pitch detection unit between maximum values, 27 continuous determination unit, 73 filter bank unit, 74 stationarity determination unit, 86 coefficient memory / coefficient selection unit, 140 Basic waveform creation section, 150 Basic waveform replacement section

Claims (20)

Sound source signal emphasizing means for emphasizing a desired sound source signal among input sound signals obtained by mixing acoustic signals from a plurality of sound sources and collected by a plurality of sound collecting means;
Pitch detecting means for detecting the pitch of the desired sound source signal in the input acoustic signal;
Sound source signal separation comprising: sound source signal separation means for separating the desired sound source signal from the input sound signal based on the detected pitch and the sound source signal enhanced by the sound source signal enhancement means apparatus. The sound source signal separating means is
Filter means for separating the desired sound source signal from the output signal from the sound source signal enhancing means;
The sound source signal separation device according to claim 1, further comprising: filter coefficient output means for outputting a filter coefficient of the filter means based on detection information from the pitch detection means.   The filter coefficient output means outputs a filter coefficient having a characteristic that allows the frequency characteristic of the filter means to pass a frequency component that is an integral multiple of the frequency of the pitch detected by the pitch detection means. 2. The sound source signal separation device according to 2.   The filter coefficient output means includes storage means in which filter coefficients corresponding to several kinds of pitches are stored in advance, and the filter coefficient corresponding to the pitch is stored from the storage means according to the pitch detected by the pitch detection means. 4. A sound source signal separation device according to claim 3, wherein High-frequency processing means for processing the consonant band of the output signal from the sound source signal enhancing means;
The consonant band of the output signal from the sound source signal enhancing means is extracted and sent to the high frequency processing means, the band other than the consonant of the output signal from the sound source signal enhancing means is extracted and sent to the filter means, and the sound source signal enhancing 3. The sound source signal separation device according to claim 2, further comprising: filter bank means for taking out a vowel band of an output signal from the means and sending it to the pitch detection means.   3. The sound source signal separation device according to claim 2, wherein the plurality of sound collecting means are left and right stereo microphones.   The sound source signal emphasizing unit corrects and adds a delay time difference of sound propagation from the desired sound source to the plurality of sound collecting units to the acoustic signals from the plurality of sound collecting units, 3. The sound source signal separation device according to claim 2, wherein only a sound signal from a desired sound source is emphasized.   3. The sound source signal separation apparatus according to claim 2, wherein the pitch detection means performs pitch detection using two wavelengths of the pitch of the desired sound source signal as detection units. The sound source signal separating means is
A basic waveform creating means for creating a basic waveform based on detection information from the pitch detecting means, using a stationary part in which at least substantially the same pitch continues in the output signal from the sound source signal enhancing means;
2. A sound source according to claim 1, further comprising basic waveform replacement means for replacing at least a part of the signal based on the input acoustic signal with a repetitive waveform of the basic waveform generated by the basic waveform generation means. Signal separation device.   10. The sound source signal separation device according to claim 9, wherein the pitch detection means performs pitch detection using two wavelengths of the pitch of the desired sound source signal as detection units.   The sound source signal separation device according to claim 9, wherein the plurality of sound collecting means are left and right stereo microphones.   The sound source signal emphasizing unit corrects and adds a delay time difference of sound propagation from the desired sound source to the plurality of sound collecting units to the acoustic signals from the plurality of sound collecting units, The sound source signal separation device according to claim 9, wherein only a sound signal from a desired sound source is emphasized.   10. The basic waveform generating means generates a basic waveform by adding and averaging the two portions of the pitch in units of the stationary portion where the pitch of the desired sound source signal is continuous. The sound source signal separation device described. A step of emphasizing a desired sound source signal among input sound signals obtained by mixing sound signals from a plurality of sound sources and collected by a plurality of sound collecting means;
Detecting a pitch of the desired sound source signal in the input acoustic signal;
And a step of separating the desired sound source signal from the input sound signal based on the detected pitch and the sound source signal emphasized in the emphasizing step. Sound source signal emphasizing means for emphasizing a desired sound source signal of an input sound signal obtained by mixing sound signals from a plurality of sound sources and collected by a plurality of sound collecting means;
A period detecting means for detecting a two-wavelength period with a detection unit of two wavelengths of the pitch in the output signal from the sound source enhancing means;
Continuous determination means for determining whether or not at least substantially the same pitch is continuous based on a change in the two-wavelength period detected by the period detection means, and outputting pitch information in accordance with the determination result. Pitch detector.   16. The pitch detection apparatus according to claim 15, wherein the plurality of sound collecting means are left and right stereo microphones.   The sound source signal emphasizing unit corrects and adds a delay time difference of sound propagation from the desired sound source to the plurality of sound collecting units to the acoustic signals from the plurality of sound collecting units, The pitch detection apparatus according to claim 15, wherein only a sound signal from a desired sound source is emphasized. A sound source signal emphasizing step for emphasizing a desired sound source signal of an input acoustic signal mixed with sound signals from a plurality of sound sources and collected by a plurality of sound collecting means;
A period detection step of detecting a two-wavelength period using two wavelengths of pitch in the output signal obtained by the sound source enhancement step as a detection unit;
A continuous determination step of determining whether or not at least substantially the same pitch is continuous based on a change in the two-wavelength cycle detected by the cycle detection step, and outputting pitch information in accordance with the determination result. Pitch detection method. Pitch detection means for performing pitch detection with a wavelength unit that is a multiple of 2 of the pitch of a desired sound source signal of an input acoustic signal obtained by mixing acoustic signals from a plurality of sound sources;
And a sound source signal separation means for separating a desired sound source signal based on the detected pitch. A step of performing pitch detection with a wavelength unit that is a multiple of 2 of the pitch of a desired sound source signal of an input acoustic signal obtained by mixing acoustic signals from a plurality of sound sources, as a detection unit;
And a step of separating a desired sound source signal based on the detected pitch.

Images