JP4471780B2 - Audio signal processing apparatus and method - Google Patents

Description

  The present invention relates to an audio signal processing apparatus and method for detecting a pitch period from an input audio signal and compressing or expanding the time axis of the input audio signal based on the pitch period.

Time-axis compression / expansion processing (audio signal processing) that increases or decreases the playback speed of audio signals (audio signals) without changing the pitch when changing the tempo (speed) of karaoke or changing the playback speed of video Example) is required.
Conventionally, in Non-Patent Document 1 and Non-Patent Document 2, a portion having a strong periodicity of an audio signal is found, and the audio signal is omitted or repeated (inserted) in units of the cycle (pitch cycle) (based on the pitch cycle). ) A technique for performing time-axis compression / decompression processing is shown. In this technology, a signal corresponding to a pitch period that is omitted in an audio signal is added to a signal corresponding to the next pitch period by cross-fading weighting, or a signal corresponding to a pitch period that is inserted is added to a signal corresponding to the pitch period before and after the signal. A technique called PICOLA (Pointer Interval Control OverLap and Add, overlap addition method by pointer movement amount control) is employed in which the signal is obtained by overlapping and adding by cross-fading weighting.

FIG. 5 schematically shows a waveform of an audio signal when time axis compression is performed by the PICOLA method.
First, as shown in FIG. 5 (a), a pointer is set at the start position Po1 of the range of the audio signal to be subjected to time axis compression (omission of the audio signal), and the audio signal from the pointer position Po1 is A pitch period P (a period having a strong periodicity) is detected. An example of a method for detecting the pitch period P will be described later.
Next, as shown in FIG. 5 (b), a signal a ′ obtained by overlapping and adding two signals a and b corresponding to the pitch period P (the length of the pitch period P) from the pointer position Po1 by cross-fading weighting. Is generated. That is, when the two signals a and b are combined (added), as indicated by broken lines W1 and W2 in FIG. 5A, the weight for the signal a fades out (gradually decreases) as the time axis advances. The weight for b is weighted so that it fades in (increases gradually) as the time axis advances.
Next, the signal a is deleted (omitted) and the signal b is replaced with the signal a ′. Thereby, the time axis compression for one pitch period P is completed. Here, since the signal a ′ set in the omitted portion of the audio signal is a signal that is overlapped and added by weighting of the cross fade, the connection with the audio signals before and after the smooth becomes smooth, and the time axis compression with less sense of incongruity is possible. It becomes.
Next, assuming that the target compression ratio is Rx (0 <Rx <1), the pointer is reset to a position Po2 advanced by C (= P × Rx / (1-Rx)) from the position of Po1. The audio signal after the compression processing from the position Po1 to the position Po2 is output, and the same time axis compression processing is repeated from the pointer position Po2. As a result, a compressed audio signal having a length of C is generated (output) from the original audio signal having a length of P + C, and the time axis for achieving the target compression ratio Rx (= C / (P + C)). Compression is done.

On the other hand, FIG. 6 schematically shows the waveform of an audio signal when time axis expansion is performed by the PICOLA method.
First, as shown in FIG. 6A, a pointer is set at the start position Po3 of the range of the audio signal to be subjected to time axis expansion (audio signal insertion), and the audio signal from the pointer position Po3 is A pitch period P (a period having a strong periodicity) is detected.
Next, as shown in FIG. 6 (b), a signal a ′ obtained by overlapping and adding two signals a and b corresponding to the pitch period P (the length of the pitch period P) from the pointer position Po3 by cross-fading weighting. Is generated. As shown by broken lines W3 and W4 in FIG. 6 (a), the weight for the signal a fades in (increases gradually) as the time axis advances, and the weight for the signal b is Weighting is performed so that fade-out (gradual decrease) occurs as the time axis advances.
Next, the signal a ′ is inserted between the signals a and b. Thereby, the time base extension for one pitch period P is completed. Here, since the inserted signal a ′ is a signal that is overlapped and added by weighting the crossfade, the connection with the audio signals before and after that becomes smooth, and the time axis can be expanded with little discomfort.
Next, assuming that the target expansion ratio is Ry (0 <Ry <1), the pointer is reset to a position Po4 advanced by P + S (S = P × 1 / (Ry−1)) from the position Po3. Then, the audio signal after the expansion process from the position Po3 to the position Po4 is output, and the same time axis expansion process is repeated from the pointer position Po4. As a result, a compressed audio signal having a length of P + S is generated (output) from the original audio signal having a length of S, and a time axis for achieving the target expansion ratio Ry (= (P + S) / S). Stretching is done.

By the way, if the audio signal to be processed is a multi-channel audio signal such as a stereo audio signal, applying PICOLA to each channel requires a high-load operation for obtaining the pitch period to be executed for each channel. In addition to the extremely high computational load, the pitch period can be different for each channel, resulting in a phase difference that differs from the original audio signal between channels in the audio signal after compression / expansion processing, giving the listener a sense of incongruity. There is a problem that.
In order to solve this problem, it is effective to unify (commonize) the pitch period used for compression / expansion of audio signals in all channels.
For example, in Patent Document 1, a pitch period is detected for a signal (L + R) obtained by adding the L channel and the R channel of a stereo audio signal, and the compression / expansion processing (PICOLA) of the audio signals of both channels is performed based on the pitch period. A technique for performing the above has been proposed.
Further, Patent Document 2 proposes a technique for detecting a pitch period of a signal obtained by adding a plurality of channel signals or a channel signal having the maximum amplitude, and compressing / decompressing all the channel signals based on the pitch period. Has been.
With these technologies, it is only necessary to obtain a high-load operation for determining the pitch period for one audio signal, so that the increase in the operation load can be prevented and the audio signal after compression / decompression processing can be uncomfortable for the listener. It is possible to prevent a signal phase difference between channels from occurring.
JP 2001-5500 A JP 2002-297200 A Morita, Itakura, “Expansion and contraction of speech in time axis using autocorrelation function”, Proceedings of the Acoustical Society of Japan, March 1986, p. 199-200 Morita, Itakura, “Stretching and compressing speech on the time axis using the overlap addition method (PICOLA) with pointer movement control and its evaluation”, Proc. Of the Acoustical Society of Japan, October 1986, p. 149-150 Hiroshi Saruwatari “Basics of Blind Sound Source Separation Using Array Signal Processing”, IEICE Technical Report, April 2001, vol. EA2001-7, p. 49-56 Tomoya Takatani et al. “High fidelity blind source separation using ICA based on SIMO model”, IEICE Technical Report, January 2003, vol. US2002-87, EA2002-108

Here, audio signals from a plurality of different sound sources are mixed in an audio signal (one channel (monaural) input audio signal or a synthesized audio signal of input audio signals of a plurality of channels) whose pitch cycle is to be detected. In this case, with the techniques disclosed in Patent Document 1 and Patent Document 2, the pitch period corresponding to the sound signal of a representative sound source having the strongest periodicity is detected.
For this reason, in the techniques shown in Patent Document 1 and Patent Document 2, in the audio signal after time-axis compression or expansion when the signals of a plurality of sound sources coexist, the sound signal of one representative sound source becomes clear. However, there is a problem that the sound signal from other sound sources is unclear and leads to quality deterioration of the sound signal as a whole.
For example, when a human singing sound and a musical instrument performance sound are mixed in the input sound signal, the performance sound is clear, but the sound quality is deteriorated such that the singing sound is unclear.
Accordingly, the present invention has been made in view of the above circumstances, and its object is to detect a pitch period from an input audio signal and to compress or expand the time axis of the input audio signal based on the pitch period. When performing, even if audio signals from multiple sound sources are mixed in the input audio signal, the audio signal after compression / expansion maintains the clarity of each of the audio signals from multiple sound sources in a well-balanced sound quality. Is to provide an audio signal processing apparatus and method thereof.

In order to achieve the above object, the present invention separates an audio signal of the first predetermined frequency band from one input audio signal in which audio from a plurality of sound sources is mixed or a synthesized audio signal of input audio signals of a plurality of channels. The first signal processing means for generating an audio signal differs from the first predetermined frequency band from one input audio signal mixed with audio from a plurality of sound sources or a synthesized audio signal of a plurality of channels of input audio signals. Second signal processing means for generating an audio signal obtained by separating an audio signal of the second predetermined frequency band; and a specific signal included in the first predetermined frequency band separated by the first signal processing means A means for detecting a pitch period candidate which is a frequency component constituting an audio signal, wherein the input audio signal or the synthesized audio signal is sampled at a predetermined first pitch period sampling period. First pitch period candidate detecting means for detecting one or a plurality of pitch period candidates in ascending order of signal intensity difference between a sampled signal of one period and a sampled signal of another period; Sampling a specific audio signal included in the second predetermined frequency band separated by the signal processing means at a predetermined second pitch period sampling period different from the first pitch period sampling period; Second pitch period candidate detecting means for detecting as one or a plurality of second pitch period candidates in ascending order of difference in signal intensity between the sampled signal of one period and the sampled signal of the other period; said one or more first candidate pitch period, the one or from the common candidate and the candidate of the plurality of second pitch period, strong most periodicity one Pitch period selecting means for selecting a pitch period; and time axis adjusting means for compressing and / or expanding the time axis of the input audio signal based on the one pitch period selected by the pitch period selecting means. It may be configured as an audio signal processing device characterized by comprising.
That is, in detecting one pitch period from one input voice signal or a synthesized voice signal of a plurality of channels of input voice signals (hereinafter referred to as a pitch period detection signal), first, as a first step, the pitch period detection signal A plurality of candidates for the pitch period of the signal is detected based on the signal after being subjected to predetermined signal processing (first signal processing).
Here, the first signal processing extracts or removes a part of audio signals from a plurality of sound sources mixed in the input audio signal, emphasizes the periodicity (pitch period) of some audio signals, or It is processing such as attenuation. The plurality of pitch cycle candidates may be, for example, a predetermined number of candidates having a high evaluation value as the pitch cycle.
As a result, when the audio signals from a plurality of sound sources are mixed in the input sound signal, the sound signal of the sound source that is not necessarily representative (the strongest periodicity) among them (for example, the musical instrument performance sound is The singing voice signal and the like in the case of mixing can be extracted by the signal processing, and a plurality of pitch cycle candidates corresponding to the signal can be detected.
Further, as a second stage, based on the pitch period detection signal or a signal obtained by subjecting the pitch period detection signal to another signal process (second signal process) different from the first signal process, a plurality of pitch period candidates are selected. To select one pitch period.
The one pitch period selected in this way is extracted or emphasized by the signal processing from the signals used for detecting a plurality of candidates of the pitch period, that is, audio signals mixed in the pitch period detection signal. The pitch period corresponds to both the sound signal of the generated sound source and the sound signals of the other sound sources.
Therefore, if the compression process or the expansion process of the time axis of the input audio signal is performed based on the one pitch period detected (selected) in this way, the input audio signal is converted into the input audio signal in the processed audio signal. The clarity of each of the audio signals from a plurality of mixed sound sources can be maintained in a well-balanced manner.

Furthermore, the audio signal processing apparatus according to the present invention includes the first predetermined frequency band and the second predetermined frequency from one input audio signal from a plurality of sound sources or a synthesized audio signal of a plurality of channels of input audio signals. A third signal processing means for separating one or a plurality of third predetermined frequency band audio signals different from the band, and a third predetermined frequency band audio signal separated by the third signal processing means. Sampling is performed at a sampling period for the fourth predetermined pitch period, and the signal of one period and the other of the above-mentioned signals are sampled in ascending order of signal strength difference between the sampled signal of one period and the sampled signal of the other period. The difference from the period signal is detected as a plurality of third candidates of the pitch period detection signal as a period having a strong periodicity, and both the plurality of first candidates and the plurality of third candidates are supported. period Pitch period narrowing means for selecting a plurality of strong periods as a third candidate for the pitch period and narrowing down the plurality of pitch period candidates, and the pitch period selecting means is narrowed down by the pitch period narrowing means. It is conceivable that one pitch period is selected from the candidates.
That is, the pitch period detection signal is subjected to one or more signal processing (third signal processing) different from the signal processing described above (first signal processing), and each of the one or more signals after the processing is subjected to the processing. On the basis of this, it is also possible to narrow down a plurality of candidates for the pitch period detected by the above-described first stage processing, and to select the one pitch period from the narrowed candidates by the above-described second stage processing. Conceivable.
As a result, in an intermediate stage between the first stage and the second stage described above, another sound source different from the sound signal of the sound source to be detected or selected in the pitch period in the first stage and the second stage is selected. Extraction, enhancement, and the like (third signal processing) of audio signals are possible, and pitch periods corresponding to these audio signals are selected (narrowed down). As a result, the first and second steps described above, in the one or more intermediate stages between them, can asking you to balance well the corresponding pitch period in speech signals each of three or more different sound sources Become.
Here, it can be considered that the first signal processing is to separate the audio signal of the first predetermined frequency band by a band limiting filter.
Further, it is considered that the second signal processing and / or the third signal processing is to separate an audio signal having a frequency band different from the first predetermined frequency band by a band limiting filter. .
That is, the signal processing at each stage (first, second, and intermediate) may be, for example, band limiting filter processing for different frequency bands.
In addition, signal processing for amplifying or attenuating a signal in a specific frequency band by frequency enhancement by equalizing, signal processing for separating audio signals of a plurality of sound sources included in the input audio signal by a blind sound source separation method (BSS method), etc. Can be considered. Details of the BSS (Blind Source Separation) method are described in detail in Non-Patent Document 3, Non-Patent Document 4, and the like.

  According to the present invention, a plurality of pitch cycle candidates corresponding to the sound signal of a desired sound source extracted and enhanced by signal processing from an input sound signal or a synthesized sound signal of a plurality of input sound signals in the first stage is detected. Then, one pitch period corresponding to the sound signal of the other sound source is selected from the plurality of candidates in the second stage thereafter, and further, the pitch corresponding to the sound signal of the other sound source is selected in the intermediate stage. By narrowing down the period candidates, it is possible to select a pitch period corresponding to each of the sound signals of a plurality of sound sources in a balanced manner. Then, by performing compression processing and expansion processing on the time axis of the input audio signal based on the pitch period thus selected, a plurality of sound sources mixed in the input audio signal in the processed audio signal Therefore, it is possible to prevent the sound quality from deteriorating by keeping the clearness of each of the sound signals from the sound balance.

Hereinafter, embodiments of the present invention will be described with reference to the accompanying drawings so that the present invention can be understood. The following embodiment is an example embodying the present invention, and does not limit the technical scope of the present invention.
FIG. 1 is a block diagram showing a schematic configuration of the audio signal processing device Z1 according to the first embodiment of the present invention, and FIG. 2 shows a schematic configuration of the audio signal processing device Z2 according to the second embodiment of the present invention. FIG. 3 is a block diagram showing a schematic configuration of an audio signal processing device Z3 according to the third embodiment of the present invention, and FIG. 4 shows a schematic configuration of an audio signal processing device Z4 according to the fourth embodiment of the present invention. FIG. 5 is a block diagram, FIG. 5 is a diagram schematically showing a waveform of an audio signal when the time base compression of the audio signal is performed by the PICOLA system, and FIG. 6 is an audio when the time axis expansion of the audio signal is performed by the PICOLA system. FIG. 7 is a diagram schematically showing the waveform of a signal, FIG. 7 is a diagram showing an example of a waveform of an audio signal (original sound) used for time axis compression / expansion processing, and FIGS. 8 and 9 are audio signals (original sound) ) To extend the time axis using the conventional method FIG. 10 is a diagram showing an example of the waveform of the subsequent signal, FIG. 10 is a diagram showing an example of the waveform of the signal after the time axis extension is performed on the audio signal (original sound) shown in FIG. It is a graph showing the time change of the pitch period detected with the example of the conventional method and the method of this invention with respect to the example of the waveform of an audio | voice signal, and the music audio | voice signal.

<First Embodiment>
Hereinafter, the audio signal processing device Z1 according to the first embodiment of the present invention will be described with reference to the block diagram shown in FIG.
As shown in FIG. 1, the audio signal processing device Z1 includes a first filter (1), a pitch period detection unit 2, a pitch period selection unit 3, and a signal compression / expansion unit 4.
The first filter (1) performs band-limiting filter processing (an example of first signal processing) on an external input monaural signal M (an example of one input audio signal and a pitch period detection signal). Is.
The pitch period detection unit 2 receives a signal after filtering by the first filter (1) and detects a plurality of candidates for the pitch period of the signal (an example of pitch period candidate detection means).
The pitch cycle selection unit 3 inputs the input monaural signal (an example of a pitch cycle detection signal), and based on the signal, from among a plurality of pitch cycle candidates detected by the pitch cycle detection unit 2, One pitch period used for signal compression or expansion is selected (an example of pitch period selection means).
The signal compression / decompression unit 4 inputs one pitch cycle selected by the pitch cycle selection unit 3 (pitch cycle selection means), and uses this, for example, the PICOLA system (FIGS. 5 and 6). ) To compress and expand the time axis of the input monaural signal M (an example of an input audio signal) (an example of a time axis adjusting unit).
The speech processing device Z1 shown in FIG. 1 and speech processing devices Z2 to Z4 according to other embodiments to be described later are configured as processing circuits or DSPs (Digital Signal Processors) each composed of a CPU, a memory, and the like. In addition, a processing program that realizes processing (steps) performed by each component may be executed by a predetermined computer.
The audio signal processing device Z1 is characterized in that pitch period detection is performed in two stages by the first filter (1), the pitch period detection unit 2, and the pitch period selection unit 3. This will be described in detail below.

これらは，ｊサンプルだけ離れた信号値どうしの差（絶対値又は２乗値）を計算し，その差が小さいほど周期ｊにおける周期性が強い（即ち，周期ｊごとに似た波形が現れる）として評価するものである。従って，ｊ＝Ｎ₀〜Ｎそれぞれについて，（１）式又は（２）式による評価値を計算し，その評価値が最も小さいもの（最も周期性評価が高いもの）から所定の規則に従った複数個分のｊをピッチ周期の複数候補として検出（算出）する。
上記以外にも，例えば，ｊの範囲を複数区間に分割し，その分割区間毎に最も周期性評価の高いもの（前記評価値の最も小さいもの）を選択する方法も考えられる。即ち，ｊの区間をＮ₀〜Ｎ₁，Ｎ₁〜Ｎ₂，…，Ｎ_k〜Ｎ（但し，Ｎ₀＜Ｎ₁＜Ｎ₂＜…＜Ｎ_k＜Ｎ）というように分割し，分割区間各々において周期性評価が最大となる（例えば，（１）式や（２）式による評価値が最小となる）ｊをピッチ周期の複数候補とする。 << First Stage >>
First, the first filter (1) performs band-limiting filter processing such as a band-pass filter, a low-pass filter, and a high-pass filter on the input monaural signal M.
In the first filter (1), when audio signals from a plurality of sound sources are mixed in the input monaural signal M, the sound signal of a sound source that is not necessarily representative (having the strongest periodicity) among them. However, it passes only the band (for example, 200 Hz to 8 KHz in the case of human voice) of an audio signal (for example, a singing audio signal when musical instrument performance sounds are mixed) for which a clear sense after compression / decompression is desired. Filter processing is performed.
Then, a plurality of pitch cycle candidates corresponding to the signal after the signal processing are detected by the pitch cycle detection unit 2. Here, an example of a method for detecting (calculating) a plurality of pitch cycle candidates by the pitch cycle detection unit 2 will be described below.
All candidates j (j represents the number of samples of the digital audio signal) and converted to time, which are considered to be appropriate as the pitch period of the input monaural signal M (for example, an audio signal in which singing voice or instrument sound is mixed) The pitch period is “j × sampling period.”), A predetermined range of j = N _{0 to} N is set in advance, and the digital audio signal after the signal processing (filter processing) by the first filter (1) is pitched. A cycle evaluation target signal X _i, and (2N + 1) point sample signals X _i (i = 0 to 2N, i ≧ 1) for each of the pitch cycle candidates j (N _{0 to} N). Assess sexual strength. Then, a plurality of pitch cycle candidates are obtained based on the most periodic evaluation result. For example, in order from the one evaluated to have the strongest periodicity, a predetermined number (plural), or ones evaluated to have a periodicity stronger than a preset evaluation value, or those A plurality of pitch period candidates are obtained by combining the above.
In this case, when the time range i (number of samples) of the signal X _i to be evaluated for periodicity is 0 to N (where the maximum time range of the evaluation target signal to be referred to is 0 to 2N), It can be considered that the evaluation function of the strength of periodicity is the following formula (1) or (2).

These calculate the difference (absolute value or square value) between signal values separated by j samples, and the smaller the difference, the stronger the periodicity in period j (that is, a similar waveform appears for each period j). Is to be evaluated. Therefore, for each j = N _{0 to} N, the evaluation value according to the expression (1) or (2) is calculated, and the evaluation value is the smallest (the one with the highest periodicity evaluation) and the predetermined rule is followed. A plurality of j's are detected (calculated) as a plurality of pitch cycle candidates.
In addition to the above, for example, a method in which the range of j is divided into a plurality of sections and the one having the highest periodicity evaluation (the one having the smallest evaluation value) is selected for each divided section. That is, the section j is divided into N _{0 to} N ₁ , N _{1 to} N ₂ ,..., N _{k to} N (where N ₀ <N ₁ <N ₂ <... <N _k <N). Let j be the largest candidate for the pitch period in which the periodicity evaluation is maximum in each section (for example, the evaluation value according to the expressions (1) and (2) is minimum).

<< Second Stage >>
Next, based on the input monaural signal M, the pitch cycle selection unit 3 selects one pitch cycle used for compression / expansion from among a plurality of pitch cycle candidates obtained in the first stage.
Specifically, the input monaural signal M (digital audio signal) is set as the evaluation signal X _i for the pitch period, and the sample signal X _{i for} (2N + 1) points (i = 0 to 2N, i ≧ 1). After evaluating the strength of the periodicity for each of the pitch period candidates obtained in the first stage, the pitch period having the strongest periodicity is set as one pitch period used for compression / expansion. The pitch period evaluation method is the same as in the first stage.
One pitch period selected in this way is the signal used for detecting a plurality of candidates for the pitch period, that is, the audio signal mixed in the input monaural signal M (pitch period detection signal). The pitch period corresponds to both the sound signal of the sound source extracted by the filter (1) and the sound signals of other sound sources.
Then, the signal compression / decompression unit 4 uses the one pitch period detected by the two-stage processing based on the input monaural signal M to obtain a desired compression rate (expansion rate) for the input monaural signal M. ) Is time-axis compressed (expanded), and the compressed (expanded) audio signal M ′ is output. Here, the above-described PICOLA method is adopted as the compression / decompression method.
In the audio signal M ′ after compression / decompression processing output in this way, the clarity of each of the audio signals from a plurality of sound sources mixed in the input audio signal can be maintained in a well-balanced manner, and the sound quality is improved. .

<Second Embodiment>
Next, an audio signal processing device Z2 according to a second embodiment of the present invention will be described using the block diagram of FIG.
As shown in FIG. 2, the audio signal processing device Z2 is obtained by adding a synthesized signal generation unit 5 as a new component to the audio signal processing device Z1.
When the input audio signal is a multi-channel input audio signal such as a stereo audio signal, the pitch period differs for each channel when the signals with the detected pitch period and compression / expansion are synthesized for each channel signal. Therefore, a phase difference different from that of the original audio signal occurs between channels in the audio signal after compression / expansion processing, which gives the listener a sense of incongruity.
In order to solve this problem, it is effective to unify (commonize) the pitch period used for compression / decompression of audio signals in all channels.
Therefore, in the audio signal processing device Z2, the synthesized signal generation unit 5 generates a synthesized audio signal (an example of a pitch period detection signal) of a multi-channel input stereo signal (an example of an input audio signal), and synthesizes the synthesized audio signal. Based on the audio signal, one pitch cycle is obtained through two stages of processing in the same manner as the signal processing device Z1.
As the composite signal generation unit 5, for example, an addition of each channel signal (L + R in the case of two stereo channels), a signal (L + R) obtained by adding each channel signal, and a signal (LR) obtained by subtraction. It is possible to generate the generated voice signal and use the higher one of the power (amplitude) as the synthesized voice signal.
Then, the signal compression / decompression unit 4 uses the one pitch period detected by the two-stage processing based on the synthesized speech signal, for both channel signals of the stereo signal (L, R). Time-axis compression (expansion) is performed at a desired compression rate (expansion rate), and audio signals L ′ and R ′ after compression (expansion) are output. Here, the above-described PICOLA method is adopted as the compression / decompression method.
In this way, all the channel signals are compressed / expanded based on one pitch period P obtained from a plurality of channels of audio input signals, so that the calculation load increases and the listener is uncomfortable. It is possible to prevent the phase difference between the channels after compression and expansion. Such a configuration is also an example of an embodiment of the present invention.

<Third Embodiment>
Next, an audio signal processing device Z3 according to a third embodiment of the present invention will be described using the block diagram of FIG.
As shown in FIG. 3, the audio signal processing device Z3 is a new component of the audio signal processing device Z1, and performs band-limiting filter processing (second signal processing) on the input signal to the pitch period selection unit 3. The second filter (6) is applied. The filter characteristic of the second filter (6) is different from the filter characteristic of the first filter (1).
In this way, in the pitch cycle selection unit 3 (an example of pitch cycle selection means) in the second stage, the signal processing of the first filter (1) is applied to the input monaural signal M (an example of a pitch cycle detection signal). A configuration is also conceivable in which one pitch period is selected from a plurality of candidates for the pitch period based on a signal subjected to filter processing (an example of second signal processing) different from the above.
By this filtering process (signal processing) in the second stage, for example, the sound signal from the sound source with the strongest periodicity is removed, or only the sound signal from the desired sound source is extracted. The sound source signal to be used can be arbitrarily selected, and desired sound quality adjustment can be performed on the signal (M ′) after compression / decompression.
Of course, it is also conceivable to apply the configuration in which signal processing is performed in the second stage to the audio signal processing device Z2 (processing device for a plurality of channels of input audio signals (stereo audio signals, etc.)).

<Fourth embodiment>
Next, an audio signal processing device Z4 according to a fourth embodiment of the present invention will be described using the block diagram of FIG.
As shown in FIG. 4, the audio signal processing device Z4 detects a plurality of pitch cycle candidates and selects one pitch cycle in the first stage and the second stage as new components to the audio signal processing apparatus Z1. A pitch cycle candidate intermediate selection unit 20 for further narrowing down a plurality of pitch cycle candidates detected in the first step is added at an intermediate stage between the two.
The pitch cycle candidate intermediate selection unit 20 applies a filter process (first signal process) different from the process of the first filter (1) (first signal process) to the input monaural signal M (an example of a pitch period detection signal). .., 1N, and the plurality of signals after being subjected to the filter processing based on each of the plurality of third filters 11, 12,... , 2N (one example of pitch cycle narrowing means). The pitch cycle intermediate selection units 21, 22,..., 2N sequentially narrow down the plurality of pitch cycle candidates detected by the candidate detection means).
Each of the third filters (11 to 1N) has a filter characteristic for extracting a sound signal of another sound source different from a sound signal of a sound source that is a target of pitch period detection (selection) in the first stage and the second stage. To do.
Here, the pitch cycle intermediate selectors (21 to 2N) are connected in series with each other, and each of the first filter (1) and the input monaural signal M (an example of a pitch cycle detection signal) is connected to each other. A signal subjected to different filter processing is set as the pitch cycle evaluation target signal X _i, and a plurality of pitch cycle candidates (21 to 2 (N−1)) output from the preceding pitch cycle intermediate selection unit (21 to 2 (N−1)) ( For the pitch cycle intermediate selection unit 21 in the first stage, the pitch cycle is based on the result of evaluating the strength of the periodicity for each of the plurality of pitch cycle candidates detected by the pitch cycle detection unit 2. The multiple candidates are sequentially narrowed down to a small number (multiple) candidates. The method of narrowing down (selecting a plurality of pitch periods) is the same as the method of selecting a plurality of pitch period candidates from all pitch period candidates in the pitch period detection unit 2.
With such a configuration, it becomes possible to select a pitch period corresponding to an audio signal of a sound source different from the first stage and the second stage in an intermediate stage between the first stage and the second stage. Thus, it is possible to obtain one pitch period corresponding to each of the audio signals of three or more different sound sources in a well-balanced manner.
Then, one pitch is selected from a plurality of pitch period candidates narrowed down by the pitch period selecting section 3 (pitch cycle selecting means) by the pitch cycle intermediate selecting section (21 to 2N, an example of pitch cycle narrowing means). Select a period.

Next, the function and effect of the present invention will be described with reference to the speech waveforms shown in FIGS. 7 to 10, the horizontal axis (time axis) has a width of 0.2 seconds, the sampling rate of each audio signal is 44100 Hz, and the period when the pitch period is detected. The entire range of pitch periods for which the sex evaluation is performed is 350 to 1400 samples (corresponding to all the above-mentioned pitch cycle candidates N _{0 to} N).
FIG. 7A shows an example of a waveform of a simulation signal (corresponding to the input monaural signal M and the synthesized speech signal, an example of a pitch period detection signal, hereinafter referred to as an original sound) used for pitch period detection. (B) and (c) represent the waveforms of two different audio signals (hereinafter referred to as the original sound component 1 and the original sound component 2) included in the original sound. The original sound component 1 (b) is a 50 Hz sine wave, and the original sound component 2 (c) is a 533 Hz sine wave.
On the other hand, FIG. 8 (a) shows the waveform of a signal obtained by subjecting the original sound to a 1.41-times time base expansion process by the PICOLA method using a pitch period (equivalent to 533 Hz) obtained from the original sound component 2. It represents. FIGS. 8B and 8C show the expanded signals corresponding to the original sound component 1 and the original sound component 2 included in the expanded signal shown in FIG. 8A, respectively.
The signal waveform after the original sound extension process shown in FIG. 8A is close to the waveform of the original sound (FIG. 7A) except that the time axis is extended from the viewpoint of maintaining sound quality. Is preferred. However, as shown in FIG. 8A, when the time axis is expanded using the pitch cycle most suitable for one of the original sound components 2, the waveform of the original sound is greatly different. This is because, as shown in FIG. 8 (b), the waveform after expansion corresponding to the other original sound component 1, that is, the signal on the low frequency side which is not considered at all in the selection of the pitch period used for time axis expansion. This is because the waveform is greatly distorted.

On the other hand, FIG. 9A shows a waveform of a signal obtained by subjecting the original sound to 1.41 times time axis expansion processing by the PICOLA method using a pitch period (equivalent to 50 Hz) obtained from the original sound component 1. It is a thing. FIGS. 9B and 9C show the waveforms of the expanded signals corresponding to the original sound component 1 and the original sound component 2 included in the expanded signal shown in FIG. 9A, respectively. .
Also in this case, similarly to the case shown in FIG. 8, the signal waveform after the original sound expansion process shown in FIG. 9A is greatly different from the waveform of the original sound. This is because, as shown in FIG. 9C, power (amplitude) attenuation occurs in the signal after expansion of the original sound component 2 on the high frequency side, which is not considered at all in the selection of the pitch period used for time axis expansion. Because.
Such a waveform difference (difference between the waveforms in FIG. 8A and FIG. 9A with respect to the waveform in FIG. 7A) appears as a significant deterioration in sound quality.

Next, an example in which pitch period detection and time axis expansion are performed by the configuration of the audio signal processing device Z3 (FIG. 1) will be described with reference to FIG.
Here, the first filter (1) is a filter that extracts only the original sound component 2, and the second filter (6) is a filter that extracts only the original sound component 1.
In addition, in the first stage, the pitch period detection unit 2 detects a plurality of pitch period candidates, and the entire range of pitch periods (all candidates, a range of 350 to 1400 samples) is equally divided into four sections. , A process of detecting the one having the highest periodicity evaluation for each section (the one having the smallest evaluation value according to the above formula (1)) as a plurality of candidates (four candidates).
Further, in the selection process of one pitch period by the pitch period selection unit 3 in the second stage, the one having the highest periodicity evaluation (the above-mentioned (4) The process of selecting the one having the smallest evaluation value according to 1) as the one pitch period was applied.
FIG. 10A shows the original sound (FIG. 7A) using the pitch period obtained by the signal processing device Z1 through the pitch period detection (selection) in two stages accompanied by signal processing (filter processing). ) Is a waveform of a signal that has been extended by 1.41 times the time axis.
FIGS. 10B and 10C show the waveforms of the expanded signals corresponding to the original sound component 1 and the original sound component 2 included in the expanded signal shown in FIG. 10A, respectively. .
As shown in FIGS. 10 (a) to 10 (c), it can be seen that by applying the present invention, an output waveform without significant deterioration with respect to the waveform of the original sound can be obtained. This is because, in the first stage, the original sound component 2 is extracted from the original sound by the first filter (1), and a plurality of candidates having a pitch period corresponding to the original sound component 2 are detected. This is because one pitch cycle corresponding to the original sound component 1 is selected from among them, and therefore, one pitch cycle corresponding to both the original sound component 1 and the original sound component 2 is selected in a balanced manner. Further, by providing an intermediate stage process between the first stage and the second stage as in the signal processing device Z4 (FIG. 4), a pitch period corresponding to the sound signals of more sound sources in a well-balanced manner. Can be selected.
An audio signal (for example, FIG. 10 (a)) obtained by compressing / decompressing the original sound using such a pitch period is shown in FIG. 8 (a), FIG. 9 (a), etc. As compared with the audio signal after compression / decompression using the detection result of the conventional pitch period detection process, the sound quality is less deteriorated in hearing.

FIG. 11 is a graph (a) showing an example of a waveform of a music sound signal in which singing voice and instrument sound are mixed, and a graph showing a time change of the pitch period detected by the conventional method for the music sound signal. b) and a graph (c) showing a time change of the pitch period detected by the method of the present invention. 11B and 11C, the vertical axis represents the detected pitch period (number of samples), the horizontal axis represents the time axis (the numerical value on the horizontal axis represents that 44100 samplings are performed per second. Represents the time converted into the number of samplings (number of samples)).
Here, the sampling rate of the audio signal is 44100 Hz, and the entire range of the pitch period for which the periodicity is evaluated when detecting the pitch period is 350 to 1400 samples (described above) in both the conventional method and the method of the present invention. It corresponds to all pitch cycle candidates N _{0 to} N).
In the method of the present invention (FIG. 11C), the audio signal processing device Z1 (FIG. 1) is used, and the first filter (1) has a cutoff frequency of 200 Hz and 8 KHz, and a slope characteristic thereof. This is a case where an IIR filter of −12 dB / oct is used.
In addition, in the first stage, the pitch period detection unit 2 detects a plurality of pitch period candidates, and the entire range of pitch periods (all candidates, a range of 350 to 1400 samples) is equally divided into four sections. , A process of detecting the one having the highest periodicity evaluation for each section (the one having the smallest evaluation value according to the above formula (1)) as a plurality of candidates (four candidates).
Further, in the selection process of one pitch period by the pitch period selection unit 3 in the second stage, the one having the highest periodicity evaluation (the above-mentioned (4) The process of selecting the one having the smallest evaluation value according to 1) as the one pitch period was applied.
In the conventional technique FIG. 11 (b), the detected pitch periods are scattered in the vicinity of the upper and lower limits, and the variation is large, whereas in the technique FIG. 11 (c) of the present invention, the variation is small. Thus, it can be seen that the pitch period is extracted relatively smoothly before and after about 700 samples.
This indicates that the pitch period corresponding to the singing voice is detected (selected) by the filter (voice band limitation), and that the pitch period without any sense of incongruity is detected (selected) for the entire music voice signal. The method of the invention objectively indicates that the sound quality on hearing is improved.

In the embodiment described above, the band limiting filter process that can separate the audio signals from a plurality of sound sources with a relatively light calculation load is applied as the signal processing in the first stage.
In addition, as the signal processing in the first stage, signal processing for amplifying or attenuating a signal in a specific frequency band by frequency enhancement by equalizing, or a plurality of signals included in the input audio signal by a blind sound source separation method (BSS method) Signal processing for separating the sound signal of the sound source is also conceivable.
In the frequency emphasis by equalizing, for example, an amplification gain is set for the frequency band of a specific sound source, and the amplitude of the audio signal in that frequency band is amplified (emphasized) by performing FIR filter processing or the like. Thereby, the pitch period corresponding to the sound signal of a specific sound source can be obtained.
Further, the sound source separation based on the BSS method is effective in that sound sources are automatically separated and a sound signal of each sound source can be obtained without designating the frequency bands of each of a plurality of sound sources in advance. However, the calculation load increases.

  The present invention can be used for audio signal processing that performs time-base compression / expansion of an audio signal.

The block diagram showing the schematic structure of the audio | voice signal processing apparatus Z1 which concerns on 1st Embodiment of this invention. The block diagram showing schematic structure of the audio | voice signal processing apparatus Z2 which concerns on 2nd Embodiment of this invention. The block diagram showing schematic structure of the audio | voice signal processing apparatus Z3 which concerns on 3rd Embodiment of this invention. The block diagram showing the schematic structure of the audio | voice signal processing apparatus Z4 which concerns on 4th Embodiment of this invention. The figure which represented typically the waveform of the audio | voice signal at the time of time-axis compression of an audio | voice signal by a PICOLA system. The figure which represented typically the waveform of the audio | voice signal when the time-axis expansion | extension of an audio | voice signal is performed by a PICOLA system. The figure showing an example of the waveform of the audio | voice signal (original sound) used for a time-axis compression / expansion process. The figure showing an example of the waveform of the signal after performing time-axis expansion | extension by the conventional method to the audio | voice signal (original sound) shown in FIG. The figure showing an example of the waveform of the signal after performing time-axis expansion | extension by the conventional method to the audio | voice signal (original sound) shown in FIG. The figure showing an example of the waveform of the signal after performing time-axis expansion | extension by the method of this invention to the audio | voice signal (original sound) shown in FIG. The graph showing the time change of the pitch period detected with the example of the conventional method and the method of this invention with respect to the example of the waveform of a music audio | voice signal, and the music audio | voice signal.

Explanation of symbols

Z1 to Z4 ... audio signal processing devices 1, 11 to 1N, 6 ... filter 2 ... pitch cycle detection unit 3 ... pitch cycle selection unit 4 ... signal compression / expansion unit 5 ... synthesized signal generation unit 20 ... pitch cycle candidate intermediate selection unit 21 to 2N: Pitch period intermediate selection unit

Claims (5)

First signal processing means for generating an audio signal obtained by separating an audio signal of a first predetermined frequency band from an input audio signal in which audio from a plurality of sound sources is mixed or a synthesized audio signal of input audio signals of a plurality of channels When,
A voice signal in a second predetermined frequency band different from the first predetermined frequency band is separated from one input voice signal in which voices from a plurality of sound sources are mixed or a synthesized voice signal of a plurality of channels of input voice signals. Second signal processing means for generating an audio signal;
Means for detecting a pitch period candidate which is a frequency component constituting a specific audio signal included in the first predetermined frequency band separated by the first signal processing means, the input audio signal or The synthesized speech signal is sampled at a predetermined first pitch period sampling period, and one or a plurality of pitches in ascending order of difference in signal intensity between the sampled signal of one period and the sampled signal of the other period First pitch period candidate detecting means for detecting as a period candidate;
A specific audio signal included in the second predetermined frequency band separated by the second signal processing means is transmitted at a predetermined second pitch period sampling period different from the first pitch period sampling period. Second pitch period candidate detection that is sampled and detected as one or a plurality of second pitch period candidates in order of increasing signal strength difference between the sampled signal of one period and the sampled signal of the other period Means,
Pitch cycle selection for selecting one pitch cycle having the strongest periodicity from candidates common to the one or more first pitch cycle candidates and the one or more second pitch cycle candidates Means,
Time axis adjusting means for compressing and / or expanding the time axis of the input audio signal based on the one pitch period selected by the pitch period selecting means;
An audio signal processing apparatus comprising: One or more third predetermined frequencies different from the first predetermined frequency band and the second predetermined frequency band from one input audio signal from a plurality of sound sources or a synthesized audio signal of input audio signals of a plurality of channels Third signal processing means for separating audio signals in the frequency band of
The audio signal of the third predetermined frequency band separated by the third signal processing means is sampled at the sampling period for the fourth predetermined pitch period, the sampled signal of one period and the other period sampled Detecting a difference between one signal of the period and the signal of the other period as a period having a strong periodicity as a plurality of third candidates of the pitch period detection signal in order of increasing signal intensity difference from the signal of A pitch that selects a plurality of highly periodic periods corresponding to both the plurality of first candidates and the plurality of third candidates as a third candidate for the pitch period, and narrows down the plurality of pitch period candidates. Period narrowing means,
2. The audio signal processing apparatus according to claim 1, wherein the pitch cycle selecting unit selects one pitch cycle from the candidates narrowed down by the pitch cycle narrowing unit.   The audio signal processing apparatus according to claim 1 or 2, wherein the first signal processing is to separate the audio signal of the first predetermined frequency band by a band limiting filter.   The said 2nd signal processing and / or said 3rd signal processing isolate | separate the audio | voice signal of a frequency band different from a said 1st predetermined frequency band by a band-limiting filter. Audio signal processing device. A first signal processing step of generating an audio signal obtained by separating an audio signal of a first predetermined frequency band from an input audio signal in which audio from a plurality of sound sources is mixed or a synthesized audio signal of input audio signals of a plurality of channels When,
A voice signal in a second predetermined frequency band different from the first predetermined frequency band is separated from one input voice signal in which voices from a plurality of sound sources are mixed or a synthesized voice signal of a plurality of channels of input voice signals. A second signal processing step for generating an audio signal;
A step of detecting a pitch period candidate which is a frequency component constituting a specific audio signal included in the first predetermined frequency band separated by the first signal processing step, the input audio signal or The synthesized speech signal is sampled at a predetermined first pitch period sampling period, and one or a plurality of pitches in ascending order of difference in signal intensity between the sampled signal of one period and the sampled signal of the other period A first pitch period candidate detecting step of detecting as a period candidate;
The specific audio signal included in the second predetermined frequency band separated by the second signal processing step is transmitted at a predetermined second pitch period sampling period different from the first pitch period sampling period. Second pitch period candidate detection that is sampled and detected as one or a plurality of second pitch period candidates in order of increasing signal strength difference between the sampled signal of one period and the sampled signal of the other period Process,
Pitch cycle selection for selecting one pitch cycle having the strongest periodicity from candidates common to the one or more first pitch cycle candidates and the one or more second pitch cycle candidates Process,
A time axis adjustment step of compressing and / or expanding the time axis of the input audio signal based on the one pitch cycle selected by the pitch cycle selection step;
An audio signal processing method comprising:

Images