JP2932481B2 - Pitch detection method - Google Patents

Description

Description: BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention relates to a detection method for detecting, for example, a pitch of a musical tone or the like, and particularly to an audio processing unit (APU) for digitally processing a musical tone. The present invention relates to a method of detecting a pitch.

[Summary of the Invention]

According to the present invention, the phase of each frequency component obtained by Fourier-transforming an input digital signal obtained by converting an analog signal into a digital signal is aligned, and then Fourier-transformed again to detect the period of the peak value of the output data. Accordingly, it is an object of the present invention to provide a pitch detection method capable of detecting the pitch of an analog signal with a small number of samples and with high accuracy by detecting the pitch of the analog signal.

[Conventional technology]

Generally, a sound source used for an electronic musical instrument, a video game device, or the like includes, for example, an analog sound source such as a VCO, a VCA, a VCF, and the like.
SG (programmable sound generator) and waveform
It is roughly classified into digital sound sources such as ROM read type. In recent years, as one type of the digital sound source, a sampler sound source that uses a sound source data obtained by sampling a raw musical instrument sound or the like and digitally processing the stored data in a memory or the like has been widely known (for example, a special sampler sound source). See JP-A-62-264099 and JP-A-62-267798).

In this sampler sound source, since a memory for sound source data storage generally requires a large capacity, various methods for saving the memory have been proposed, for example, a looping process using periodicity of a musical sound waveform, A typical example thereof is a bit compression process using nonlinear quantization or the like. Note that the looping process is also a method for continuously outputting a sound for a longer time than the original duration of the sampled musical sound. That is, for example, when considering a musical tone signal waveform, a formant portion in which the periodicity of a waveform including non-pitch components such as a keystroke noise of a piano and a breath noise of a wind instrument generally occurs immediately after the start of sounding is generated. The same waveform repeatedly appears in the basic period corresponding to the musical pitch (pitch, pitch). By setting n loops (n is an integer) of this repetitive waveform as a looping section and repeating the reproduction as needed, a long-lasting sound can be obtained with a small memory capacity.

[Problems to be solved by the invention]

As a method of knowing the pitch of a musical tone in the above-described looping processing, conventionally, for example, a low-pass filter (LPF) is applied to a waveform of musical tone data to remove high-frequency noise components, and a zero cross point of the waveform after passing through the LPF is determined. A method of measuring the pitch (pitch) by counting the frequency of a musical tone data waveform is performed. However, in the above-described method, the frequency of the pitch cannot be measured unless a large number of zero-cross points are counted. Therefore, it is difficult to use for processing a sound in which a musical sound disappears in a short time.

As another method of finding the pitch, for example, there is a method of performing fast Fourier transform (FFT) on musical tone data, detecting a peak of the musical tone data, and measuring the peak. However, in this method, when the frequency of the pitch (pitch) is lower than the sampling frequency fs, the peak of the frequency of the fundamental tone cannot be effectively extracted, and the accuracy is not good.
In addition, a fundamental tone component may be much smaller than a harmonic component for some musical tones, and in this case also, it is difficult to effectively extract the frequency peak of the fundamental tone.

The present invention has been proposed in view of the above-described circumstances, and it is possible to detect the pitch (pitch) of a sound source from sound source data with a small number of samples, and to have a variation in pitch detection accuracy depending on the frequency of the sound source data. It is an object of the present invention to provide a small and highly accurate pitch detection method.

[Means for solving the problem]

In order to achieve the above object, a pitch detection method according to the present invention includes a step of performing a Fourier transform on an input digital signal obtained by converting an analog signal into a digital signal, a step of obtaining an absolute value of each of the obtained frequency components, The method further comprises a step of performing a Fourier transform on the absolute value of each of the obtained frequency components again, and a step of detecting a pitch of the analog signal by detecting a cycle of a peak value of the obtained output data.

A specific example of this will be described with reference to the flowchart of FIG. 1. In step S11, an input digital signal obtained by converting an analog signal into a digital signal is fetched, Fourier transform is performed in step S12, and an absolute value can be obtained in step S13. After aligning the phases of the respective frequency components, the Fourier transform is performed again in step S14, and the pitch of the analog signal is detected by detecting the cycle of the peak value of the output data in step S15. Is what you do.

[Action]

According to the present invention, an input digital signal obtained by converting an analog signal into a digital signal is subjected to a fast Fourier transform (FFT), and a phase difference component of the obtained signal after the FFT processing is forcibly set to zero. By performing the FFT processing (inverse FFT) again, the waveform of the fundamental tone (pitch) becomes clear. Therefore, the peak of the fundamental frequency can be easily measured.

〔Example〕

First, prior to the description of the embodiment of the present invention, the above-described looping processing will be briefly described with reference to a tone signal waveform shown in FIG. In general, immediately after the start of sound production, non-pitch components such as piano tapping noise and wind instrument breath noise are included, so that a formant part FR, in which the periodicity of the waveform is unclear, occurs. The same waveform repeatedly appears in the basic cycle corresponding to (pitch, pitch). The n period of the repetitive waveform component (n is an integer) as a looping section LP, the looping section LP is represented by the inter-looping points looping start point LP _S and the looping end point LP _E. And the above formant part FR and looping section
LP is recorded on a storage medium, and the formant
By repeatedly reproducing the looping section LP subsequent to the reproduction of the FR, a musical sound can be generated for an arbitrary long time.

Hereinafter, an embodiment of the present invention will be described with reference to the drawings. It goes without saying that the present invention is not limited to the following examples.

FIG. 3 is a functional block diagram showing specific examples of functions from sampling an input tone signal to recording it on a storage medium when the sound source data compression encoding method according to the embodiment of the present invention is applied to a sound source data forming apparatus. FIG. Input terminal in this case
As the input tone signal supplied to 10, for example, a signal directly collected by a microphone or a signal obtained by reproducing a digital audio signal recording medium or the like can be used in the form of an analog signal or a digital signal. .

First, in the sampling processing function block 11 shown in FIG. 3, the input tone signal is sampled at a frequency of, for example, 38 kHz, and extracted as 16-bit digital data per sample. This sampling processing corresponds to A / D conversion processing when the input tone signal is an analog signal, and corresponds to sampling rate conversion and bit number conversion processing when the input signal is a digital signal. Things.

Next, in the pitch detection function block 12, a fundamental tone frequency (fundamental frequency) f ₀ for determining a musical tone pitch (pitch) of the digital tone signal obtained by the above-described sampling processing, that is, pitch information is detected.

The principle of detection in the pitch detection function block 12 will be described. Here, the musical tone signal as the sampling sound source is
In many cases, the fundamental frequency is considerably lower than the sampling frequency fs, and it is difficult to identify the pitch with high accuracy only by detecting the peak of the musical tone on the frequency axis. Therefore, it is necessary to use some means to use the spectrum of the overtone component of the musical tone.

First, assuming that the waveform of a tone signal whose pitch is to be detected is f (t), the tone waveform f (t) is represented by the amplitude a of each harmonic component.
(Ω) and phase φ (ω), the tone waveform f
(T) is a Fourier-expanded equation, Can be represented by Here, if the phase shift φ (ω) of each overtone is all zero, It can be expressed by the following equation. The peaks of the musical tone waveform (t) whose phases are aligned in this manner are points at integer multiples of the period of all overtones of the musical tone waveform (t) and at t = 0.
This is nothing but the period of the fundamental tone.

Based on this principle, the procedure of pitch detection will be described with reference to a functional block diagram shown in FIG.

In FIG. 4, tone data is supplied from a real part data input terminal 31 and “0” is supplied from an imaginary part data input terminal 32 to a fast Fourier transform (FFT) function block 33.

Here, in a fast Fourier transformation performed by the fast Fourier transform function block 33, the musical tone signal to estimate the pitch is x (t), also a harmonic component included in the sound signal x (t) a _n cos ( 2πf _n t + θ) ... Then, x (t) becomes Rewrite this in complex notation, Here, cosθ = (exp (jθ) + exp (−jθ)) / 2. When this equation is Fourier transformed, Here, δ (ω−ω _n ) is a delta function.

In the next function block 34, the norm (absolute value, that is, the square root of the sum of the squares of the real part and the imaginary part) of the data after the fast Fourier transform is calculated.

That is, taking the absolute value Y (ω) of X (ω) cancels the phase component, This is performed to match the phases of all the high-frequency components of the musical tone data. The phase components can be aligned by setting the imaginary part to zero.

Next, the calculated norm is supplied to the fast Fourier transform (corresponding to the inverse FFT in this case) function block 36 as the real part data, and “0” is supplied to the imaginary part data input terminal 35 to perform the inverse FFT. To restore the music data. That is, the inverse Fourier transform is It is. The restored tone data after the inverse Fourier transform is extracted as a waveform that can be expressed by synthesizing a cosine wave in which all high-frequency components have the same phase.

Thereafter, the peak of the restored sound source data is detected by the peak detection function block 37. Here, the peak is a point where the extreme values (peaks) of all the high frequency components of the musical tone data coincide with each other, and in the next function block 38, the detected peak values are classified (sorted) from the larger value. . By measuring the period of the detected peak, the pitch of the tone signal can be known.

FIG. 5 is a diagram for explaining a configuration for detecting the maximum value (peak) of the musical sound data in the peak detection function block 37 of FIG.

In this case, the musical tone data has many peaks (extreme values) having different values, and the pitch of the musical tone can be known by finding the maximum value of the musical tone data and detecting its cycle.

That is, in FIG. 5, the tone data string after the inverse Fourier transform is transferred to an N + 1-stage shift register via an input terminal 41.
Is supplied to the 42, being sent to register _{a -N / 2 ... a 0 ...} a N / 2 sequentially through to the output terminal 43 of each stage of the shift register 42. The N + 1-stage shift register 42 acts as a window having a width of N + 1 samples for the tone data string, and N + 1 samples of the tone data string are sent to the maximum value detection circuit 44 via the window. That is,
The tone data is first input to the register a- _{N / 2 and} then transmitted sequentially to the register _{aN / 2.
-N / 2 ... a 0 ... N} + 1 samples of each tone data from a _{N / 2} are sent to the maximum value detection circuit 44.

The maximum value detection circuit 44 sets the data of the register a ₀ as a peak value when, for example, the value of the central register a ₀ in the shift register 42 becomes the maximum among the values of the data of the N + 1 samples. This is detected and output from the output terminal 45. The window width N + 1 can be set arbitrarily.

Returning to FIG. 3, the envelope detection function block 13 performs an envelope detection process using the pitch information on the digital tone signal after the sampling process to obtain a so-called envelope waveform of the tone signal. ing. This is achieved by sequentially connecting peak points of the tone signal waveform as shown in FIG. 6A, for example.
The waveform is as shown in FIG. B, and represents a change in level (or volume) over time immediately after sound generation. This envelope waveform generally has the ADSR (attack time /
Decay time / sustain level / release time)
In many cases. As a specific example of where tone signal, when considering the piano sound like be pronounced according to keying operation, the attack time T _A is a target increases the volume gradually keys of the keyboard is pressed (key on) The decay time T _D represents the time required to reach the volume, and the decay time T _D represents the time required to reach the next volume (for example, the volume of the continuous sound of the instrument) from the volume reached by the attack time T _S , and the sustain level L _S represents the volume of the sustained sound is maintained until the key-off by releasing the pressing of the key, the release time T _R represents the time until the sound from the above-mentioned key-off disappears. Each of the above times T _A , T _D , T
_R may also indicate the slope or rate of volume change. Further, in addition to these four parameters, more envelope parameters may be used.

Here, in the envelope detection function block 13, simultaneously with the envelope waveform information represented by each parameter such as ADSR (attack time T _A / decay time T _D / sustain level L _S / release time T _R ) as described above. In order to extract the above-mentioned formant portion with the attack waveform remaining, information indicating the entire decay rate of the signal waveform is obtained. The decay rate information, for example, as shown in FIG. 7, between time pronunciation (when a key on) of the attack time T _A has a value "1" of the reference, which represents the subsequent waveform monotonously attenuated is there.

Here, an example of the configuration of the envelope detection function block 13 in FIG. 3 will be described with reference to the functional block diagram in FIG.

The principle of the envelope detection is similar to the envelope detection of a so-called AM (amplitude modulation) signal. That is, the envelope is detected by considering the pitch of the tone signal as the frequency of the carrier of the AM signal. The envelope information is used for reproducing a musical sound, and the musical sound is formed based on the envelope information and the pitch information.

From the musical tone data supplied to the input terminal 51 in FIG. 8, the absolute value of the peak value data of the musical tone is obtained in the absolute value output function block 52. The absolute value data is sent to a functional block 55 of a FIR (finite impulse response) type digital filter. Here, the above FIR filter function block 55
Acts as a low-pass filter.
By supplying the filter coefficient formed in the function block 54 to the FIR filter function block 55 based on the pitch information supplied to the input terminal 53, the cutoff characteristic of the low-pass filter is determined.

Here, the filter characteristic is, for example, the characteristic shown in FIG. 9, and has a zero point in the fundamental tone (frequency f ₀ ) of the musical tone signal and its overtone frequency. For example, from the tone signal shown in FIG. 6A, envelope information as shown in FIG. 6B is detected by attenuating the fundamental and harmonic frequencies by the FIR filter. Note that the characteristics of the filter coefficient are represented by the following equations.

H (f) = k · (sin (πf / f ₀ )) / f... F _{0 in} this equation indicates a fundamental frequency (pitch) of the tone signal.

Next, the peak value data of the signal of the formant part FR shown in FIG. 2 and the peak value data of the signal of the looping section LP (loop ) Will be described.

First functional block 14 for generating the above loop data
In FIG. 10, the peak value data of the sampled tone signal is divided (or multiplied by the reciprocal) by the data of the previously detected envelope waveform (FIG. 6B) to perform envelope correction. Crest value data of a signal having a constant amplitude waveform as shown is obtained. By subjecting this envelope-corrected signal (peak value data) to filtering processing, a signal (peak value data) in which components other than the pitch components are attenuated or the pitch components are relatively emphasized is obtained. Here, the pitch component is that an integer multiple of the frequency component of the fundamental frequency f _0. Specifically, in order to remove low-frequency components such as vibrato contained in the envelope-corrected signal, the signal passes through an HPF (high-pass filter), and then has a frequency characteristic as shown by a one-dot chain line in FIG. integral multiples of the frequency band is the frequency characteristic of the pass band of the fundamental frequency f _0, by the intervention of the comb filter having the above
By passing only the pitch components included in the HPF output signal to attenuate other non-pitch components and noise components, and passing through an LPF (low-pass filter) as necessary, the signal after passing through the comb filter The superimposed noise component is removed.

That is, when considering a tone signal such as the sound of a musical instrument as the input signal, since the tone signal usually has a fixed pitch (pitch, pitch), its frequency spectrum has a solid line in FIG. as shown in, distributed as near the energy in the vicinity of the frequency of an integer multiple of the fundamental frequency f ₀ corresponding to the pitch of the musical tone itself is concentrated is obtained.
On the other hand, it is known that general noise components have a uniform frequency distribution. Therefore, by passing the input tone signal through a comb filter having a frequency characteristic as shown by a dashed line in FIG. 11, only a frequency component (a so-called pitch component) that is an integral multiple of the fundamental frequency f ₀ of the tone signal is passed or left as it is. The other components (non-pitched components and part of noise) are emphasized and attenuated, and as a result, the S / N ratio can be improved. Here, the frequency characteristic of the comb filter shown in dashed line in the FIG. 11, the following formula H (f) = [(cos (2πf / f 0) +1) / 2] represented by the ^N · · · It is. In this equation, f ₀ is the fundamental frequency of the input signal (the frequency of the fundamental tone corresponding to the pitch), and N is the number of stages of the comb filter.

The tone signal with the noise component reduced in this way is
It is sent to the repetitive waveform extraction circuit, and after the repetition waveform extraction circuit extracts an appropriate repetition waveform section such as the looping section LP in FIG. 2 described above, it is sent to a storage medium such as a semiconductor memory and recorded. You. Since the tone signal data recorded on this storage medium has attenuated non-pitch components and some noise components, it is possible to reduce noise when repeatedly playing back the repetitive waveform section, so-called looping noise. it can.

Note the HPF, comb filter, the frequency characteristic of the LPF is adapted to be set based on the fundamental frequency f ₀ is the pitch information detected by the pitch detection function block 12 first.

Then, in the loop interval between detection blocks 16 of FIG. 3, with respect to the musical tone signal other than pitch component is attenuated by the filtering process, by detecting a suitable repetitive waveform sections, looping start point LP _S and looping setting the looping points between end point LP _E.

That is, the loop section detection function block 16 selects two looping points that are relatively separated by a repetition period (an integer multiple) corresponding to the pitch (pitch) of the musical tone signal. Will be described.

When performing looping processing on musical tone data, the looping interval must be an integral multiple of the fundamental period of the musical tone signal (the reciprocal of the fundamental tone frequency). Therefore, if the pitch of the musical tone is accurately identified, it can be easily determined.

In other words, a looping interval is determined in advance, two points separated by the interval are extracted, and a looping point is set by evaluating the correlation or similarity of signal waveforms near the two points. As an example of the evaluation function, a description will be given of a function using convolution (synthesis product, convolution) for a sample of a signal waveform near each of the two points. That is, the convolution operation is sequentially performed on all sets of points to evaluate the correlation or similarity of the signal waveforms. Here, in the evaluation by the convolution described above, for example, the tone data is sequentially input to the shift register, and the tone data captured by each register is converted into, for example, a DSP described later.
(Digital signal processing device), each of which is input to a product-sum device, and the product-sum device calculates and outputs the convolution. Thus convolution obtained is to set the looping start point LP _S and looping end point LP _E of two points becomes maximum.

That is, in Figure 12, the candidate points of the looping start point LP _S and a _0, the candidate points of the looping end point LP _E as b _0, a plurality of front and rear vicinity of the candidate point a ₀ of the looping start point LP _S , For example, 2N + 1 points of each peak value data, a _-N .., a _-2 , a _-1 , a ₀ , a ₁ , a ₂ , ... a _N , looping end point LP _E same number before and after the vicinity of the candidate point b ₀ (2N
+1) points are expressed as b _-N .., b _-2 , b _-1 ,
Assuming that b ₀ , b ₁ , b ₂ ,... b _N , the evaluation function E (a ₀ ,
b ₀ ) is Can be determined. This is an equation for obtaining a convolution centered on the points a ₀ and b ₀ . Then, the set of the candidate points a ₀ and b ₀ is sequentially changed, and the evaluation function E for all the looping point candidate points is changed.
Is determined, and a point at which the value becomes the maximum among all the obtained evaluation functions E is defined as a looping point.

Further, the looping point can be obtained by the least square method of the error in addition to the method of obtaining the looping point from the convolution as described above. That is, the candidate points a ₀ and b ₀ of the looping point by the least square method are Can be represented by the following equation. In this case, a ₀ and b ₀ that minimize the value of the evaluation function ε may be obtained.

Further, the loop interval detection block 16 described above, calculates the pitch conversion ratio based on the above looping start point LP _S and the looping end point LP _E as required. This pitch conversion ratio is used as time axis correction value data in the time axis correction processing in the next function block 17. This time axis correction process is performed to make the pitches of the various sound source data uniform when actually recording the various sound source data in a storage unit such as a memory, and the pitch detection is performed instead of the pitch conversion ratio. The pitch information detected in the function block 12 may be used.

The pitch normalization operation in the time axis correction function block 17 will be described with reference to FIG.

FIG. 13A shows a tone signal waveform before time-base correction processing (mainly, time-base expansion / compression processing), and FIG. 13B shows a corrected waveform after the above-mentioned expansion. The time axis in FIGS. 13A and 13B is marked on a block-by-block basis in the quasi-instantaneous bit compression encoding process described later.

In the waveform A before the time axis correction, the looping section LP and the block are normally unrelated in the normal case. However, as shown in FIG. 13B, the looping section LP is an integer of the block length (block cycle). times (m times) and a way to time-scale modification processing, further boundary position of the block is shifted in the time axis direction to coincide with the looping start point LP _S and looping end point LP _E. That is, the looping section
By starting point of the LP LP _S and end point LP _E is time base correction so that the boundary position of a predetermined block (time scale modification and shift), the looping process in blocks of an integer number (m pieces) And normalization of the pitch of the sound source data at the time of recording can be realized. Here, "0" may be filled as the peak value data during the deviation ΔT from the block boundary generated at the head of the tone signal waveform due to the time shift.

FIG. 14 shows a block structure when the peak value data of the waveform after the time axis correction is divided into blocks for bit compression encoding processing to be described later. The number of peak value data in one block (sample (Number, number of words). In this case, the pitch normalization generally means the second
A time axis expansion process is performed so that the number of words in the n-cycle of the constant tone Tw of the tone signal waveform shown in FIG. it, still preferably, in the time axis processing to match the start point LP _S and end point LP _E looping section LP to block border position on the time axis (shift)
It is to make it. When the points LP _S and LP _E coincide with the block boundary position as described above, it is possible to reduce an error caused by block switching at the time of decoding in the bit compression encoding system.

Here, the word WLP _S and WLP _E of the portion indicated by oblique lines in FIG within 1 block of Figure 14 A is a looping start point of the figure correction waveform LP _S and the looping end point LP _E (exact points in LP _E Is a word indicating the sample at the point immediately before the. Note that the case of not performing the shift process, since the looping start point LP _S and end point LP _E do not necessarily coincide with the block boundary, the first
4 As shown in FIG. B, set position of the word WLP _S, WLP _E is set at an arbitrary position in the block. However, the number of words between the above word WLP _S to word WLP _E is an integral multiple of the number of words h in one block (m times),
The pitch is normalized.

Here, as described above, various methods can be considered for the time axis companding process of the tone signal waveform for making the number of words in the looping section LP an integral multiple of the number h of words in one block.
For example, it can be realized by interpolating the peak value data of the sampled waveform. As a specific example, a filter configuration for oversampling processing or the like can be used.

By the way, the looping cycle of the actual tone waveform has a fraction with respect to the sampling cycle unit, and the looping start point
If the deviation in the sampling peak value at the sampling peak value and looping end point LP _E in LP _S occurs, by interpolation processing using oversampling like, from a position near (sampling period of looping end point LP _E even if such finding the peak value to conform to the sampling the peak value of the short distance looping start point LP _S at position) of, with non-integer multiple of the (fractional sampling period interpolated samples were also included) realized looping cycle It is possible to do. Such a looping cycle of a non-integer multiple of the sampling cycle can also be set to an integer multiple of the block cycle by the time axis correction processing. For example, when performing the time axis companding processing using 256 times oversampling, , to reduce the error in the wave height value between the looping start point LP _S and the end point LP _E 1/256 can be realized more smoothly looping playback.

The waveform for which the looping section LP has been determined as described above and the time axis correction (compression expansion) processing has been performed is performed by the following functional block.
In FIG. 21, loop data is generated by connecting the looping sections LP back and forth as shown in FIG. That is, FIG. 15 shows a loop data waveform obtained by cutting out only the looping section LP from the tone waveform after the time axis correction (FIG. 13B) and arranging a plurality of looping sections LP. it is obtained by arranging the respective one of the looping end point LP _E and the other looping start point LP _S of a plurality of looping sections LP and sequentially connected. This loop data waveform is generated by the loop data generation function block 21.

The loop data, because it is formed by connecting multiple looping section LP, just before the start block containing the word WLP _S corresponding to each looping start point LP _S of the connection formed loop data waveform, looping (the exact point immediately before the point LP _E) termination point LP _E so that the data of the end block containing the word WLP _S corresponding to is arranged as it is. In principle, when the encoding process of bit compression encoding, just before the position of the start block of the looping section LP ₀ to be stored, it is sufficient that at least the end block is present. In more general terms, at the time of the bit compression encoding in block units, the parameters of the start block (information of bit compression encoding for each compression block, for example, range information and filter selection information described later) are What is necessary is just to form based on the data of an end block. this is,
This technique is also applicable to a case where a tone signal of only loop data having no formant part, which will be described later, is used as a sound source.

In this way, at the time of the above encoding, the looping start point
For the LP _S and the end point LP _E is over each of the front and rear multiple samples, so that each same data lined.
Therefore, that the parameters of the time of bit compression encoding for each of the blocks immediately preceding each of these points LP _S and LP _E becomes the same as, reducing the looping playback error upon decoding (noise) it can. That is,
Music data to be looped and reproduced is stable without connection noise. In the present embodiment, the number of data samples of the looping section LP arranged immediately before the start block is about 500 samples.

Next, in the data generation process of the signal of the formant part FR, first, similarly to the functional block 14 at the time of generating the loop data, an envelope correction process is performed in a functional block 18. In this case, however, the envelope correction is performed by dividing the above-described sampled tone signal by the envelope waveform (FIG. 7) containing only the decay rate information as described above. (Peak value data). That is, in the output signal of FIG. 16, the envelope of the above-mentioned attack portion (during the time T _A ) is left, and the other portions have a constant amplitude.

This envelope-corrected signal is subjected to a filtering process in a functional block 19 as necessary. For the filter processing in the functional block 19, for example, a comb filter having a frequency characteristic as shown by a dashed line in FIG. 11 similar to the functional block 15 is used. That this comb filter has a frequency characteristic as to attenuate the relatively non-pitch component emphasizes the integral multiple of the frequency band component of the fundamental frequency f ₀ corresponding to the pitch, also the comb filter described above The frequency characteristic is set based on the pitch information (basic frequency f ₀ ) detected by the pitch detection function block 12. Such a signal is used to generate data of a signal of a formant part in sound source data finally recorded on a storage medium such as a memory.

In the next function block 20, the above function block 17
The same time axis correction is performed on the formant part generation signal. This is because the pitch of each sound source is made uniform (normalized) by performing compression and expansion on the time axis based on the pitch conversion ratio obtained in the function block 16 or the pitch information detected in the function block 12.
It is for.

Next, in the function block 22, the loop data and the formant part generation data that have been time-axis corrected using the same pitch conversion ratio or pitch information are mixed. At this time, a hamming window is applied to the formant part generation signal from the functional block 20,
Form a fade-out type signal that attenuates with time in the part to be mixed with the loop data, and applies the same Hamming window to the loop data from the functional block 20 in this case. By forming a fade-in type signal that increases with time in the portion to be mixed with the formant signal, and mixing (cross-fading) these signals, a tone signal is finally obtained as sound source data. I have. Here, as loop data to be recorded on a storage medium such as a memory, noise in a looping reproduction (looping noise) can be reduced by extracting data of one looping section that is separated from the cross-fade part to some extent. . In this manner, the peak value data of the sound source signal including the formant portion FR, which is a waveform portion including a non-pitch component from the time of sound generation, and the looping section LP, which is a repetitive waveform portion including only the pitch component, is obtained.

In addition, a process of connecting each part such that the start point of the loop data signal is connected to the position of the looping start point in the formant part generation signal may be considered.

By the way, actually, loop section detection and looping processing,
When further performing mixing of the loop data and formant portion, by the hand of man operations leave the rough mixed with trial and error repeated Listen, loop points (looping start point LP _S and the looping of the time End point L
_PE ) Higher precision processing is performed based on information.

That is, prior to the high-precision loop section detection in the functional block 16, the loop section detection and the mixing and the like are manually performed while repeating the audition according to the procedure shown in the flowchart of FIG. Such high-precision processing (step S26 and subsequent steps) is performed.

In FIG. 17, in the first step S21, the loop point is detected with relatively coarse accuracy while using, for example, the zero-cross point of the signal waveform or visually confirming the display of the signal waveform, and looping is performed in step S22. Process and repeatedly play the waveform between the above loop points,
In the next step S23, it is determined whether or not the human being listens to the sample by listening. In the case of a failure, the process returns to the first step S21 to detect the loop point again. If a good audition result is obtained by repeating this, the process proceeds to the next step S24, where the signal is mixed with the above-mentioned signal for the formant section by crossfading, etc. It is determined whether the transition is good. If defective, the process returns to step S24 to repeat the mixing. Thereafter, the process proceeds to step S26, in which the loop section detection function block 16 performs high-accuracy loop section detection. Specifically, the loop section detection including the interpolation sample is performed, for example, at the time of 256 times oversampling, the loop section detection is performed with an accuracy of 1/256 of the sampling period, and the pitch conversion ratio for the pitch normalization is performed in the next step S27. Is calculated. Based on the pitch conversion ratio, the time axis correction processing in the functional blocks 17 and 20 is performed in the next step S28, and the loop data is generated in the functional block 21 in the next step S29. Then, in step S30, the mixing process in the functional block 22 is performed. In the processing after step S26, the processing from step S21
The loop point information and the like obtained up to S25 are used. Note that steps S21 to S25 may be omitted and full automation such as looping processing may be achieved.

Formant part obtained by such a mixing process
The peak value data of the signal composed of the FR and the looping section LP is subjected to a bit compression encoding process in the next functional block 23.

Various types of the above-described bit compression encoding method are conceivable.
No. 8629 and Japanese Unexamined Patent Publication No. 62-003516, etc., a quasi-instantaneous companding type, in which a block is formed for each fixed number of words (h samples) of peak value data and bit compression is performed in block units. The high-efficiency coding method shall be used.
This will be schematically described with reference to FIG.

In FIG. 18, the high-efficiency bit compression encoding system comprises a recording-side encoder 70 and a reproduction-side decoder.
90 and the input terminal 71 of the encoder 70.
Is supplied with peak value data x (n) of the sound source signal.

This input signal (peak value data) x (n) is calculated by a predictor
The prediction signal (the peak value data) (n) of the prediction signal from the predictor 72 is supplied to an FIR (finite impulse response type) digital filter 74 comprising an adder 72 and an adder 73.
73 is sent as a subtraction signal. In the adder 73, the prediction signal (n) is obtained from the input signal x (n).
Is subtracted to output a prediction error signal or a difference output d (n) in a broad sense. The predictor 72 generally has p past inputs x (n-p), x (n-p + 1),.
The prediction value (n) is calculated by a linear combination of x (n-1). Note that the FIR filter 74 is hereinafter referred to as an encoding filter.

In the high-efficiency bit compression encoding system, data of the excitation data within a certain time, that is, input data of a certain number of words h is divided into blocks, and the encoding filter 74 having the optimum characteristic is selected for each block. I am trying to do it. This is because a plurality of (for example, four) encoding filters having different characteristics are provided in advance, and a filter having an optimum characteristic, that is, a filter capable of obtaining the highest compression ratio is selected from these filters. It can be realized by doing. However, in the structure of a general digital filter, a plurality of sets (for example, four sets) of coefficients of the predictor 72 of one encoding filter 74 shown in FIG. In many cases, an operation substantially equivalent to selecting one of the plurality of encoding filters is performed by switching and selecting the set of coefficients in a time-division manner.

Next, the difference output d (n) as the prediction error is sent to a bit compressor composed of a shifter 75 for gain G and a quantizer 76 via an adder 81, for example, a floating point. A compression process or a ranging process is performed so that the exponent part in the display form corresponds to the gain G and the mantissa part corresponds to the output from the quantizer 76.
That is, the range is switched by shifting the input data by the number of bits according to the gain G by the shifter 75, and requantization is performed by the quantizer 76 to extract a certain number of bits of the bit-shifted data. I have. Here, a noise shaping circuit (noise shaper) 77 obtains a so-called quantization error corresponding to an error between an output and an input of the quantizer 76 by an adder 78, and converts the quantization error into a shifter having a gain G- ¹ . 79
, And a prediction signal of the quantization error is fed back to the adder 81 as a subtraction signal.
Give feedback. In this way, requantization by the quantizer 76 and error feedback by the noise shaping circuit 77 are performed, and the output (n) is output from the output terminal 82.
Is taken out.

The output d '(n) from the adder 81 is obtained by subtracting the prediction signal (n) of the quantization error from the noise shaper 77 from the difference output d (n). The output d ″ (n) from the shifter 75 is obtained by multiplying the gain G by the output d ′ (n) from the output adder 81. The output (n) from the quantizer 76 is the quantum Error e (n) in the process of quantization and the shifter 75
Is added to the output d ″ (n), and the quantization error e (n) is extracted by the adder 78 of the noise shaper 77. The quantization error e (n) is
Through the shifter 79 of the gain G ^-1 , one of the past r inputs
The prediction signal (n) of the quantization error is obtained through the predictor 80 that takes the next combination.

The above-mentioned sound source data is subjected to the above-described encoding processing, output as the output (n) from the quantizer 76, and taken out through the output terminal 82.

Next, the mode selection information as the optimum filter selection information is output from the prediction / range adaptation circuit 84, for example, the predictor 72 and the output terminal 87 of the encoding filter 74.
The range information for determining the gain G and the gain G- ¹ or the bit shift amount is output to the shifters 75 and 79 and the output terminal 86.

Next, the output (n) from the output terminal 82 of the encoder 70 is transmitted to the input terminal 91 of the decoder 90 on the reproduction side, or the signal '(n) obtained by recording and reproduction is supplied. Have been. This input signal
(N) is sent to the adder 93 via the shifter 92 having the gain G ^-1 . The output x '(n) from the adder 93 is sent to the predictor 94 to become a predicted signal' (n),
(N) is sent to the adder 93 and added to the output "(n) from the shifter 92. The added output is output from the output terminal 95 as a decoded output '(n).

The range information and the mode selection signal output from the output terminals 86 and 87 of the encoder 70 and transmitted, recorded, or reproduced are input to the input terminals 96 and 97 of the decoder 90, respectively. The range information from the input terminal 96 is sent to the shifter 92 and gain
G- ¹ is determined, and the mode selection information from the input terminal 97 is sent to the predictor 94 to determine a prediction characteristic. This predictor 94
Are selected as those of the predictor 72 of the encoder 70.

In the decoder 90 having such a configuration, the shifter 92
Is the product of the input signal '(n) and the gain G- ¹ . The output' (n) of the adder 93 is the output '(n) of the shifter 92. n) and the prediction signal '(n).

Next, FIG. 19 shows the bit compression encoding encoder 70
1 shows an example of output data of one block from the above, and this one block of data is composed of 1-byte header information (compression parameter information or additional information).
It is composed of FR and 8-byte sample data D _{A0 to} D _B3 . The header information RF includes the 4-bit range information, the 2-bit mode selection information, or the filter selection information, two 1-bit flag information, for example, information LI indicating the presence or absence of a loop, and a waveform end block. (End block) is composed of information EI indicating whether or not (end block). Here, the peak value data of one sample is bit-compressed and represented by 4 bits, and the data D _{A0 to} D _B3 include 4-bit data D _{A0H to} D _{B3L for} 16 samples. .

Next, FIG. 20 shows each block of the peak value data which has been subjected to the quasi-instantaneous (blocking) bit compression encoding corresponding to the head portion of the tone signal waveform as shown in FIG.
In FIG. 20, only the peak value data is shown omitting the above-mentioned header, and one block is set to 8 samples for the sake of illustration. However, it can be set arbitrarily such as 16 samples per block. Of course. This is the same in the case of FIG.

Here, the quasi-instantaneous bit compression code system is a mode for directly outputting the input tone signal, that is, a straight mode.
Selects a mode in which a signal having the highest compression rate is obtained from the PCM mode and a mode in which a tone signal is output through a filter, that is, a primary or secondary difference filter mode, and transmits tone data as an output signal. It is something to do.

When a musical tone is sampled and recorded in a storage medium such as a memory, the tone signal waveform of the musical tone starts to be captured at the tone generation start point KS.
When a filter mode requiring an initial value, such as a primary or secondary difference filter mode, is selected in the first block from, this initial value needs to be prepared in advance. It is desirable to have a form that does not require a value. For this reason, during a period preceding the tone generation start point KS, a pseudo input signal for selecting the straight PCM mode (a mode for directly outputting an input tone signal) is added, and then signal processing including the input signal is performed. I am trying to do it.

That is, specifically, in FIG. 20, a block in which data is all “0” is arranged as the pseudo input signal prior to the sound generation start point KS, and all data “0” from the head of this block are arranged. Is subjected to bit compression processing as sampling peak value data and taken in. This is, for example, to create a block in which one block of data is all “0” and store it in a memory or the like before use,
Alternatively, when sampling a tone,
, Data can be obtained by starting sampling from an input signal in which all data is "0" (that is, a silent portion before the start of sound generation). The number of blocks of the pseudo input signal is at least one.

The tone data including the pseudo input signal formed as described above is subjected to signal compression processing by the high-efficiency bit compression encoding system as shown in FIG. 18 and recorded in a storage medium such as a memory. The compressed signal is reproduced.

Therefore, when the tone data including the pseudo input signal is reproduced, the straight PCM mode is selected as the filter at the start of reproduction (the block portion of the pseudo input signal). There is no need to set in advance.

Here, there is a concern about a delay in the sound generation start time due to the pseudo input signal (since the data is all "0" and there is no sound) at the start of reproduction. However, for example, when the sampling frequency is 32 kHz and one block is 16 samples,
The delay of the sounding time is about 0.5 msec, which is not a problem that is not a delay that can be discerned by hearing.

By the way, digital signal processing for generating the above-mentioned bit compression encoding processing and other sound source data is often implemented by software using a digital signal processing device (DSP). A software-like configuration using a DSP is often used for reproducing sound source data. FIG. 21 shows an example of an audio / sound source unit that handles sound source data.
1 shows an example of the overall configuration of a system including a processing unit (APU) 107 and its periphery.

In FIG. 21, for example, a host computer 104 provided in a general personal computer device, a digital electronic musical instrument, a video game machine, or the like is connected to an APU 107 as the sound source unit. Sound source data and the like are loaded into the APU 107. The APU 107 includes at least a CPU (central processing unit) 103 such as a microprocessor, a DSP (digital signal processing unit) 101, and a memory 102 in which sound source data and the like are stored as described above. It is. That is, at least sound source data is stored in the memory 102, and various kinds of processing including reading control of the sound source data by the DSP 101, such as looping processing, bit decompression (decompression) processing, pitch conversion processing, envelope addition, echo (Reverb) processing or the like is performed.
The memory 102 is also used as a buffer memory for these various processes. The CPU 103 controls operations and contents of these various processes of the DSP 101.

Furthermore, the above sound source data from the memory 102 is
The digital sound source data finally obtained by performing the above-described various processings at 101 is converted into an analog signal by a digital / analog (D / A) converter 105 and
06 will be supplied.

It should be noted that the present invention is not limited to only the above-described embodiment. For example, in the above-described embodiment, the sound source data is formed by connecting the formant part and the looping section, but only the looping section is used. It can be easily applied to the case where sound source data is formed. Further, the decoder side configuration and the external memory for sound source data may be supplied as a ROM cartridge or an adapter. In addition, the present invention can be applied not only to the sound source of a tone signal but also to speech synthesis.

〔The invention's effect〕

According to the pitch detection method of the present invention, the input digital signal is Fourier-transformed, the absolute value of each obtained frequency component is taken, and the absolute value of each frequency component is again subjected to Fourier transform (inverse Fourier transform). Since the frequency of the peak of the data generated by the inverse Fourier transform is detected,
The pitch can be detected with a small number of samples, and there is little variation in accuracy due to the frequency of the sample.

Therefore, it is possible to detect the pitch of the sound source from the sound source data of a small number of samples, and it is possible to obtain a highly accurate pitch detection method in which the variation in the pitch detection accuracy due to the frequency of the sound source data is small. Further, the present invention can be realized by a combination of a fast Fourier transform and a simple peak detector constituted by a shift register.

[Brief description of the drawings]

1 is a flowchart showing the principle of the pitch detection method of the present invention, FIG. 2 is a tone signal waveform diagram, FIG. 3 is a functional block diagram for explaining a specific example of the signal recording method of the present invention,
FIG. 4 is a functional block diagram for explaining a pitch detecting operation, FIG. 5 is a block diagram for explaining a peak detecting operation, FIG. 6 is a waveform diagram of a tone signal and an envelope, FIG.
FIG. 8 is a waveform diagram of the decay rate information of the tone signal, FIG. 8 is a functional block diagram for explaining the envelope detection operation, FIG. 9 is a characteristic diagram of the FIR filter, and FIG. 10 is a diagram of the tone signal after envelope correction. FIG. 11 is a characteristic diagram of a comb filter, FIG. 12 is a waveform diagram for explaining an operation of setting an optimal looping point, and FIG.
FIG. 13 is a waveform diagram showing the tone signal before and after the time axis correction, and FIG.
FIG. 15 is a schematic diagram showing a structure of a block for quasi-instantaneous bit compression for peak value data after time axis correction, FIG. 15 is a waveform diagram showing loop data obtained by repeatedly connecting waveforms in a looping section, and FIG. Is a waveform diagram showing formant part generation data after envelope correction based on decay rate information, FIG. 17 is a flowchart for explaining operations before and after actual looping processing, and FIG. 18 is a quasi-instantaneous bit compression encoding system. FIG. 19 is a block circuit diagram showing a schematic configuration, FIG. 19 is a schematic diagram showing a specific example of one block of data obtained by quasi-instantaneous bit compression encoding, and FIG. FIG. 21 is a schematic diagram showing a configuration example of a system including an audio processing unit (APU) and its periphery.

Claims (1)

(57) [Claims] A step of performing a Fourier transform on an input digital signal obtained by converting an analog signal into a digital signal; a step of obtaining an absolute value of each of the obtained frequency components; and a step of performing a Fourier transform on the obtained absolute value of each of the frequency components again. A pitch detecting method, comprising: detecting a pitch of the analog signal by detecting a cycle of a peak value of the obtained output data.