JP4428435B2 - Pitch converter and program - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To obtain an output sound of natural sound quality by a pitch conversion device which uses phase vocoder technology. <P>SOLUTION: An amplitude spectrum (A) is obtained by analyzing the frequency of an input speech waveform through FFT (Fast Fourier Transform) analytic processing. A plurality of local peaks P<SB>0</SB>to P<SB>2</SB>etc., of spectrum intensity are detected on the amplitude spectrum (A) and spectrum distribution regions R<SB>0</SB>etc., are designated by the local peaks. As shown in (B), the spectrum distribution regions R<SB>0</SB>etc., are moved on a frequency axis according to an input pitch to vary the pitch. The spectral distribution region having a pitch frequency approximate to a peak frequency relating to setting and having a spectral intensity of a local peak most approximate to an envelope value relating to the designation is selected and the amplitude spectrum data and phase spectral data of the spectrum distribution region are copied and are used for sound signal generation. <P>COPYRIGHT: (C)2008,JPO&amp;INPIT

Description

  The present invention relates to a pitch conversion apparatus and program suitable for use in singing synthesis and the like, and more particularly to improvement of pitch conversion technology using phase vocoder technology.

  Conventionally, a pitch conversion technique using a phase vocoder technique is known (see, for example, Non-Patent Document 1). Also, a singing synthesizing apparatus that changes pitches using such pitch conversion technology has been proposed and known by the same applicant as the present application (see, for example, Patent Document 1). The pitch changing process in this type of singing voice synthesizing apparatus will be described with reference to FIG.

FIG. 13A shows an amplitude spectrum obtained by frequency analysis of the speech waveform of the original speech by FFT (Fast Fourier Transform) analysis processing. On such an amplitude spectrum, a plurality of local peaks P _{0 to} P ₂ are specified, and a spectral distribution region such as R ₀ including the spectra before and after each local peak is specified. Local peaks P ₀ , P ₁ , and P ₂ correspond to a fundamental tone, a first harmonic (a second harmonic having a frequency twice that of the fundamental), and a second harmonic (a third harmonic having a frequency that is three times that of the fundamental), respectively. It is a peak, and R ₀ , R ₁ and R ₂ are spectral distribution regions corresponding to the peaks P ₀ , P ₁ and P ₂ , respectively. fa, fb, fc, and fd are the lower limit frequencies of the spectrum distribution regions R ₀ , R ₁ , R ₂ , and R ₃ , respectively, and the upper limit frequencies of the spectrum distribution regions R ₀ , R ₁ , and R ₂ are the lower limit frequencies fb, respectively. , Fc, fd are set to slightly lower frequencies.

In the pitch change process, a pitch increase process as shown in FIG. 13B is performed as an example. In the pitch increase process, the amplitude spectrum distribution is moved to the high pitch side on the frequency axis for each spectrum distribution region so as to obtain a pitch (pitch) higher than the original voice. That is, if the frequency of the fundamental tone peak of the original voice is f ₀ and the peak frequency of the fundamental tone after the pitch rise is f ₀₁ , the pitch change ratio T is T = f ₀₁ / f ₀ . The amplitude spectrum distribution in the region R ₀ is moved to the high frequency side on the frequency axis so that the peak P ₀ of the fundamental tone after the pitch rise is located at the frequency f ₀₁ = f ₀ T. Also, assuming that the frequencies of the first and second harmonic peaks P ₁ and P ₂ of the original voice are f ₁ and f ₂ , respectively, the peak P ₁ of the first harmonic after the pitch increase becomes a frequency f ₁₁ = f ₁ T. The region R ₁ is moved so that the amplitude spectrum distribution in the region R ₁ is located on the high frequency side on the frequency axis, and the second harmonic overtone peak P ₂ after the pitch rise is located at the frequency f ₂₁ = f ₂ T. ₂ is moved to the high pitch side on the frequency axis.

In the example shown in FIG. 13B, after the pitch rises to the spectrum envelope EVb having the same shape as the spectrum envelope (spectrum envelope of the original speech) EVa connecting the peaks P _{0 to} P ₂ shown in FIG. Since the amplitude spectrum distribution is arranged so as to match the peaks P _{0 to} P ₂ , the tone after the pitch rise becomes the same as the tone of the original voice. To obtain a timbre different from the original voice, give the spectrum envelope EVb a shape different from the spectrum envelope EVa, and then adjust the amplitude spectrum distribution so that the peaks P _{0 to} P ₂ after the pitch increase are matched to the spectrum envelope EVb. What is necessary is just to arrange.

On the other hand, a phase spectrum corresponding to the amplitude spectrum of FIG. Based on such a phase spectrum, a phase spectrum distribution is determined for each of the aforementioned spectrum distribution regions. FIG. 14 shows an amplitude spectrum distribution am ₀ and a phase spectrum distribution ph ₀ in a certain spectrum distribution region. For simplicity, the amplitude spectrum distribution am ₀ has a simple shape different from the amplitude spectrum distribution in the region R ₀ in FIG. In FIG. 14, f ₀ is the peak frequency corresponding to the local peak, φ ₀ is the peak phase corresponding to the peak frequency f ₀ , and f _L and f _U represent the lower limit frequency and the upper limit frequency of the spectrum distribution region, respectively. .

FIG. 15 shows the amplitude spectrum distribution AM ₀ and the phase spectrum distribution PH ₀ when the pitch is increased by the above-described pitch change process. The amplitude spectrum distribution AM ₀ is obtained by moving the amplitude spectrum distribution am ₀ to the high pitch side on the frequency axis so that the peak after the pitch rise is located at the frequency f ₀₁ = f ₀ T. The phase spectrum distribution PH ₀ is obtained by changing the phase spectrum distribution ph ₀ on the frequency axis so that the peak phase after pitch increase is located at the frequency f ₀₁ = f ₀ T (corresponding to the frequency change of the amplitude spectrum distribution am ₀ ). while moving to the treble side is a modification corresponding to the phase of each spectral bin in the pitch increase in the amplitude spectrum distribution it is ₀ in phase spectrum distribution after the movement. Here, each spectrum bin is a phase spectrum corresponding to each frequency in the phase spectrum distribution.

In order to correct the phase of each spectral bin, the phase change amount Δφ ₀ is obtained according to the following equation 1, and Δφ ₀ is added to the phase of each spectral bin. In Equation 1, Δt represents a frame interval (frame period).

For example, the spectrum bin corresponding to the peak frequency f ₀₁ after the pitch rise is set to a phase of φ ₀ + Δφ ₀ by adding Δφ ₀ to the peak phase φ ₀ . For other spectral bins, the phase is obtained by adding Δφ ₀ for each spectral bin. As a result, a phase spectrum distribution PH ₀ as shown in FIG. 15 is obtained. In Figure 15, F _L and F _U represent each a lower limit frequency and upper limit frequency of the spectrum distribution region after elevating pitch. In FIG. 14, the difference between the frequencies f ₀ and f _L is (f ₀ −f _L ), and the difference between the frequencies f _U and f ₀ is (f _U −f ₀ ). The lower limit frequency _FL and the upper limit frequency _FU are set corresponding to these differences (f ₀ −f _L ) and (f _U −f ₀ ), respectively.

The amplitude spectrum data representing the amplitude spectrum distribution for each spectrum distribution region shown in FIG. 13B and the amplitude spectrum distribution of FIG. 13B for each spectrum distribution region, respectively, and described above with reference to FIGS. The phase spectrum data representing the phase spectrum distribution subjected to such correction processing is converted into a time-domain audio signal by inverse FFT processing or the like. As a result, an audio signal having a pitch T times higher than that of the original audio can be obtained. As such an audio signal, it is possible to obtain a signal having a timbre different from that of the original voice by changing the spectrum envelope as described above.
J. et al. Laroche and M.M. Dolson, “New Phase-Vocoder Techniques for Real-Time Pitch Shifting, Chorusing, Harmonizing, and Other Exotic Audio Modifications” Audio Eng. Soc. , Vol. 47, no. 11, 1999 November JP 2003-255998 A

  According to the above-described voice conversion technology, voice conversion is performed without separating the frequency analysis result of the voice waveform into a harmonic component and a non-harmonic component, so that the non-harmonic component does not resonate and a natural converted sound is obtained. Should be. In addition, a natural converted sound should be obtained even if it is a voiced frictional sound or a plosive sound. However, according to the research of the inventors of the present application, it has been found that there are some points to be improved in order to obtain a more natural sound quality.

First, in the pitch change processing described above with reference to FIG. 13, it is assumed that perfect harmonic frequencies 2f ₀ and 3f ₀ are obtained by doubling and triple the peak frequency f _{0 of the} fundamental tone, respectively, as shown in FIG. 13 (A). The peak frequency f ₁ of the first overtone shifts to a higher sound side than 2f _0, or the peak frequency f ₂ of the second overtone shifts to a lower sound side than 3f ₀ , and none of them matches the corresponding perfect harmonic frequency. Many. This stems from the fact that the actual human voice is not perfectly periodic.

When the pitch increase process is performed on the amplitude spectrum distribution for each spectrum distribution region shown in FIG. 13A as described above with reference to FIG. 13B, the peak frequencies f ₁₁ and f ₂₁ of the first and second overtones are obtained. F ₁ T and f ₂ T, respectively, and deviate greatly from the perfect harmonic frequencies 2f ₀₁ and 3f ₀₁ assumed for the peak frequency f _{01 of the} fundamental tone. That is, before the pitch rise, the difference Δf ₁ between the peak frequency f ₁ of the first overtone and the perfect harmonic frequency 2f ₀ is Δf ₁ = f ₁ −2f ₀ , which is completely equal to the peak frequency f ₂ of the second overtone. The difference Δf ₂ from the harmonic frequency 3f ₀ is Δf ₂ = f ₂ −3f ₀ , whereas after the pitch rise, the difference Δf ₁₁ between the peak frequency f ₁₁ of the first harmonic and the perfect harmonic frequency 2f ₀₁ is obtained. Δf ₁₁ = f ₁ T−2f ₀₁ = f ₁ T−2f ₀ T = (f ₁ −2f ₀ ) T, and the difference Δf ₂₁ between the peak frequency f ₂₁ of the second harmonic and the perfect harmonic frequency 3f ₀₁ is Δf ₂₁ = f ₂ T-3f ₀₁ = f ₂ T-3f ₀ T = (f ₂ −3f ₀ ) T.

If the absolute value of the difference Delta] f ₁ and Delta] f ₂ to be compared with the absolute value of each difference Delta] f ₁₁ and Delta] f _21, absolute value of the difference Delta] f ₁₁ and Delta] f ₂₁ are each turned T times the absolute value of the difference Delta] f ₁ and Delta] f ₂ I can see that As described above, when the time-domain sound signal is generated based on the amplitude spectrum distribution for each spectrum distribution region as shown in FIG. 13B, the peak frequency of the harmonics is greatly shifted from the complete harmonic frequency. There is a problem that the sound quality of the output sound becomes unnatural. It has also been confirmed that the unnaturalness of sound quality becomes more pronounced as the pitch change ratio T increases.

In addition, in the amplitude spectrum distribution for each spectrum distribution region as shown in FIG. 13B, for example, the spectrum missing regions Q ₁ and Q are arranged on one side and the other side of the amplitude spectrum distribution including the peak P _{0. 2} occurs. For this reason, there is a problem that the output sound is not as fresh as the original sound.

  Although FIG. 13 shows an example of the pitch increasing process as the pitch changing process, a pitch decreasing process can also be performed as the pitch changing process. In the pitch reduction process, the amplitude spectrum distribution is moved to the low frequency side on the frequency axis for each spectrum distribution region. FIG. 16 shows an example of the pitch / tone change processing according to the research of the inventors of the present application. In this example, the pitch reduction processing is performed.

FIG. 16A shows an amplitude spectrum similar to that shown in FIG. 13A, and the same parts are denoted by the same reference numerals and detailed description thereof is omitted. In the pitch reduction process, the amplitude spectrum distributions of the spectrum distribution regions R ₀ , R ₁ , and R ₂ are moved to the bass side on the frequency axis as shown in FIG. In the amplitude spectrum distribution after moving, the peak frequency corresponding to the peak _{P 0} corresponding peaks _{P 01} is _{F 0,} the peak frequency corresponding to the peak _{P 1} corresponding peak _{P 11} is _{F 2,} the peak _{P 2} corresponding the peak frequency corresponding to the peak _{P 21} of an _{F 5.}

A predetermined spectral envelope EVc having a shape different from that of the original speech spectral envelope EVa is assumed. In order to sufficiently represent such an envelope EVc, the peak frequency F ₁ is between the peak frequencies F ₀ and F _2, and the peak frequencies F ₃ and F are between the peak frequencies F ₂ and F _{5. 4} and peak frequencies F ₆ and F ₇ are set on the high frequency side of the peak frequency F ₅ , respectively. The peak frequencies _{F 1,} select the spectral distribution region _{R 0} having the peak frequency _{f 0} closest to _{F 1} in the spectrum distribution region _R 0 to R _2, copies the amplitude spectrum distribution from this region _{R 0.} The amplitude spectrum distribution related to the copy is moved to the high sound side on the frequency axis so that the peak frequency is changed from f ₀ to F ₁ .

As for the peak frequencies F ₃ and F ₄ , the spectrum distribution region R ₁ having the peak frequency f ₁ closest to F ₃ and F ₄ is selected in the same manner as described above for the peak frequency F ₁ , and F ₁ is selected from this region R _{1. 3,} corresponding to F ₄ to copy the amplitude spectrum distribution. The first amplitude spectrum distribution relating to the copy is moved to the bass side on the frequency axis so that the peak frequency is changed from f ₁ to F ₃ . Further, the second amplitude spectrum distribution related to the copy is moved to the high pitch side on the frequency axis so that the peak frequency is changed from f ₁ to F ₄ .

For the peak frequencies F ₆ and F ₇ , the amplitude spectrum distribution of the spectrum distribution region R ₂ is copied corresponding to F ₆ and F ₇ in the same manner as described above for the peak frequencies F ₃ and F ₄ . The first amplitude spectrum distribution relating to the copy is moved to the high frequency side on the frequency axis so that the peak frequency is changed from f ₂ to F ₆ . The second amplitude spectrum distribution related to the copy is moved to the high frequency side on the frequency axis so that the peak frequency is changed from f ₂ to F ₇ .

In the amplitude spectrum distribution corresponding to each of the peak frequencies F ₀ , F ₂ , and F ₅ , the spectrum intensity of each spectrum bin is corrected so that the peaks P ₀₁ , P ₁₁ , and P ₂₁ are matched with the spectrum envelope EVc. Here, each spectrum bin is an amplitude spectrum corresponding to each frequency in the amplitude spectrum distribution. In the amplitude spectrum distribution corresponding to the peak frequency F _1, it modifies the spectral intensity of each spectral bin to match the peak P ₀₂ to the spectral envelope EVc.

In the amplitude spectrum distribution corresponding to the peak frequency F _3, to modify the spectral intensity of each spectral bin to match the peak P ₁₂ to the spectral envelope EVc. In the amplitude spectrum distribution corresponding to the peak frequency F _4, to modify the spectral intensity of each spectral bin to match the peak P ₁₃ to the spectral envelope EVc.

In the amplitude spectrum distribution corresponding to the peak frequency F _6, it modifies the spectral intensity of each spectral bin to match the peak P ₂₂ to the spectral envelope EVc. In the amplitude spectrum distribution corresponding to the peak frequency F _7, it modifies the spectral intensity of each spectral bin to match the peak P ₂₃ to the spectral envelope EVc.

As a result of the pitch / timbre change processing as described above, as shown in FIG. 16B, eight amplitude spectrum distributions corresponding to the peak frequencies F _{0 to} F ₇ have peaks P ₀₁ , P ₀₂ , P _{11 to} P _13. , P _{21 to} P ₂₃ are arranged in accordance with the spectrum envelope EVc.

According to the above-described pitch / tone color changing process, a spectrum distribution region corresponding to a predetermined peak frequency (for example, F ₁ ) defined on one side of a certain peak (for example, peak P ₀₁ ) after the pitch is changed is the predetermined distribution frequency. Since a spectrum distribution region having a peak frequency (for example, f ₀ ) closest to the peak frequency is selected and the amplitude spectrum distribution of the selected spectrum distribution region is copied and used for generating an audio signal, it is easy to obtain a natural timbre. However, for example, as the spectrum distribution regions corresponding to the peak frequencies F ₃ and F ₄ , respectively, the amplitude spectrum distribution of the spectrum distribution region having the peak P ₁ is copied and the spectrum intensity of each spectrum bin is increased and used. There is a problem that the tone becomes strong with noise. That is, a peak having a small spectrum intensity is relatively unstable. When the spectrum intensity is increased, the instability is further expanded and a noise-like impression is given.

17 and 19 show an example in which the time position of the analysis window is changed in the FFT analysis processing according to the research of the inventors of the present application. In these figures, t represents time. Tp indicates one period of the input voice waveform, and this one period corresponds to the pitch of the input voice. t _{S1 to} t _S3 all indicate vocal cord vibration start positions.

In the example of FIG. 17, to obtain the peak phase shown in FIG. 18 by performing an FFT analysis of the center W _C of the analysis window FW in a state matching the center position near between the vocal cords vibrate start position t _S2, t _S3 It was. In FIG. 18, the horizontal axis represents the frequency f, and the vertical axis represents the phase (0 to 2π). f ₀ is the peak frequency of the fundamental tone, and f _{1 to} f ₅ are all the peak frequencies of the harmonics. According to FIG. 18, it can be seen that the peak phases φ _{0 to} φ ₅ corresponding to the peak frequencies f _{0 to} f ₅ are not uniform in value.

In the example of FIG. 19, to obtain a peak phase shown in FIG. 20 by a state where the center W _C of the analysis window FW tailored to vocal fold vibration start position t _S3 performs FFT analysis. In FIG. 20, the same parts as those in FIG. As can be seen from FIG. 20, the peak phases φ ₀ ′ to φ ₅ ′ are substantially aligned around a certain value. Such a phase-matched state is a characteristic of a natural speech waveform. When the phases at the analysis window positions shown in FIG. 19 are different, as described above, when generating a time domain audio signal, a waveform that does not look like a voice is generated, resulting in an unnatural output sound. End up. In other words, when voice conversion is performed using the phase spectrum shown in FIG. 18, the sound quality of the output sound becomes unnatural.

  An object of the present invention is to provide a novel pitch conversion apparatus and program that can solve the above problems and obtain an output sound with a natural sound quality.

The pitch converter according to the present invention is
Input means for inputting pitch information indicating a pitch different from the original sound;
Spectral distribution including a local peak and a spectrum before and after each local peak among a plurality of local peaks of spectral intensity based on an amplitude spectrum obtained by subjecting the sound waveform of the original sound to frequency analysis processing Generates amplitude spectrum data representing the amplitude spectrum distribution in the region with respect to the frequency axis, and generates phase spectrum data representing the phase spectrum distribution with respect to the frequency axis for each spectrum distribution region based on the phase spectrum obtained by the frequency analysis processing. Generating means for
First correcting means for correcting the amplitude spectrum data by moving the amplitude spectrum distribution represented by the amplitude spectrum data on the frequency axis according to the pitch information for each spectrum distribution region;
Setting means for setting a desired peak frequency on one side of the peak frequency corresponding to at least one local peak in the amplitude spectrum distribution represented by the amplitude spectrum data related to the correction by the first correction means;
Spectral envelopes corresponding to the peak frequencies respectively corresponding to a plurality of local peaks in the amplitude spectrum distribution represented by the amplitude spectrum data related to the correction by the first correction means and the peak frequencies related to the setting by the setting means Indicating means for indicating an envelope value to form
A spectrum distribution region having a spectrum distribution region having a spectral intensity of a local peak closest to the envelope value instructed by the instruction unit corresponding to the peak frequency related to the setting is indicated by the amplitude spectrum data related to generation by the generation unit Selecting means for selecting from among spectrum distribution regions having a peak frequency in a predetermined approximate relationship with the peak frequency related to the setting in
A first copy means for copying the amplitude spectrum data and phase spectrum data of the spectrum distribution region related to the selection by the selection means from the amplitude spectrum data and the phase spectrum data related to the generation by the generation means;
The amplitude spectrum data related to the copy is corrected by moving the amplitude spectrum distribution on the frequency axis according to the setting in the amplitude spectrum distribution represented by the amplitude spectrum data related to the copy by the first copying means. Second correcting means for
In the amplitude spectrum distribution represented by the amplitude spectrum data related to the correction by the first correction means, the spectrum intensity of the local peak for each amplitude spectrum distribution is associated with the peak frequency corresponding to the local peak by the indicating means. The spectral intensity of each spectral bin is corrected to match the indicated envelope value, and the spectral intensity of the local peak in the amplitude spectral distribution represented by the amplitude spectral data related to the correction by the second correcting means is indicated by the indicating means. A third correcting means for correcting the spectral intensity of each spectral bin so as to match the envelope value indicated corresponding to the peak frequency according to the setting;
The phase spectrum distribution represented by the phase spectrum data related to the generation by the generation means is corrected for each spectrum distribution region corresponding to the pitch change by the first correction means, and is copied by the first copy means. Fourth correcting means for correcting the phase spectrum distribution represented by the phase spectrum data according to the frequency change in the second correcting means;
Conversion means for converting the amplitude spectrum data related to the correction by the first to third correction means and the phase spectrum data related to the correction by the fourth correction means into a sound signal in the time domain It is.

  According to the above pitch conversion device, the peak corresponding to at least one local peak in the amplitude spectrum distribution represented by the amplitude spectrum data (amplitude spectrum data subjected to the pitch changing process) related to the correction by the first correction means. A desired peak frequency is set on one side of the frequency. This is to increase the number of local peaks that represent the spectral envelope.

  Further, a spectrum envelope is formed corresponding to each of the peak frequencies corresponding to the plurality of local peaks in the amplitude spectrum distribution represented by the amplitude spectrum data related to the correction by the first correction means, and the peak frequency related to the above setting. An envelope value to be indicated is indicated.

  The spectrum distribution region having the spectrum intensity of the local peak closest to the spectrum envelope value indicated corresponding to the peak frequency related to the setting is the peak related to the setting in the spectrum distribution region indicated by the amplitude spectrum data related to the generation by the generating means Amplitude spectrum data and phase spectrum data selected from the spectrum distribution region having a peak frequency having a predetermined approximate relationship with the frequency, and the amplitude spectrum data and phase spectrum data of the selected spectrum distribution region are generated by the generation means Copied from Then, the amplitude spectrum data and the phase spectrum data related to the copy are used for sound signal generation after undergoing necessary correction.

  As described above, the above pitch converter selects a spectrum distribution region having a pitch frequency before the pitch change close to the peak frequency related to the setting and having a spectral intensity of the local peak closest to the envelope value related to the instruction. Since the amplitude spectrum data and phase spectrum data in this spectrum distribution region are copied and used for sound signal generation, it becomes easy to obtain a natural tone color. In addition, when adjusting the spectral intensity of the local peak to the envelope value, it is not necessary to increase the spectral intensity of each spectral bin in the amplitude spectral data so much, so the sound quality of the output sound is a natural sound quality without noise. Become.

In the above pitch converter,
Calculating means for setting a plurality of candidate values of the time shift amount from the peak phase of the fundamental tone with respect to the phase spectrum data, and calculating the peak phase of the fundamental tone and the nth harmonic for each candidate value;
A group of peak phases corresponding to a candidate value that is closest to the flatness is selected from among a plurality of groups of peak phases respectively corresponding to the plurality of candidate values, and a fundamental tone and an nth harmonic in the selected group are selected. Fifth correcting means for correcting the peak phase of the fundamental tone and the n-th overtone in the phase spectrum data so as to coincide with the peak phase of
Instead of the fourth correction means, each phase corresponds to the pitch change in the first correction means for each spectrum distribution region in the phase spectrum distribution represented by the phase spectrum data related to the correction in the fifth correction means. And a sixth correction means for correcting
With respect to the phase spectrum data related to the correction by the sixth correction means, the time shift amount to the peak phase of the fundamental tone before the pitch change is calculated in consideration of the pitch change amount by the first correction means and Seventh correcting means for correcting the peak phase of the fundamental tone and the n-th overtone in the phase spectrum data related to the correction by the sixth correcting means according to the amount of time shift;
In the spectrum distribution region corresponding to the fundamental tone in the phase spectrum data related to the modification by the seventh modifying means, the peak phase other than the fundamental peak phase corresponds to the amount of change in the fundamental peak phase by the fifth and seventh modifying means. In the spectral distribution region corresponding to the nth harmonic, the phase other than the peak phase of the nth harmonic is corrected corresponding to the amount of change in the peak phase of the nth harmonic by the fifth and seventh correction means. Correction means,
The conversion means converts the amplitude spectrum data related to the correction by the first to third correction means and the phase spectrum data related to the correction by the fifth to eighth correction means into sound signals in the time domain. It may be made to do.

  According to this aspect, since the frequency analysis process is performed on the sound waveform of the original sound in a state where the center of the analysis window is shifted from the vocal cord vibration start position, in the phase spectrum distribution represented by the phase spectrum data generated by the generation unit, FIG. , 18 as described above, the peak phases of the fundamental tone and the n-th overtone are inconsistent. However, such an uneven state of peak phases is corrected by the calculation means and the second to fifth correction means.

  A plurality of candidate values for the amount of time shift from the peak phase of the fundamental tone are set for the phase spectrum data, and the peak phase of the fundamental tone and the nth harmonic is calculated for each candidate value. A group of peak phases corresponding to a candidate value that is in the closest phase alignment state is selected from among a plurality of groups of peak phases respectively corresponding to a plurality of candidate values, and the fundamental tone and n harmonics in the selected group are selected. The peak phase of the fundamental tone and the n-th overtone in the phase spectrum data is corrected so as to coincide with the peak phase.

  In the phase spectrum distribution represented by the phase spectrum data related to such correction, each frequency is corrected corresponding to the pitch change for each spectrum distribution region. Thereafter, the time shift amount to the peak phase of the fundamental tone before the pitch change is calculated in consideration of the pitch change amount, and the fundamental tone and the nth harmonic in the phase spectrum data according to the previous correction according to the calculated time shift amount The peak phase of is corrected again. The time shift at this time is performed in order to restore the time shift for phase alignment.

  Since the phase correction up to this point is for the peak phase, it is necessary to correct phases other than the peak phase. Therefore, in the phase spectrum data related to the re-correction, in the spectrum distribution region corresponding to the fundamental tone, the phase other than the peak phase of the fundamental tone is corrected in accordance with the amount of change in the peak phase of the fundamental tone. A phase other than the peak phase of the nth harmonic is corrected in accordance with the amount of change in the peak phase of the harmonic.

  When the phase spectrum data subjected to the correction as described above is used for sound signal generation, the waveform of the sound signal to be generated has a characteristic of a natural sound waveform in which the peak phases are aligned at the vocal cord vibration start position, Output sound with natural sound quality can be obtained.

In the above pitch converter,
At least one amplitude spectrum of the amplitude spectrum distribution represented by the amplitude spectrum data related to the correction by the first correction means, which is a noise component area within the spectrum distribution area indicated by the amplitude spectrum data generated by the generation means. A second copy means for copying a spectrum bin from a noise component region whose frequency band coincides with a spectrum lack region generated on one side of the distribution;
A fifth correction of amplitude spectrum data representing the at least one amplitude spectrum distribution among the amplitude spectrum data related to the correction so as to add a spectrum bin related to the copy by the second copy means to the spectrum absence region. Correction means,
The conversion means converts the amplitude spectrum data related to the correction by the first to third and fifth correction means and the phase spectrum data related to the correction by the fourth correction means into a sound signal in the time domain. You may make it convert.

  According to this aspect, the noise component region in which the frequency band coincides with the spectrum lack region generated on one side of at least one amplitude spectrum distribution among the amplitude spectrum distributions represented by the amplitude spectrum data related to the correction by the first correction means. Since spectral bins are copied from (region of low spectral intensity sufficiently far from local peak frequency) and added to the spectrum lacking region, the rawness of the original voice is reflected in the output sound. Output sound with natural sound quality.

  According to the present invention, a spectral distribution region having a peak frequency before the pitch change close to the peak frequency related to the setting and having a spectral intensity of a local peak closest to the indicated envelope value can be selected, or by frequency analysis. Align the peak phases that were not uniform in the obtained phase spectrum by calculation, or add spectrum bins copied from the noise component area whose frequency band matches the spectrum missing area of the amplitude spectrum distribution after pitch increase to the spectrum missing area. As a result, it is possible to generate an output sound with a natural sound quality.

  FIG. 1 shows a circuit configuration of a pitch converter according to an embodiment of the present invention. This pitch converter is configured to be controlled by a small computer 10.

  The bus 11 includes a CPU (Central Processing Unit) 12, a ROM (Read Only Memory) 14, a RAM (Random Access Memory) 16, a voice input unit 18, a control parameter input unit 20, an external storage device 22, and a display. A unit 24, a D / A (digital / analog) conversion unit 26, a MIDI (Musical Instrument Digital Interface) interface 28, a communication interface 30 and the like are connected.

  The CPU 12 executes various processes related to pitch conversion according to a program stored in the ROM 14, and the processes related to pitch conversion will be described later with reference to FIGS.

  The RAM 16 includes various storage units that are used as a working area when the CPU 12 performs various processes. As a storage unit related to the implementation of the present invention, for example, there are input data storage areas corresponding to the input units 18 and 20, respectively, and the details will be described later.

  The audio input unit 18 includes a microphone for inputting an audio signal, an audio input terminal, and the like, and includes an A / D (analog / digital) converter that converts the input audio signal into digital waveform data. Digital waveform data relating to the input is stored in a predetermined area in the RAM 16.

  The control parameter input unit 20 includes a keyboard capable of inputting characters, numbers, etc., a pointing device such as a mouse, and a parameter setting device such as a volume, and sets various control parameters used for pitch conversion processing. Is possible. As the control parameter, a pitch, a timbre, etc. can be set. Control parameter data representing the control parameter related to the setting is stored in a predetermined area in the RAM 16.

  The external storage device 22 is detachable from one or more types of recording media among HD (hard disk), FD (flexible disk), CD (compact disk), DVD (digital multipurpose disk), MO (magneto-optical disk) and the like. Is. When a desired recording medium is mounted on the external storage device 22, data can be transferred from the recording medium to the RAM 16. If the mounted recording medium is writable like HD or FD, the data in the RAM 16 can be transferred to the recording medium.

  As the program recording means, a recording medium of the external storage device 22 can be used instead of the ROM 14. In this case, the program recorded on the recording medium is transferred from the external storage device 22 to the RAM 16. Then, the CPU 12 is operated according to the program stored in the RAM 16. In this way, it is possible to easily add a program or upgrade a version.

  The display unit 24 includes a display such as a liquid crystal display, and can display various information such as a frequency analysis result to be described later.

  The D / A converter 26 converts the digital audio signal generated by the pitch conversion process into an analog audio signal. The analog audio signal sent from the D / A conversion unit 26 is converted into sound by a sound system 32 including an amplifier, a speaker, and the like.

  The MIDI interface 28 is provided for performing MIDI communication with a MIDI device 34 separate from the pitch conversion device. In the present invention, the MIDI interface 28 receives data for pitch conversion from the MIDI device 34. Used for.

  The communication interface 30 is provided to perform information communication with another computer 38 via a communication network (for example, a LAN (local area network), the Internet, a telephone line, etc.) 36. Programs and various data necessary for the implementation of the present invention may be fetched from the computer 38 to the RAM 16 or the external storage device 22 via the communication network 36 and the communication interface 30 in response to a download request.

  Next, an example of the voice conversion process will be described with reference to FIG. In step 40, an audio signal is input from the input unit 18 via a microphone or an audio input terminal, A / D converted, and digital waveform data representing the audio waveform of the input audio signal is stored in the RAM 16. FIG. 17 shows an example of the input speech waveform. Also, pitch information indicating the pitch of the sound to be generated (higher or lower pitch than the input sound signal) is input from the input unit 20 and stored in the RAM 16.

  In step 42, a section waveform is cut out for each frame of the stored digital waveform data (dividing the digital waveform data).

  In step 44, frequency analysis is performed by FFT analysis processing for each frame to detect a frequency spectrum (amplitude spectrum and phase spectrum). Data representing the frequency spectrum is stored in a predetermined area of the RAM 16. FIG. 5A shows an example of an amplitude spectrum obtained by performing frequency analysis by FFT analysis processing.

  Next, in step 46, the pitch is detected for each frame based on the amplitude spectrum, pitch data representing the detected pitch is generated, and stored in a predetermined area of the RAM 16.

In step 48, a plurality of local peaks of the spectrum intensity (amplitude) are detected on the amplitude spectrum for each frame. In order to detect the local peak, a method of detecting a peak having the maximum amplitude value for a plurality of neighboring peaks (for example, four) can be used. FIG. 5A shows a plurality of detected local peaks P ₀ , P ₁ , P ₂ .

In step 50, a spectrum distribution region corresponding to each local peak on the amplitude spectrum is designated for each frame, and amplitude spectrum data representing the amplitude spectrum distribution in the region with respect to the frequency axis is generated and stored in a predetermined region of the RAM 16. Let As a method for designating a spectral distribution region, a method of cutting the frequency axis in half between two adjacent local peaks and allocating each half to a spectral distribution region including the closer local peak can be employed. . FIG. 5A shows an example in which spectral distribution regions R ₀ , R ₁ , R ₂ ... Each including local peaks P ₀ , P ₁ , P ₂ .

In step 52, phase spectrum data representing the phase spectrum distribution in each spectrum distribution region with respect to the frequency axis based on the phase spectrum for each frame is generated and stored in a predetermined region in the RAM 16. FIG. 14 shows an amplitude spectrum distribution am ₀ and a phase spectrum distribution ph ₀ in a spectrum distribution region of a certain frame.

  The processing in steps 54 to 68 is performed on the amplitude spectrum data or phase spectrum data of each frame. In step 54, the amplitude spectrum distribution arrangement is changed to change the pitch in accordance with the pitch information related to the input in step 40 with respect to the amplitude spectrum data.

FIG. 5 shows an example of the pitch changing process according to the present invention. The same parts as those in FIG. As the pitch changing process, a pitch increasing process is performed. In step 40, pitch information indicating a pitch higher than the input audio signal is input. Assuming that the frequency (peak frequency of the fundamental tone) corresponding to the pitch data obtained in step 46 is f ₀ and the frequency corresponding to the pitch information related to the input is f ₀₁ , the pitch change ratio T is T = f ₀₁ / f ₀ It becomes.

In the pitch increase processing, as shown in FIG. 5A, perfect harmonic frequencies 2f ₀ and 3f ₀ are assumed, in which the peak frequency f _{0 of the} fundamental tone is doubled and tripled, respectively. Then, the difference Δf ₁ = (f ₁ −2f ₀ ) between the peak frequency f ₁ of the first harmonic and the perfect harmonic frequency 2f ₀ is held, and the difference between the peak frequency f ₂ of the second harmonic and the perfect harmonic frequency 3f ₀ is maintained. The difference Δf ₂ = (f ₂ −3f ₀ ) is held. The difference is held by storing difference data representing the differences Δf ₁ and Δf ₂ in a predetermined area in the RAM 16.

Next, the amplitude spectrum distribution in the region R ₀ is moved to the high pitch side on the frequency axis so that the fundamental tone peak P ₀ is located at the frequency f ₀₁ = f ₀ T. That is, the amplitude spectrum data in the region R ₀ is corrected to enable such movement (specifically, the frequency of each spectral bin is corrected in the amplitude spectrum distribution). In addition, it is assumed that perfect harmonic frequencies 2f ₀₁ and 3f ₀₁ are obtained by doubling and triple the peak frequency f ₀₁ of the fundamental tone after the pitch rise. While adopting the frequency f _{11 =} 2f 01 ₊ Δf ₁ which is shifted in response to full harmonic frequency 2f ₀₁ to the difference Delta] f ₁ described above as a peak frequency of the first harmonic of the post elevating pitch, the second harmonic of the post elevating pitch the peak frequency employs the full harmonic frequency _f 21 = _{3f 01} + Δf ₂ of the frequency _{3f 01} is shifted in response to the difference Delta] f ₂ described above.

Peak P ₁ of the first harmonic of the post elevating pitch is moved amplitude spectrum distribution in the region R ₁ so as to be positioned on the frequency f _{11 =} 2f 01 ₊ Δf ₁ treble side on the frequency axis (i.e., such movement modifying the amplitude spectrum data of the region R ₁ to enable any). Further, the peak P ₂ of the second harmonic of the post elevating pitch is moved amplitude spectrum distribution region R ₂ so as to be positioned in the frequency f _{21 =} 3f 01 ₊ Δf ₂ treble side on the frequency axis (i.e., like this to permit a movement to correct the amplitude spectrum data region R _2).

According to the pitch change processing described above with reference to FIG. 5, even if the pitch change ratio T is increased, the peak frequencies f ₁₁ and f ₂₁ of the harmonics after the pitch increase are not significantly shifted from the perfect harmonic frequencies 2f ₀₁ and 3f ₀₁ , respectively. . Therefore, an output sound with natural sound quality can be obtained.

As described above with reference to FIG. 5, the process of holding the difference such as Δf ₁ before the pitch change and reflecting it in the harmonic frequency such as f ₁₁ after the pitch change is not limited to the case where the pitch is increased but also when the pitch is decreased. Can be applied. When the pitch is lowered, the frequency difference is small, but an output sound with natural sound quality can be obtained as in the case of the pitch rise.

Step 56 of FIG. 2 modifies the spectral intensity of each spectral bin in the amplitude spectral distribution to match the local peak with respect to the amplitude spectral data to the spectral envelope. In the example shown in FIG. 5B, the spectrum envelope EVb having the same shape as the spectrum envelope (spectrum envelope of the original speech) EVa connecting the peaks P _{0 to} P ₂ shown in FIG. The spectral intensity of each spectral bin is corrected in the amplitude spectral distribution so that the peaks P _{0 to} P ₂ are matched. As a result, the same timbre as the original voice can be obtained. When it is desired to obtain a tone different from the original voice, the spectral envelope EVb is appropriately changed as described above with reference to FIG. 13, and the spectral intensity of each spectral bin is corrected in the amplitude spectral distribution according to the changed spectral envelope EVb. Good.

  FIG. 6 shows an example of the pitch / timbre change process according to the present invention. In this example, the pitch reduction process is performed in step 54 as the pitch change process, and a timbre different from the original voice is given as the timbre change process. Processing is performed at step 56. In FIG. 6, the same parts as those in FIG. 16 are denoted by the same reference numerals.

FIG. 6A shows an amplitude spectrum distribution for each spectrum distribution region similar to that shown in FIG. In the pitch reduction process, the amplitude spectrum distributions of the spectrum distribution regions R ₀ , R ₁ , and R ₂ are moved to the bass side on the frequency axis as shown in FIG. 6B (that is, such movement is possible). Correct the amplitude spectrum data accordingly). In the amplitude spectrum distribution after movement, the peak frequencies corresponding to the local peaks P ₀₁ , P ₁₁ , and P ₂₁ are F ₀ , F ₂ , and F ₅ , respectively.

In the tone color changing process, a predetermined spectrum envelope EVc having a shape different from that of the spectrum envelope EVa of the original sound is assumed. In this case, the spectrum envelope EVc cannot be sufficiently expressed only by the amplitude spectrum distribution corresponding to the peak frequencies F ₀ , F ₂ , and F ₅ . Therefore, the peak frequency F ₁ is between the peak frequencies F ₀ and F ₂ , the peak frequencies F ₃ and F ₄ are between the peak frequencies F ₂ and F _5, and the high frequency side of the peak frequency F ₅ is Peak frequencies F ₆ and F ₇ are set, respectively. This setting process may be performed by operating the mouse or keyboard of the input unit 20, or may be executed by reading out frequency information stored in association with the type of tone (or envelope EVc) desired to be changed. It may be.

Next, a spectrum envelope value (envelope value for forming the spectrum envelope EVc) is designated for each peak frequency of F _{0 to} F ₇ . In this case, in the amplitude spectrum distribution of the spectrum distribution region R _{0 to} R ₂ shown in FIG. 6A, the spectrum intensity is enveloped for any of the local peaks P _{0 to} P ₂ for each peak frequency of F _{0 to} F _7. Specify as a value. When designating the envelope value, the graph shown in FIG. 7 can be used.

FIG. 7 indicates the spectrum intensity as an envelope value for each peak frequency after the pitch change with respect to the amplitude spectrum distribution of FIG. 6 (A), and the peak frequencies F _{0 to} F ₇ after the pitch change are indicated on the x-axis. , Y-axis shows peak frequencies f _{0 to} f ₃ before the pitch change, and right side shows spectral intensities M _{0 to} M ₃ of local peaks P _{0 to} P ₃ corresponding to the peak frequencies f _{0 to} f ₃ , respectively. Show. The graph of FIG. 7 is named as a timbre mapping function diagram by the inventor of the present application, and is convenient for use in timbre setting. Line N (x) corresponds to the case where the timbre of the original voice is not changed, and line K (x) corresponds to the case where the timbre of the original voice is changed.

When the graph of FIG. 7 is used, the lower limit frequency fa of the region R ₀ to the upper limit frequency fd of the region R ₂ so as to connect the peaks P _{0 to} P ₂ in the amplitude spectrum distribution of the regions R _{0 to} R ₂ of FIG. A spectral envelope EVa that extends up to is created by interpolation processing or the like. Further, the display screen of the display unit 24 includes the amplitude spectrum distribution (including the envelope EVa) of the regions R _{0 to} R ₂ in FIG. 6A and the peak frequencies F ₀ , F ₂ , F in FIG. 6B. _The amplitude spectrum distribution corresponding to ₅ and the graph of FIG. 7 are displayed. In such a display state, mark ₁ like K in position according to the specified by performing a positioned against the cursor to a desired position the mouse or by operating the keyboard or the like on the graph of FIG. 7 of the input unit 20 is displayed Is done. As a result, an envelope value is instructed, and the envelope value according to the instruction is stored in the RAM 16 and displayed as a dot on the envelope EVa. The position designation on the display screen may be performed using an input pen or the like.

As an envelope value corresponding to the peak frequency F ₀ (corresponding to the point P ₀₁ in FIG. 6B), by specifying the position of the mark K ₀ , a frequency y = K (F ₀ ) (a frequency lower than f ₀ ) ) slightly larger spectral intensity than the spectral intensity (M ₀ on the envelope EVa corresponding to) is indicated. As an envelope value corresponding to the peak frequency F ₁ (corresponding to the point P ₀₂ in FIG. 6B), by specifying the position of the mark K ₁ , a frequency (frequency higher than f ₀ ) becomes y = K (F ₁ ). ) slightly smaller spectral intensity than the spectral intensity (M ₀ on the envelope EVa corresponding to) is indicated.

Similarly, as the envelope value corresponding to the peak frequency F ₂ (corresponding to the point P ₁₁ ), the spectrum intensity corresponding to y = K (F ₂ ) by the position designation of the mark K ₂ (spectrum intensity slightly smaller than M ₁ ). ) Is instructed. Further, as the envelope value corresponding to the peak frequency F ₃ (corresponding to the point P ₁₂ ), the spectrum intensity corresponding to y = K (F ₃ ) (spectrum intensity slightly smaller than M ₀ ) is designated by the position designation of the mark K _3. As indicated, the envelope value corresponding to the peak frequency F ₄ (corresponding to the point P ₁₃ ) is the spectrum intensity corresponding to y = K (F ₄ ) by the position designation of the mark K ₄ (a spectrum slightly smaller than M ₀ ). Strength) is indicated. Further, envelope values corresponding to the peak frequencies F ₅ , F ₆ , and F ₇ (corresponding to points P ₂₁ , P ₂₂ , and P ₂₃ ) are set to y = K (by position designation of the marks K ₅ , K ₆ , and K ₇ ). F ₅ ), y = K (F ₆ ), and spectrum intensity corresponding to y = K (F ₇ ) (spectrum intensity in the vicinity of peak P ₂ ) are respectively indicated.

When an envelope value is indicated for each peak frequency of F _{0 to} F ₇ , the envelope value can be indicated in any of the spectrum distribution regions R _{0 to} R ₂ , but in order to obtain a natural tone, F ₀ for each peak frequency of the to F ₇ to indicate the envelope values in the spectral distribution region having a pitch change previous peak frequency near the peak frequency is desirable. In the graph of FIG. 7, the peak frequencies of F _{0 to} F ₇ are shown on the x-axis, and the peak frequencies f _{0 to} f ₃ before the peak change are shown on the y-axis, so that the peaks of F _{0 to} F ₇ are shown. For each frequency, the envelope value can be indicated with respect to the peak frequency before the pitch change close to that. In the example of FIG. 7, the peak frequencies F ₀ , F ₁ , F ₃ , and F ₄ are in the spectrum distribution region R ₀ , the peak frequency F ₂ is in the spectrum distribution region R ₁ , and the peak frequencies F _{5 to} F ₇ are in the spectrum distribution region R ₀ . each instructs the envelope values in the spectral distribution region R _2.

In the above example, the envelope value is designated using the graph of FIG. 7, but the envelope value can be designated without using the graph of FIG. For example, the display screen of the display unit 24 includes the amplitude spectrum distribution (including the envelope EVa) of the regions R _{0 to} R ₂ in FIG. 6A and the peak frequencies F ₀ , F ₂ , F in FIG. ₅ and an amplitude spectrum distribution corresponding to ₅ are displayed. In such a display state, the envelope value can be indicated by specifying the position for each peak frequency of F _{0 to} F ₇ on the envelope EVa. Each envelope value related to the instruction is displayed as a point on the envelope EVa. Also, be omitted display envelope EVa, you can instruct the envelope value by specifying a position in the vertical direction (or oblique lateral direction) for example a peak P ₀ as a reference. In this case, the envelope value according to the instruction may be displayed as a point at the designated position in the vicinity of the reference peak. Further, it is possible to instruct the envelope value using the spectrum envelope EVc. For example, the display screen of the display unit 24 displays the amplitude spectrum distribution corresponding to the peak frequencies F ₀ , F ₂ , and F ₅ in FIG. 6B and the envelope EVc. In such a display state, the envelope value may be indicated by designating points P ₀₁ , P ₀₂ , P _{11 to} P ₁₃ , and P _{21 to} P ₂₃ on the envelope EVc. As the envelope EVc, an envelope having an arbitrary shape can be drawn on the display screen using an input pen or the like.

Next, for the peak frequencies F ₁ , F ₃ , F ₄ , F ₆ , and F ₇ according to the setting, a spectral distribution region having the spectral intensity of the local peak closest to the envelope value is determined for each envelope value according to the instruction. select. In this case, for each peak frequency of F ₁ , F ₃ , F ₄ , F ₆ , and F _7, an indication is made from among a plurality of spectral distribution regions having a peak frequency before pitch reduction that has a predetermined approximate relationship with the peak frequency. A spectral distribution region having the spectral intensity of the local peak closest to the envelope value is selected. As the predetermined approximate relationship, for example, a relationship in which the closeness rank is in the range of 1 to 2 can be adopted. This is because it is easier to obtain a natural tone when the peak frequency is closer before and after the pitch change. In the example of FIG. 7, for the peak frequency F ₁ , the spectrum distribution region having the peak P ₀ is selected because the peak frequency closest to the position of the mark K ₁ is f ₀ and the peak spectral intensity is M _0. Is done. Similarly, the spectrum distribution region having the peak P ₀ is selected for the peak frequencies F ₃ and F ₄ , and the spectrum distribution region having the peak P ₂ is selected for the peak frequencies F ₆ and F ₇ . The selection process can be automatically performed based on an envelope value instruction operation using a mouse or a keyboard of the input unit 20.

Next, the amplitude spectrum data and the phase spectrum data of each spectrum distribution region related to the selection are copied from the amplitude spectrum data and the phase spectrum data related to the generation in steps 50 and 52. For the peak frequencies F ₁ , F ₃ and F ₄ , the amplitude spectrum data and phase spectrum data in the spectrum distribution region R ₀ are copied, and for the peak frequencies F ₆ and F ₇ , the amplitude spectrum data in the spectrum distribution region R ₂ and Copy phase spectrum data.

Next, for each amplitude spectrum data related to the copy, the amplitude spectrum distribution is moved on the frequency axis so that the peak frequency corresponding to the local peak in the amplitude spectrum distribution represented by the amplitude spectrum data is changed to the peak frequency related to the setting. (Ie, modify the amplitude spectrum data to allow such movement). For example, for the amplitude spectrum data copied corresponding to the peak frequency F ₁ , the amplitude spectrum distribution is moved to the high pitch side on the frequency axis so that the peak frequency is changed from f ₀ to F ₁ . The amplitude spectrum data respectively copied to correspond to the peak frequency F _3, F ₄ moves, the peak frequency of the amplitude spectrum distribution to change respectively from f ₀ to F _3, F ₄ treble side on the frequency axis To do. The amplitude spectrum data respectively copied to correspond to the peak frequency F _6, F ₇ move, the amplitude spectrum distribution to change respectively the peak frequency F _6, F ₇ from f ₂ treble side on the frequency axis To do.

Next, in the amplitude spectrum distribution represented by the amplitude spectrum data that has peak frequencies F ₀ , F ₂ , and F ₅ by the pitch reduction process, the spectral intensity of the local peak for each amplitude spectrum distribution is indicated in the envelope according to the previous instruction. The spectral intensity of each spectral bin is modified to match the value. For example, in the amplitude spectrum distribution corresponding to the peak frequency F ₀ , the spectrum intensity of each spectrum bin is corrected so that the spectrum intensity of the local peak matches the envelope value (corresponding to the point P ₀₁ ) according to the previous instruction. Even in the amplitude spectrum distribution corresponding to each of the peak frequencies F ₂ and F ₅ , the spectrum of each spectrum bin so that the spectrum intensity of the local peak matches the envelope value (corresponding to the points P ₁₁ and P ₂₁ ) according to the previous instruction. Correct the strength.

Such correction of the spectrum intensity is performed in the same manner for each amplitude spectrum data related to the copy. That is, in the amplitude spectrum distribution corresponding to each of the peak frequencies F ₁ , F ₃ , F ₄ , F ₆ , and F ₇ , the spectral intensity of the local peak is set as the envelope value (points P ₀₂ , P ₁₂ , P ₁₃ , P ₂₂ , and P ₂₃ ), and the spectral intensity of each spectral bin is corrected.

According to the pitch / tone color changing process as described above, as shown in FIG. 6B, eight amplitude spectrum distributions corresponding to the peak frequencies F _{0 to} F ₇ are peaks P ₀₁ , P ₀₂ , P _{11 to} P _13. , P _{21 to} P ₂₃ are arranged in accordance with the spectrum envelope EVc. In FIG. 6B, the amplitude spectrum distribution is overlapped for each adjacent spectrum distribution region. However, for each adjacent spectrum distribution region, the low frequency side spectrum distribution region is near the center frequency position of both regions. By newly setting the upper limit frequency and the lower limit frequency of the spectrum distribution region on the treble side, overlapping of the amplitude spectrum distribution can be prevented. Alternatively, the spectral intensities of spectral bins having the same frequency may be simply added as they are at the portions where the amplitude spectral distributions overlap for each adjacent spectral distribution region. Note that the timbre changing process as described above with reference to FIG. 6 can be performed not only when the pitch is lowered but also when the pitch is raised.

  According to the pitch / timbre change processing described above with reference to FIG. 6, the spectral distribution region having the pitch intensity before the pitch change close to the set peak frequency and having the local peak spectral intensity closest to the indicated envelope value. Is selected, and the amplitude spectrum data and phase spectrum data in this spectrum distribution region are copied and used for generating an audio signal, so that it is easy to obtain a natural timbre. In addition, when adjusting the spectral intensity of the local peak to the envelope value, it is not necessary to increase the spectral intensity of each spectral bin in the amplitude spectral data so much, so the sound quality of the output sound is a natural sound quality without noise. Become.

Then, (without going through the steps 58, 60) according to the route _{J 1} in FIG. 3 proceeds to step 62. In step 62, the phase spectrum distribution arrangement is changed for the phase spectrum data in response to the change of the amplitude spectrum distribution arrangement for the amplitude spectrum data. That is, when the pitch increase process described above with reference to FIG. 5 or the pitch decrease process described above with reference to FIG. 6 is performed, the phase spectrum distribution represented by each phase spectrum data related to the generation in step 52 is described above with reference to FIGS. Thus, correction is made for each spectral distribution region corresponding to the pitch change in step 54. When the pitch reduction process described above with reference to FIG. 6 is performed, the phase spectrum distribution represented by each phase spectrum data related to copying corresponds to the phase spectrum distribution and corresponds to the frequency change of the amplitude spectrum distribution related to copying. To correct. For example, for the phase spectrum distribution corresponding to the peak frequency F ₁ , the phase spectrum distribution is corrected in response to the frequency change from f ₀ to F ₁ . The phase spectrum distributions corresponding to the other peak frequencies F ₃ , F ₄ , F ₆ and F ₇ are similarly corrected.

After this, (without going through the steps 64, 66) according to the route _{J 2} in Figure 3 proceeds to step 68. The process of step 68 will be described later with reference to FIG.

8 to 11 show an example of phase alignment processing according to the present invention. In these drawings, the horizontal axis indicates the frequency f and the vertical axis indicates the phase (0 to 2π). Figure 8 shows the same peak phase to that shown in FIG. 18, these peak phase, for example the analysis window FW of the center W _C a vocal cord vibration start position t _S2 as described above with reference to FIG. _17, t _S3 are those obtained by performing an FFT analysis in a state matching the center position near between, it is in irregular state peak phase phi ₀ to [phi] ₅ respectively corresponding to the peak frequency f ₀ ~f _5. The peak phases φ _{0 to} φ ₅ belong to six phase spectrum distributions (corresponding to f _{0 to} f ₅ respectively) represented by the phase spectrum data of a certain frame related to the generation in step 52.

In the phase alignment process, the time (time shift amount) required from the state of FIG. 17 to the state of FIG. 19 is obtained, and the peak phases φ _{0 to} φ ₅ of FIG. As shown in FIG. _5, the peak phases φ ₀ ′ to φ ₅ ′ are converted into a flat shape. In step 58 of FIG. 3, a large number of time shift amount candidate values from the fundamental peak phase φ ₀ are set for the phase spectrum data, and the fundamental and overtone peak phases are calculated for each candidate time shift amount. In order to set the candidate value of the time shift amount, the phase candidate value φ _0C is set to about 40 to 80 points between ₀ and 2π, and the time shift is performed according to the following equation 2 for each candidate value φ _0C. setting a candidate value TS _C amount.

Here, f ₀ is the peak frequency of the fundamental tone. As an example, if the candidate value phi _0C phase and 40 points, it is 40 possible values TS _C of the time shift amounts. Then, the peak phase of the fundamental and overtone is calculated according to the formula for a number 3 for each candidate value TS _C.

Here, i is the number of the peak phase, i = 0 for the fundamental tone, i = 1 for the first harmonic, i = 2 for the second harmonic, and so on. For a single candidate value _{TS C, f 1 = 2f 0} , f 2 = the 3f 0 _... to, fundamental tone of the peak phase _{φ 0C = φ 0 + 2πf 0} × TS C, the peak phase of the first harmonic is phi _1C = φ ₁ + 2π × 2f ₀ × TS _C , and the peak phase of the second overtone is φ _2C = φ ₂ + 2π × 3f ₀ × TS _C. When the number of fundamental and harmonic and N, N-number of peak phase is calculated for one of the candidate values TS _C. Since the candidate value TS _C is 40, when the N-number of peak phase corresponding to one candidate value TS _C and 1 group, is required 40 group peak phase.

Next, in step 60 of FIG. 3, an average value φave of N peak phases is obtained for each group, and a sum Σabs (φic−φave) of absolute deviation amounts of the peak phases φic from this average value is obtained. The state in which the sum of the absolute deviation amounts is the smallest is the phase that is closest to the flatness (Maximally Flat Phase Alignment) state, which is hereinafter referred to as the MFPA state. Selecting a group of peak phase corresponding to the candidate value TS _C to be MFPA state.

As an example, when N = 6, the first to sixth peak phases respectively corresponding to the peak phases φ _{0 to} φ ₅ in FIG. Peak phase phi ₀ in MFPA state as shown in FIG. 9 by modifying the peak phase phi ₀ to [phi] ₅ of FIG. 8, as each of the first to sixth peak phase within a group according to selected matches '~ Φ ₅ ' is obtained. The phase spectrum data indicating the peak phases φ _{0 to} φ ₅ in FIG. 8 indicates the peak phases φ ₀ ′ to φ ₅ ′ in FIG. 9 as a result of correction.

  Next, in step 62 of FIG. 3, the phase spectrum distribution arrangement is changed with respect to the phase spectrum data in response to the change of the amplitude spectrum distribution arrangement with respect to the amplitude spectrum data. As an example, when a pitch reduction process similar to that described above with reference to FIG. 6 is performed, as shown in FIG. 10, each phase distribution in the phase spectrum data represented by the phase spectrum data related to the correction in FIG. Correct the frequency to accommodate the pitch drop.

In the example of FIG. 10, the peak frequencies F ₀₁ , F ₁₁ , F ₂₁ , F ₃₁ , F ₄₁ , and F ₅₁ have peak frequencies f ₀ , f ₁ , f ₂ , f ₃ , f ₄ , and f ₅ as frequency axes, respectively. The peak frequencies F ₀₂ , F ₁₂ , F ₂₂ , F ₃₂ , F ₄₂ , and F ₅₂ are peak frequencies f ₀ , f ₁ , f ₂ , f ₃ , f ₄ , it is modified with a f ₅ to the copy process. When the frequency of each spectrum bin is changed in the phase spectrum distribution to which the peak phase φ ₀ ′ belongs in response to the change of the peak frequency from f ₀ to F ₀₁ , the peak phase φ ₀₁ ′ is at the position corresponding to the peak frequency F _01. can get. Similarly, peak phases φ ₀₂ ′, φ are respectively located at positions corresponding to the peak frequencies F ₀₂ , F ₁₁ , F ₁₂ , F ₂₁ , F ₂₂ , F ₃₁ , F ₃₂ , F ₄₁ , F ₄₂ , F ₅₁ , F _{52. 11 ', φ 12', φ} 21 ', φ 22', φ 31 ', φ 32', φ 41 ', φ 42', φ 51 ', φ 52' is obtained. If the peak frequencies F ₀₂ , F ₁₂ , F ₂₂ , F ₃₂ , F ₄₂ , and F ₅₂ are not set in the timbre change process, the peak phases φ ₀₂ ′, φ ₁₂ ′, φ ₂₂ ′, φ _{32 ', φ 42', φ} 52 ' does not exist.

  Next, in step 64 of FIG. 3, the amount of time shift to the peak phase of the fundamental tone before phase alignment is obtained in consideration of the amount of phase change accompanying the change of pitch with respect to the phase spectrum data, and each time shift amount is determined according to the obtained time shift amount. Correct the peak phase. The time shift at this time is performed in order to restore the time shift for phase alignment. The time shift amount TS at this time can be obtained according to the following equation (4).

Here, φ ₀ is the peak phase of the fundamental tone before phase alignment (see FIG. 8), Δφ ₀ is the phase change amount associated with the pitch change, and is obtained by the equation (1), and φ ₀ ′ is after phase alignment. The peak phase of the fundamental tone (see FIG. 9) and T represent the pitch change ratio. After obtaining the time shift amount TS, the phase change amount Δφ _P is obtained according to the expression Δφ _P = 2πf _P × TS according to the time shift amount TS, and the obtained phase change amount Δφ _{P is set} to each peak phase in FIG. In addition, the new peak phase of FIG. 11 is calculated. Here, f _P indicates a peak frequency such as F ₀₁ after the pitch change. For example, the peak phase φ ₀₁ corresponding to the peak frequency F ₀₁ can be obtained by obtaining Δφ ₀₁ by the equation Δφ ₀₁ = 2πF ₀₁ × TS and then adding Δφ ₀₁ to φ ₀₁ ′ in FIG. Similarly, the peak phases φ ₀₂ , φ ₁₁ , and F ₅₂ corresponding to the peak frequencies F ₀₂ , F ₁₁ , F ₁₂ , F ₂₁ , F ₂₂ , F ₃₁ , F ₃₂ , F ₄₁ , F ₄₂ , F ₅₁ , F ₅₂ , respectively. _{φ 12, φ 21, φ 22} , φ 31, φ 32, is _{φ 41, φ 42, φ 51} , φ 52 is determined. FIG. 11 shows the result of correcting each peak phase in FIG. 10 according to the time shift amount TS.

Since the phase correction up to this point is for the peak phase, it is necessary to correct a phase other than the peak phase for each spectrum distribution region. Therefore, in step 66 of FIG. 3, the phase of the spectrum bin other than the peak phase is corrected in accordance with the change amount of the peak phase with respect to the phase spectrum data. For example, the phase spectrum distribution belongs peak phase phi _{01, 'a} peak phase change amount to, phi ₀₁ in FIG. 10' phi ₀ in FIG. 9 phi ₀ in FIG. 8 (same as phi _{0 'in} FIG. 9) corresponding to the sum of the peak phase change amount to phi ₀₁ in FIG. 11 to correct the peak phase phi ₀₁ except for the phase of each spectral bin. Similarly, in the phase spectrum distribution to which the peak phases φ ₀₂ , φ ₁₁ , φ ₁₂ , φ ₂₁ , φ ₂₂ , φ ₃₁ , φ ₃₂ , φ ₄₁ , φ ₄₂ , φ ₅₁ , φ ₅₂ belong respectively, The phase is corrected according to the change amount of the peak phase.

  According to the phase alignment process described above with reference to FIGS. 8 to 11, the peak phase that was not uniform as shown in FIG. 8 is corrected by calculation so as to be in the MFPA state as shown in FIG. When spectrum data is used for voice signal generation, the waveform of the generated voice signal has the characteristics of a natural voice waveform in which the peak phases are aligned at the vocal cord vibration start position, and an output sound with natural sound quality is obtained. Can do.

  In step 68 of FIG. 3, a spectrum bin as a noise component is added to the spectrum lack region with respect to the amplitude spectrum data whose pitch has been increased. The process of step 68 is performed only when the pitch increase process is performed as the pitch change process in step 54, and is not performed when the pitch decrease process described above with reference to FIG. 6 is performed.

FIG. 12 shows an example of the spectrum addition processing, and the same parts as those in FIG. FIG. 12A shows an amplitude spectrum similar to that shown in FIG. In the pitch increase process, the amplitude spectrum distributions of the regions R ₀ and R ₁ are moved to the high pitch side on the frequency axis as shown in FIG. 12B, and therefore one side and the other side of the amplitude spectrum distribution having the peak P _0. Each have a spectral lack region Q ₁ and Q ₂ . Spectral lack region Q ₂ are present between the amplitude spectrum distribution having the amplitude spectrum distribution and a peak P ₁ has a peak P _0. A spectrum lack region such as Q ₁ does not exist in the amplitude spectrum of natural speech (for example, one shown in FIG. 12A).

In the spectrum distribution region R ₀ shown in FIG. 12A, the regions K ₁₁ and K _{12 that} are sufficiently separated from the peak P ₀ (for example, 55 Hz or more) are both noises including residual components separated from the main lobe. It is a component area. Also in the spectrum distribution region R ₁ , the regions K ₂₁ and K _{22 that} are sufficiently separated from the peak P ₁ are both noise component regions. In step 68, as shown in FIG. 12, added to the spectrum lack region Q ₁ by copying the spectral bin from the spectrum lack region Q _1, a noise component region k ₁ frequency bands match _(partial region K ₁₁₎ with modifying the amplitude spectrum data as such to be added to the spectrum lack region Q ₂ by copying the spectral bins from the noise component region k ₂ where the spectral lack region Q ₂ frequency bands match _(partial region K ₂₁₎ To correct the amplitude spectrum data.

  According to the spectrum addition process in step 68, the rawness of the original sound can be reproduced, and the sound quality of the output sound is improved. Note that the spectrum addition processing in step 68 can be performed not only when the pitch increase processing described above with reference to FIG. 5 is performed but also when the pitch increase processing described above with reference to FIG. 13 is performed.

  In step 70, the amplitude spectrum data and the phase spectrum data are converted into a time domain audio signal (digital waveform data). This conversion process can be performed by steps 70a to 70c as an example. That is, in step 70a, inverse FFT processing is performed on the frequency domain frame data (amplitude spectrum data and phase spectrum data) to obtain a time domain audio signal. In step 70b, a windowing process is performed on the audio signal in the time domain. This process multiplies the time-domain audio signal by a time window function. In step 70c, overlap processing is performed on the time domain audio signal. This process connects audio signals in the time domain while overlapping waveforms for sequential frames.

  In step 72, the audio signal is output to the D / A converter 26. As a result, sound related to pitch conversion is generated from the sound system 32.

  The present invention is not limited to the above-described embodiment, and can be implemented in various modifications. For example, the following changes are possible.

  (1) In the above embodiment, the voice signal input from the input unit 18 is converted into digital waveform data to perform pitch conversion. However, the digital waveform data representing the voice waveform of the original voice is stored in the storage means (ROM 14, ROM). RAM 16 or external storage device 22), and the desired digital waveform data may be read from the storage means by keyboard operation of input unit 20 to perform pitch conversion. Also, digital waveform data representing the voice waveform of the original voice may be acquired via the interface 28 or 30 to perform pitch conversion.

  (2) Pitch data, amplitude spectrum data, and phase spectrum data obtained in steps 46, 50, and 52 are stored in the storage means as described above, and desired pitch data and amplitude are obtained by operating the keyboard of the input unit 20 or the like. The pitch conversion may be performed by reading the spectrum data and the phase spectrum data from the storage means. In addition, pitch conversion may be performed by acquiring pitch data, amplitude spectrum data, and phase spectrum data corresponding to the voice waveform of the original waveform via the interface 28 or 30.

  (3) The present invention can be applied not only to pitch conversion of speech but also to pitch conversion of musical sounds.

It is a block diagram which shows the circuit structure of the pitch converter based on one Embodiment of this invention. It is a flowchart which shows a part of pitch conversion process. It is a flowchart which shows a part of other pitch conversion process of FIG. It is a flowchart which shows another part of the pitch conversion process of FIG. It is a spectrum figure for demonstrating the pitch change process which concerns on this invention. It is a spectrum figure for demonstrating the pitch and timbre change process which concerns on this invention. It is a graph which shows a spectrum envelope value for every peak frequency after a pitch change regarding the amplitude spectrum of FIG. It is a figure which shows an example of a peak phase. It is a figure which shows the peak phase obtained by giving a time shift and selection process to the peak phase of FIG. It is a figure which shows the peak phase obtained by giving a spectrum distribution arrangement change process to the peak phase of FIG. It is a figure which shows the peak phase obtained by giving a time shift process to the peak phase of FIG. It is a spectrum figure for demonstrating the spectrum addition process at the time of pitch rise. It is a spectrum figure for demonstrating the conventional pitch change process. It is a spectrum figure which illustrates the amplitude spectrum distribution and phase spectrum distribution before a pitch change. It is a spectrum figure which illustrates the amplitude spectrum distribution and phase spectrum distribution after pitch change. It is a spectrum figure for demonstrating the pitch and timbre change process which concerns on inventors' research. It is a wave form diagram which shows an example of the time position of the analysis window in the FFT analysis process which concerns on inventors' research. It is a figure which shows the peak phase obtained by the FFT analysis in the analysis window position of FIG. It is a wave form diagram which shows the other example of the time position of the analysis window in the FFT analysis process which concerns on inventors' research. It is a figure which shows the peak phase obtained by the FFT analysis in the analysis window position of FIG.

Explanation of symbols

  10: small computer, 11: bus, 12: CPU, 14: ROM, 16: RAM, 18: voice input unit, 20: control parameter input unit, 22: external storage device, 24: display unit, 26: D / A Conversion unit, 28: MIDI interface, 30: communication interface, 32: sound system, 34: MIDI device, 36: communication network, 38: other computer.

Claims (6)

Input means for inputting pitch information indicating a pitch different from the original sound;
Spectral distribution including a local peak and a spectrum before and after each local peak among a plurality of local peaks of spectral intensity based on an amplitude spectrum obtained by subjecting the sound waveform of the original sound to frequency analysis processing Generates amplitude spectrum data representing the amplitude spectrum distribution in the region with respect to the frequency axis, and generates phase spectrum data representing the phase spectrum distribution with respect to the frequency axis for each spectrum distribution region based on the phase spectrum obtained by the frequency analysis processing. Generating means for
First correcting means for correcting the amplitude spectrum data by moving the amplitude spectrum distribution represented by the amplitude spectrum data on the frequency axis according to the pitch information for each spectrum distribution region;
Setting means for setting a desired peak frequency on one side of the peak frequency corresponding to at least one local peak in the amplitude spectrum distribution represented by the amplitude spectrum data related to the correction by the first correction means;
Spectral envelopes corresponding to the peak frequencies respectively corresponding to a plurality of local peaks in the amplitude spectrum distribution represented by the amplitude spectrum data related to the correction by the first correction means and the peak frequencies related to the setting by the setting means Indicating means for indicating an envelope value to form
A spectrum distribution region having a spectrum distribution region having a spectral intensity of a local peak closest to the envelope value instructed by the instruction unit corresponding to the peak frequency related to the setting is indicated by the amplitude spectrum data related to generation by the generation unit Selecting means for selecting from among spectrum distribution regions having a peak frequency in a predetermined approximate relationship with the peak frequency related to the setting in
A first copy means for copying the amplitude spectrum data and phase spectrum data of the spectrum distribution region related to the selection by the selection means from the amplitude spectrum data and the phase spectrum data related to the generation by the generation means;
The amplitude spectrum data related to the copy is corrected by moving the amplitude spectrum distribution on the frequency axis according to the setting in the amplitude spectrum distribution represented by the amplitude spectrum data related to the copy by the first copying means. Second correcting means for
In the amplitude spectrum distribution represented by the amplitude spectrum data related to the correction by the first correction means, the spectrum intensity of the local peak for each amplitude spectrum distribution is associated with the peak frequency corresponding to the local peak by the indicating means. The spectral intensity of each spectral bin is corrected to match the indicated envelope value, and the spectral intensity of the local peak in the amplitude spectral distribution represented by the amplitude spectral data related to the correction by the second correcting means is indicated by the indicating means. A third correcting means for correcting the spectral intensity of each spectral bin so as to match the envelope value indicated corresponding to the peak frequency according to the setting;
The phase spectrum distribution represented by the phase spectrum data related to the generation by the generation means is corrected for each spectrum distribution region corresponding to the pitch change by the first correction means, and is copied by the first copy means. Fourth correcting means for correcting the phase spectrum distribution represented by the phase spectrum data according to the frequency change in the second correcting means;
A pitch provided with conversion means for converting amplitude spectrum data related to the correction by the first to third correction means and phase spectrum data related to the correction by the fourth correction means into a sound signal in the time domain. Conversion device. A program used in a pitch conversion device including a computer, the computer being
Input means for inputting pitch information indicating a pitch different from the original sound;
Spectral distribution including a local peak and a spectrum before and after each local peak among a plurality of local peaks of spectral intensity based on an amplitude spectrum obtained by subjecting the sound waveform of the original sound to frequency analysis processing Generates amplitude spectrum data representing the amplitude spectrum distribution in the region with respect to the frequency axis, and generates phase spectrum data representing the phase spectrum distribution with respect to the frequency axis for each spectrum distribution region based on the phase spectrum obtained by the frequency analysis processing. Generating means for
First correcting means for correcting the amplitude spectrum data by moving the amplitude spectrum distribution represented by the amplitude spectrum data on the frequency axis according to the pitch information for each spectrum distribution region;
Setting means for setting a desired peak frequency on one side of the peak frequency corresponding to at least one local peak in the amplitude spectrum distribution represented by the amplitude spectrum data related to the correction by the first correction means;
Spectral envelopes corresponding to the peak frequencies respectively corresponding to a plurality of local peaks in the amplitude spectrum distribution represented by the amplitude spectrum data related to the correction by the first correction means and the peak frequencies related to the setting by the setting means Indicating means for indicating an envelope value to form
A spectrum distribution region having a spectrum distribution region having a spectral intensity of a local peak closest to the envelope value instructed by the instruction unit corresponding to the peak frequency related to the setting is indicated by the amplitude spectrum data related to generation by the generation unit Selecting means for selecting from among spectrum distribution regions having a peak frequency in a predetermined approximate relationship with the peak frequency related to the setting in
A first copy means for copying the amplitude spectrum data and phase spectrum data of the spectrum distribution region related to the selection by the selection means from the amplitude spectrum data and the phase spectrum data related to the generation by the generation means;
The amplitude spectrum data related to the copy is corrected by moving the amplitude spectrum distribution on the frequency axis according to the setting in the amplitude spectrum distribution represented by the amplitude spectrum data related to the copy by the first copying means. Second correcting means for
In the amplitude spectrum distribution represented by the amplitude spectrum data related to the correction by the first correction means, the spectrum intensity of the local peak for each amplitude spectrum distribution is associated with the peak frequency corresponding to the local peak by the indicating means. The spectral intensity of each spectral bin is corrected to match the indicated envelope value, and the spectral intensity of the local peak in the amplitude spectral distribution represented by the amplitude spectral data related to the correction by the second correcting means is indicated by the indicating means. A third correcting means for correcting the spectral intensity of each spectral bin so as to match the envelope value indicated corresponding to the peak frequency according to the setting;
The phase spectrum distribution represented by the phase spectrum data related to the generation by the generation means is corrected for each spectrum distribution region corresponding to the pitch change by the first correction means, and is copied by the first copy means. Fourth correcting means for correcting the phase spectrum distribution represented by the phase spectrum data according to the frequency change in the second correcting means;
A program for functioning as a conversion means for converting amplitude spectrum data related to correction by the first to third correction means and phase spectrum data related to correction by the fourth correction means into sound signals in the time domain. Calculating means for setting a plurality of candidate values of the time shift amount from the peak phase of the fundamental tone with respect to the phase spectrum data, and calculating the peak phase of the fundamental tone and the nth harmonic for each candidate value;
A group of peak phases corresponding to a candidate value that is closest to the flatness is selected from among a plurality of groups of peak phases respectively corresponding to the plurality of candidate values, and a fundamental tone and an nth harmonic in the selected group are selected. Fifth correcting means for correcting the peak phase of the fundamental tone and the n-th overtone in the phase spectrum data so as to coincide with the peak phase of
Instead of the fourth correction means, each phase corresponds to the pitch change in the first correction means for each spectrum distribution region in the phase spectrum distribution represented by the phase spectrum data related to the correction in the fifth correction means. And a sixth correction means for correcting
With respect to the phase spectrum data related to the correction by the sixth correction means, the time shift amount to the peak phase of the fundamental tone before the pitch change is calculated in consideration of the pitch change amount by the first correction means and Seventh correcting means for correcting the peak phase of the fundamental tone and the n-th overtone in the phase spectrum data related to the correction by the sixth correcting means according to the amount of time shift;
In the spectrum distribution region corresponding to the fundamental tone in the phase spectrum data related to the modification by the seventh modifying means, the peak phase other than the fundamental peak phase corresponds to the amount of change in the fundamental peak phase by the fifth and seventh modifying means. In the spectral distribution region corresponding to the nth harmonic, the phase other than the peak phase of the nth harmonic is corrected corresponding to the amount of change in the peak phase of the nth harmonic by the fifth and seventh correction means. Correction means,
The conversion means converts the amplitude spectrum data related to the correction by the first to third correction means and the phase spectrum data related to the correction by the fifth to eighth correction means into sound signals in the time domain. The pitch conversion device according to claim 1. The computer,
Calculating means for setting a plurality of candidate values of the time shift amount from the peak phase of the fundamental tone with respect to the phase spectrum data, and calculating the peak phase of the fundamental tone and the nth harmonic for each candidate value;
A group of peak phases corresponding to a candidate value that is closest to the flatness is selected from among a plurality of groups of peak phases respectively corresponding to the plurality of candidate values, and a fundamental tone and an nth harmonic in the selected group are selected. Fifth correcting means for correcting the peak phase of the fundamental tone and the n-th overtone in the phase spectrum data so as to coincide with the peak phase of
Instead of the fourth correction means, each phase corresponds to the pitch change in the first correction means for each spectrum distribution region in the phase spectrum distribution represented by the phase spectrum data related to the correction in the fifth correction means. And a sixth correction means for correcting
With respect to the phase spectrum data related to the correction by the sixth correction means, the time shift amount to the peak phase of the fundamental tone before the pitch change is calculated in consideration of the pitch change amount by the first correction means and Seventh correcting means for correcting the peak phase of the fundamental tone and the n-th overtone in the phase spectrum data related to the correction by the sixth correcting means according to the amount of time shift;
In the spectrum distribution region corresponding to the fundamental tone in the phase spectrum data related to the modification by the seventh modifying means, the peak phase other than the fundamental peak phase corresponds to the amount of change in the fundamental peak phase by the fifth and seventh modifying means. In the spectral distribution region corresponding to the nth harmonic, the phase other than the peak phase of the nth harmonic is corrected corresponding to the amount of change in the peak phase of the nth harmonic by the fifth and seventh correction means. Function as a correction means,
The conversion means converts the amplitude spectrum data related to the correction by the first to third correction means and the phase spectrum data related to the correction by the fifth to eighth correction means into sound signals in the time domain. The program according to claim 2. At least one amplitude spectrum of the amplitude spectrum distribution represented by the amplitude spectrum data related to the correction by the first correction means, which is a noise component area within the spectrum distribution area indicated by the amplitude spectrum data generated by the generation means. A second copy means for copying a spectrum bin from a noise component region whose frequency band coincides with a spectrum lack region generated on one side of the distribution;
A fifth correction of amplitude spectrum data representing the at least one amplitude spectrum distribution among the amplitude spectrum data related to the correction so as to add a spectrum bin related to the copy by the second copy means to the spectrum absence region. Correction means,
The conversion means converts the amplitude spectrum data related to the correction by the first to third and fifth correction means and the phase spectrum data related to the correction by the fourth correction means into a sound signal in the time domain. The pitch conversion device according to claim 1, wherein the pitch conversion device converts the pitch. The computer,
At least one amplitude spectrum of the amplitude spectrum distribution represented by the amplitude spectrum data related to the correction by the first correction means, which is a noise component area within the spectrum distribution area indicated by the amplitude spectrum data generated by the generation means. A second copy means for copying a spectrum bin from a noise component region whose frequency band coincides with a spectrum lack region generated on one side of the distribution;
A fifth correction of amplitude spectrum data representing the at least one amplitude spectrum distribution among the amplitude spectrum data related to the correction so as to add a spectrum bin related to the copy by the second copy means to the spectrum absence region. Function as a correction means,
The conversion means converts the amplitude spectrum data related to the correction by the first to third and fifth correction means and the phase spectrum data related to the correction by the fourth correction means into a sound signal in the time domain. The program according to claim 2, which is to be converted.

Images