JP2005208627A - Speech analysis and synthesis device, and program thereof - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To provide a speech analysis and synthesis device capable of avoiding the synthesis of a speech waveform that gives a feeling of phase shifting. <P>SOLUTION: An FFT part 25 conducts an FFT to master speech data of one frame of which high-frequency component was removed and to which a pitch shift was applied. A time scaling part 26 calculates a frequency amplitude, a phase, a phase difference between the present frame and the frame immediately preceding the present frame, and a phase difference that unwrapped the above phase difference, for each channel of which a frequency component was obtained by the FFT, and detects a channel B as a standard by paying attention to a peak of the frequency amplitude. With the channel B used as a standard, the time scaling part 26 calculates the phase of each channel in the synthesized speech using these calculation results. An IFFT part 27 operates the frequency component according to the phase, conducts an IFFT to the frequency component after the operation, and synthesizes the synthesis speech data of one frame. <P>COPYRIGHT: (C)2005,JPO&NCIPI

Description

  The present invention relates to a technique for analyzing a speech waveform and synthesizing a speech waveform using the analysis result.

  Some speech analysis / synthesis apparatuses that analyze speech waveforms and synthesize speech waveforms using the analysis results perform frequency analysis as speech waveform analysis. In such a speech analysis / synthesis apparatus, the synthesis of a speech waveform is specifically performed as follows. Here, in order to distinguish the synthesized speech waveform from the original speech waveform, the former is represented as “synthesized speech waveform” and the latter as “original speech waveform”.

The process until the synthesized speech waveform is synthesized is roughly divided into three stages: an analysis process, a modification process, and a synthesis process. Hereinafter, processing performed for each process in the conventional speech analysis / synthesis apparatus will be described in detail.
<Analysis process>
The original speech waveform is obtained by sampling speech at a predetermined time interval. Sampling data for a predetermined time interval (sampling number) is cut out from the original speech waveform as a frame, and short-time Fourier transform (STFT) is performed. The STFT extracts a frequency component for each frequency channel having a different frequency. The frequency component consists of a real part and an imaginary part, and the frequency amplitude and phase of each frequency channel are calculated from the frequency component. The STFT cuts out signal data for a short time and performs discrete Fourier transform (DFT). For this reason, DFT is used to include STFT. In the DFT, a fast Fourier transform (FFT) is usually used.

When pitch scaling for shifting the pitch (pitch) is performed, FFT is performed after the pitch is shifted by interpolating or thinning out the cut frame.
<Change process>
Since DFT (FFT) of the original speech waveform is performed in units of frames, synthesis of the synthesized speech waveform is also performed in units of frames. The phase θ ′ _{i, k} of the frequency channel k in the synthesized speech waveform is calculated by the following equation. When only time scaling that changes the sound duration is performed, it is not necessary to change the frequency amplitude of each frequency channel.

θ ′ _{i, k} = θ ′ _{i−1, k} + ρ · ΔΘ _{i, k} (1)
Here, ΔΘ _{i, k} represents the phase difference between the previous frame and the current frame in the frequency channel k in the original speech waveform, and ρ represents a scaling factor indicating the degree of pitch scaling. The subscript i represents a frame. The current frame (current frame) is represented by i, and the immediately preceding frame is represented by i-1. For this reason, equation (1) indicates that the phase θ ′ _{i, k} of the frequency channel k of the current frame in the synthesized speech waveform is the phase at the previous frame time in the synthesized speech waveform (after being converted by the scaling factor ρ). It is calculated by adding a multiplication value of the factor ρ to the phase difference ΔΘ _{i, k} .

The phase difference ΔΘ _{i, k} needs to be developed. In the synthesis of speech waveforms, phase unfolding and folding are important. Therefore, this will be described in detail below. In order to easily distinguish the presence / absence of expansion, the folded phase is represented by lowercase θ, and the expanded phase is represented by uppercase Θ.

The phase θ _{k, t at} an arbitrary time t of an arbitrary channel k can be expressed by the following equation.

As apparent from the equation (2), the phase θ _{k, t} can be obtained by integrating the angular velocity ω _k . On the other hand, the value obtained by arctan when calculating the phase from the frequency component calculated by DFT is limited to between −π and π. That is, it is obtained as a folded phase θ _{k, t} . As a result, a 2nπ term is missing in the phase Θ _{k, t} expressed by the following equation.

Θ _{k, t} = θ _{k, t} + 2nπ (where n = 0, 1, 2,...) (3)
In order to calculate the phase θ ′ _{k, t} by the equation (1), it is necessary to expand the folded phase θ _{k, t} . The expansion is an operation of estimating n in the equation (3), and can be estimated based on the center frequency of the channel k of the DFT.

Δθ _{i, k} = θ _{i, k} −θ _{i−1, k} (4)
In the equation (4), Δθ _{i, k} indicates a phase difference between frames of the folded phase θ _{i, k} in the channel k. The center frequency (angular velocity) Ω _{i, k} of the channel k is Ω _{i, k} = (2π · fs / N) · k when the sampling frequency is fs and the number of DFT points (frame sampling number) is N. (5)
It can ask for. At this frequency Ω _{i, k} , assuming that the time difference from the previous frame is Δt, the phase difference ΔZ _{i, k} is ΔZ _{i, k} = Ω _{i, k} · Δt (6)
Can be calculated. The time difference Δt itself is Δt = N / (fs · OVL) (7)
Is required. OVL in the equation (7) indicates an overlap factor that is a value obtained by dividing the frame size by the hop size (the number of samplings corresponding to the shift between adjacent frames).

In equation (6), phase expansion is performed. For this reason, it can describe with the following formula | equation.
ΔZ _{i, k} = Δζ _{i, k} + 2nπ (8)
If the difference between the phase difference Δθ _{i, k} calculated by the equation (4) and the phase difference Δζ _{i, k in} the equation (8) is δ (= Δθ _{i, k} −Δζ _{i, k} ),
Δθ _{i, k} −Ω _{i, k} · Δt = (Δζ _{i, k} + δ) − (Δζ _{i, k} + 2nπ)
= Δ-2nπ (9)
Can be derived. Therefore, if 2nπ on the right side of equation (9) is deleted and the range is limited to between −π and π, δ can be calculated. The δ is a phase difference actually detected in the original speech waveform (hereinafter referred to as “actual phase difference”).

If the phase difference ΔZ _{i, k} (= Ω _{i, k} · Δt) is added to the actual phase difference δ calculated as described above, the phase difference ΔΘ _{i, k} that has been phase-expanded can be obtained as follows. .
ΔΘ _{i, k} = δ + Ω _{i, k} · Δt = δ + (Δζ _{i, k} + 2nπ) = Δθ _{i, k} + 2nπ
(10)
The time-scaled phase θ ′ _{i, k} is calculated by the equations (1) and (10). However, in the method of phase expansion based on the center frequency of the channel, the actual phase difference δ needs to be | δ | <π. Since the absolute value of the maximum value δ _max is a limit value at which the signal does not transition to the adjacent channel,
| Δ _max | = (2π · fs / N) · (k + 0.5) · Δt− (2π · fs / N) · k · Δt
= (2π · fs / 2N) · (N / (fs · OVL)) = π / OVL
(11)
It becomes.

  The value of overlap factor OVL is in the relationship of OVL> 1 from the relationship of equation (11) and | δ | <π. That is, it is understood that the frames need to be overlapped for phase expansion.

  Further, in DFT, a signal of one channel normally excites a plurality of channels. When the normalized angular frequency ω, the initial phase φ, and the amplitude 1 complex sine wave fn are not windowed (equivalent to the case where a square window is applied as a window function), the DFT is obtained as follows.

  The complex sine wave fn can be expressed as follows.

From the equation (12), it can be seen that all channels having frequencies other than the angular frequency ω = (2π / N) · k are excited. Usually, since some window function is used, the number of channels to be excited and the number thereof change according to the bandwidth of the window function. When a Hanning window is used as the window function, the DFT values are: W ₀ = (1/2) N, W ₁ = − (1/4) N, W ₋₁ = − (1/4) N. 13)
This is folded into each channel. As apparent from the equation (13), even if the angular frequency ω = (2π / N) · k, the three channels are excited at a ratio of the frequency amplitude value of 1: 2: 1. When the angular frequency ω is between two adjacent channels, the four channels are excited at a ratio of the frequency amplitude value of 1: 5: 5: 1.

  In order to correctly develop the phase in all the excited channels, n in the equation (8) must be equal between the excited channels. Because of this restriction, for example, when a Hanning window is applied as the window function, the value of the overlap factor OVL needs to be 4 or more.

In the above analysis process, a frame is cut out in accordance with the overlap factor OVL having such a value, and a window function is applied to the cut out frame, and then FFT is performed. In the changing process, the frequency amplitude of each channel is manipulated as necessary while maintaining the phase of the channel obtained as described above.
<Synthesis process>
In the synthesizing process, the frequency component changed (operated) in the changing process is returned to a signal on time coordinates by inverse FFT (IFFT (Inverse Fast Fourier Transform)), and a synthesized speech waveform for one frame is generated. The speech waveform of one frame is added by overlapping the previous frame with a value that changes the overlap factor OVL according to the value of the factor ρ. Thus, a synthesized speech waveform after pitch scaling and time scaling is generated.

  In a conventional speech analysis and synthesis apparatus that synthesizes a synthesized speech waveform as described above, a speech (synthesized sound) based on the synthesized speech waveform has a phase shift called phasiness or reverbrant with respect to the speech (original sound) of the original speech waveform. There was a problem of giving a feeling. Such a phase shift feeling may adversely affect the sense of hearing because the sense of distance from the sound source in the synthesized sound becomes a sense of distance that is felt far away from that in the original sound. Very undesirable from. It occurs even when the pitch shift is relatively very small. Hereinafter, the cause will be described in detail.

As described above, it is necessary to overlap the frames in order to accurately perform the phase expansion. If an appropriate value is set for the overlap factor OVL for that purpose, the phase expansion can be executed accurately, and the phase θ ′ _{i, k} calculated by the equation (1) can be expressed in the time axis by the second term on the right side. Consistency is always preserved for the top phase. Here, the consistency of the phase on the time axis is called HPC (Horizontal Phase Coherence). On the other hand, the phase relationship between channels, that is, between frequency components is referred to as VPC (Vertical Phase Coherence).

In the conventional speech analysis / synthesis apparatus, the VPC is not stored, so that a phase shift occurs.
The reason why the VPC is not stored is that the first term on the right side of equation (1) cannot hold an accurate value. If the phase expansion factor is n, equation (1) can be modified as follows using equations (4) and (10).

θ ′ _{i, k} = θ ′ _{i−1, k} + ρ (θ _{i, k} −θ _{i−1, k} + 2nπ) (14)
Considering the case where the value of the scaling factor ρ is an integer, the phase expansion term 2nπ on the right side of the equation (14) can be deleted and can be expressed as the following equation.

When the initial phase θ ′ _{0, k} is set to be ρθ _{0, k} , the equation (15) is expressed as θ ′ _{i, k} = ρθ _{i, k} (16)
Thus, the first term on the right side of equation (1) is deleted. For this reason, VPC is also saved together with HPC, and scaling without phase shift can be performed. However, if the scaling factor ρ is a value other than an integer, the first term remains.

The first term on the right side of equation (1) is _obtained by integrating the phase-expanded phase difference conversion values (= ρ · ΔΘ _{i, k} ). In order to keep the integrated value at an accurate value, it is necessary to appropriately cope with the following points.

1) Influence of initial value 2) Transition of frequency component between channels 3) Disappearance / generation of frequency component Regarding point 1), the initial phase θ ′ _{0, k} is set to ρθ _{0, k} as described above. By setting to, the integrated value can be maintained at an accurate value.

Regarding point 2), the channel where the frequency component exists is tracked by a method such as peak picking of the frequency amplitude, and the phase difference is calculated across the channels by detecting the transition of the frequency component to another channel. When implemented, the integrated value can be maintained at an accurate value. When the frequency component (signal) transitions from the channel k to the channel k + 1, the equation (14) is expressed as θ ′ _{i, k + 1} = θ ′ _{i−1, k} + ρ (θ _{i, k + 1} −θ _{i−1). k} + 2nπ) (17)
And can be transformed. The phase expansion factor n is also calculated using the phase Ω _{i, k + 1} . However, if the tracking of the frequency component transition fails, the integration value becomes inaccurate at that time, and the VPC is not stored. In addition, when a frequency component transition occurs between channels, a situation may occur in which there is no channel of the immediately preceding frame corresponding to the transition source channel. In that case, an accurate integral value cannot be obtained due to channel mismatch.

  Regarding point 3), it is considered inevitable for normal voices and musical sounds except for special voices whose waveforms are standing waves. In particular, in the case of noise having no harmonic structure, generation / extinction of frequency components occurs randomly and frequently, so that it is almost impossible to detect it. Therefore, it can be said that avoidance is impossible.

  For this reason, in the conventional speech analysis / synthesis apparatus, it is substantially impossible to save the VPC except when the value of the scaling factor ρ is an integer. For this reason, synthesis of a speech waveform that gives a phase shift feeling cannot be avoided reliably. Accordingly, it has been desired to reliably avoid the synthesis of a speech waveform that gives a phase shift feeling.

In the speech analysis / synthesis apparatus described in Patent Document 1, the phase of the synthesized speech waveform subjected to pitch conversion is controlled by the width of overlapping frames. The control is performed in the synthesis process. The explanation that an accurate value cannot be held in the integrated value (the first term on the right side of equation (1)) is based on the assumption that the control is performed.
Japanese Patent No. 2753716

  An object of the present invention is to provide a speech analysis and synthesis apparatus that can reliably avoid synthesis of speech waveforms that give a phase shift feeling.

  The speech analysis / synthesis apparatus according to the present invention analyzes the first speech waveform, and synthesizes the second speech waveform using the analysis result, and performs frequency analysis on the first speech waveform in frame units. A frequency analysis means for extracting frequency components for each frequency channel, and a phase difference in a predetermined number of frames between the first and second speech waveforms with reference to a predetermined frequency channel among the frequency channels. A phase calculation means for calculating and calculating the phase of the second speech waveform in the current frame with reference to the frequency component extracted by the frequency analysis means for each frequency channel, using the phase difference amount; The frequency component extracted by the analysis unit is converted for each frequency channel according to the phase calculated by the phase component calculation unit, and the second audio waveform is converted into a frame by using the frequency component after the conversion. To anda speech synthesis means for synthesizing at positions.

  In addition to the phase difference amount, the phase calculation means includes the phase difference amount between the first and second speech waveforms generated between a predetermined number of previous frames and the current frame, and the current frame. Preferably, the phase of the second speech waveform in the current frame is calculated for each frequency channel using the phase of the first speech waveform at.

  It is preferable that the predetermined number of frames before is a previous frame, and the predetermined frequency channel is a frequency channel having the lowest frequency among those having frequency components. As for the speech synthesis means, it is desirable that the second speech waveform can be synthesized with an overlap factor different from the overlap factor used in the frequency analysis means. Alternatively and / or it is desirable to be able to synthesize a second speech waveform in which the pitch of the first speech waveform is shifted.

  The speech synthesis means extracts the formant of the first speech waveform from the frequency component extracted for each frequency channel by the frequency analysis means, shifts the extracted formant by an operation on the frequency component, and performs the operation. It is desirable that the second speech waveform can be synthesized using the frequency component after the conversion. It is desirable that the formant is extracted by obtaining a frequency amplitude for each frequency channel from the frequency component and performing a filtering process for the frequency amplitude. The order used in the filtering process is desirably changed as necessary based on the state of the frequency amplitude obtained for each frequency channel. As an operation for the frequency component, it is desirable to perform an operation for making the frequency amplitude smaller than the predetermined value the frequency amplitude obtained from the frequency component. Furthermore, it is desirable that the second speech waveform synthesized by the speech synthesizer can be output together with the first speech waveform.

  The program of the present invention is based on the premise that a speech analysis / synthesis apparatus that analyzes a first speech waveform and synthesizes a second speech waveform using the analysis result is executed on a frame-by-frame basis. Analyzing the frequency with the function of extracting frequency components for each frequency channel and the phase difference amount in a predetermined number of previous frames between the first and second speech waveforms with reference to a predetermined frequency channel in the frequency channel A function of calculating and referring to the frequency component extracted by the function of extracting the phase of the second speech waveform in the current frame for each frequency channel in the form using the phase difference amount, The frequency component extracted by the function for calculating is converted for each frequency channel according to the phase calculated by the function for calculating, and the second audio waveform is converted using the frequency component after the conversion. A function of combining in frame units, to realize.

  In the present invention, the first speech waveform is subjected to frequency analysis in units of frames, frequency components are extracted for each frequency channel, and a phase difference amount in a predetermined number of previous frames between the first and second speech waveforms is expressed as a frequency. Calculated with reference to a predetermined frequency channel in the channel, calculated the phase of the second audio waveform in the current frame for each frequency channel using the phase difference amount, and calculated the extracted frequency component Conversion is performed for each frequency channel according to the phase, and the second audio waveform is synthesized in units of frames using the converted frequency components.

  By constructing a phase difference amount in a predetermined number of previous frames between the first and second audio waveforms with reference to a predetermined frequency channel, the phase of each frequency channel of the second audio waveform is It can be expressed relative to the frequency channel. For this reason, the phase relationship between the frequency channels can always be properly preserved, thereby avoiding the synthesis of the second speech waveform that gives a sense of phase shift. Since the phase difference amount is a predetermined number of previous frames, adverse effects due to errors occurring in any of the frequency channels before that time are avoided or reduced. As a result, it is possible to synthesize the second sound waveform that is always good more reliably.

  The formant of the first speech waveform is extracted from the frequency component extracted for each frequency channel, the operation for shifting the extracted formant is performed on the frequency component, and the frequency component after performing the operation together with the above conversion is obtained. When the second speech waveform is synthesized by using it, the formant in the second speech waveform can be arbitrarily shifted. The shift allows the formant in the first speech waveform to be saved. For this reason, when the formant in the first speech waveform is stored, it is possible to synthesize the second speech waveform that does not give a sense of phase shift and feels natural.

Hereinafter, embodiments of the present invention will be described in detail with reference to the drawings.
<First Embodiment>
FIG. 1 is a configuration diagram of an electronic musical instrument equipped with a speech analysis / synthesis apparatus according to the first embodiment.

  As shown in FIG. 1, the electronic musical instrument includes a CPU 1 that controls the entire musical instrument, a keyboard 2 that includes a plurality of keys, a switch unit 3 that includes various switches, a program executed by the CPU 1, and various control applications. The data is input from a ROM 4 storing data, a work RAM 5 of the CPU 1, a display unit 6 including, for example, a liquid crystal display (LCD) and a plurality of LEDs, and a microphone 7 connected to a terminal (not shown). An A / D converter 8 that performs A / D conversion of an analog audio signal and outputs the audio data, a tone generator 9 that generates waveform data for tone generation in accordance with instructions from the CPU 1, and a generator 9 The D / A converter 10 that performs D / A conversion of the generated waveform data and outputs an analog audio signal, the amplifier 11 that amplifies the audio signal, and the amplifier 11 amplifies The audio signal after being configured with a speaker 12 for converting the speech, the. In these configurations, the CPU 1, keyboard 2, switch unit 3, ROM 4, RAM 5, display unit 6, A / D converter 8, and tone generator 9 are connected by a bus. The switch unit 3 includes, for example, a detection circuit for detecting a change in the state of various switches in addition to the various switches to be operated by the user.

  In the electronic musical instrument having the above-described configuration, the speech analysis / synthesis apparatus according to the present embodiment can impart an acoustic effect to the speech input from the microphone 7 by converting (shifting) the pitch (pitch) to the designated pitch. It is realized as. Voice input may be performed via an external storage device, or via a communication network such as a LAN or a public network.

FIG. 2 is a functional configuration diagram of the speech analysis / synthesis apparatus according to the first embodiment.
A voice waveform with added acoustic effects, that is, a pitch-shifted voice waveform, analyzes the frequency of the original voice waveform, extracts frequency components (spectral components) for each frequency channel, shifts the extracted frequency components, and shifts them. Synthesis (generation) is performed using the later frequency components. For this purpose, the following functional configuration is provided.

  An A / D converter (ADC) 8 shown in FIG. 2 converts an analog audio signal output from the microphone 7 into digital audio data. For example, AD conversion is performed at a sampling frequency of 22,050 Hz and 16 bits. Hereinafter, the voice data obtained by AD conversion will be referred to as “original voice data” or “original waveform data” for convenience, and the voice input to the microphone 7 will be referred to as “original voice”. .

  The input buffer 21 is a buffer that temporarily stores the original audio data output from the A / D converter 8. The frame extraction unit 22 extracts a frame that is audio data of a predetermined size from the original audio data stored in the input buffer 21. The size, that is, the number of audio data is 1024, for example. In order to perform accurate phase expansion, it is necessary to extract the frames by overlapping them. Therefore, the frames are cut out by overlapping with the overlap factor OVL. The factor OVL is set to 4. In this case, the hop size is 256 (1024/256 = 4).

  The frame extracted by the frame extraction unit 22 is output to a low pass filter (LPF) 23. The LPF 23 removes high frequency components in order to prevent the frequency components from exceeding the Nyquist frequency due to pitch shift. The pitch shifter 24 performs interpolation or thinning on the frame input from the LPF 23 according to the value of the pitch scaling factor ρ, and shifts the pitch. A general Lagrangian function, a sinc function, or the like can be used for interpolation and decimation, but in this embodiment, pitch shift (pitch scaling) is performed by Neville interpolation.

  The FFT unit 25 performs FFT on the frame after the pitch shift. The time scaling unit 26 performs time scaling for calculating the phase of the frame of the synthesized speech with respect to the frequency component of each frequency channel obtained by performing the FFT. The IFFT unit 27 returns the frequency component of each frequency channel after the time scaling to the data on the time coordinate by performing IFFT (inverse FFT), and generates and outputs audio data for one frame. The sound data is hereinafter referred to as “synthesized sound data”, and the sound generated by the sound data is referred to as “synthesized sound”. The FFT unit 25, the time scaling unit 26, and the IFFT unit 27 constitute a phase vocoder.

  The output buffer 29 is a buffer for storing synthesized voice data to be emitted from the speaker 12 as voice. The frame addition unit 28 adds the synthesized speech data for one frame input from the IFFT unit 27 so as to overlap the synthesized speech data stored in the output buffer 29. The synthesized speech data stored in the output buffer 29 is output to the D / A converter (DAC) 10 and D / A converted.

  When the value of the scaling factor ρ is 2, that is, when the pitch is doubled, the pitch shifter 24 performs decimation and halves the frame size. Therefore, if the overlap factor OVL value is the same, the size of the synthesized speech waveform stored in the output buffer 29 is about ½ of the size of the original synthesized speech waveform. Therefore, as shown in FIG. 6, the synthesized speech waveform is overlap-added to the immediately preceding frame with a value (in this case, 2) obtained by halving the value of the overlap factor OVL.

  The input buffer 21 and the output buffer 29 are areas secured in the RAM 5, for example. The units 22 to 28 except the A / D converter 8, the D / A converter 10, the input buffer 21, and the output buffer 29 execute the program stored in the ROM 4 by the CPU 1 using, for example, the RAM 5 as a work. It is realized with. Although a detailed description is omitted, the pitch shift amount is instructed by operating the keyboard 2, for example, and the degree of time scaling is instructed by operating a predetermined switch of the switch unit 3.

In the present embodiment, the phase θ ′ of each frequency channel in the synthesized speech is calculated by the following equation.
θ ′ _{i, k} = (ΔΘ _{i, k} / ΔΘ _{i, B} ) (θ ′ _{i−1, B} −θ _{i−1, B} )
+ (Ρ−1) ΔΘ _{i, k} + θ _{i, k} (18)
In the equation (18), B indicated as a subscript indicates a channel having the longest wavelength, that is, the lowest frequency component. The first term on the right side of equation (18) indicates the amount of phase difference (change) between the original speech and the synthesized speech that occurred up to frame i-1 one frame before, with reference to channel B. The second term indicates the phase difference between the original speech and the synthesized speech that occurs during the transition from the immediately preceding frame i-1 to the current frame i (hereinafter referred to as “between the immediately preceding frames” for convenience). ing. From this, equation (18) indicates that the phase θ ′ of each channel in the synthesized speech is obtained by adding these phase difference amounts to the phase θ in the current frame i.

The first term on the right side of equation (18) is a term for storing VPC, and the second term is a term for storing HPC. Here, it demonstrates concretely that they are preserve | saved.
When the phase θ [rad] is divided by the angular velocity ω [rad / sec], the unit is time [sec]. Furthermore, if the speed of sound ν [m / sec] is multiplied, the unit becomes the distance [m]. This distance will be described using the expression of phase (including phase difference).

3 and 4 are diagrams for explaining the VPC. 3 and 4, a waveform A (reference sound) is a waveform having a frequency at which the phase changes by π between times T _{1 and} T _{2 and} between times T _{2 and} T ₃ . Thereby, the corresponding distance between them is ½ (= λ / 2) of the wavelength λ of the waveform A. Waveform B is a waveform having a frequency 1.5 times that of waveform A, and waveform C is a waveform having a frequency twice that of waveform A. These times T _{1 to} T ₃ are given for convenience in order to explain the position on the waveform, phase change, and the like.

In FIG. 3, the phases of the waveforms A to C are indicated by distances with the time T ₂ as a reference point. The phase of the waveform A exists at a position away from the reference point in the positive direction by a distance ψA. Similarly, the waveform B is away from the reference point in the negative direction ψB, and the waveform C is away from the reference point in the positive direction by a distance ψC. It shows that it exists in each position. Their distance is determined from the phase. The phase is folded to obtain arctan. As a result, any distance has a length that does not exceed one wavelength.

  “ΔψBA” and “ΔψCA” in FIG. 3 indicate relative phase distances (relative phase distances) between the wavelength B and the wavelength A and between the wavelength C and the wavelength A, respectively. Accordingly, ΔΨBA = ΨB−ΨA and ΔΨCA = ΨC−ΨA.

  VPC corresponds to properly storing such relative phase distances. Specifically, as shown in FIG. 4, when the distance ΨA of the waveform A moves from the position P0 to the position P1 by the distance ΔP as time passes, the other waveforms B and C follow the distances ΨB and ΨC. This corresponds to storing and maintaining the relative phase distance with respect to the waveform A by moving the distance ΔP in the same moving direction.

  By calculating the phase so that the relative phase distance is preserved, the VPC is preserved. As a result, synthesis of synthesized speech data that gives a phase shift feeling due to phases, reverbrant, etc. can always be reliably avoided.

  When the distance is moved, the phase is obtained by arctan, and therefore the moving range needs to be within one wavelength. That is, when calculating the distance between the phase of the original speech and the phase of the synthesized speech, the phase needs to be folded.

  In FIG. 4, it is assumed that the distance of the waveform A moves by one wavelength λ and moves to the next wavelength section without going around as it is. Since the movement is one wavelength λ, the folded phase of the waveform A is the same as before. The same applies to waveform C, which is a second overtone of waveform A. However, the waveform B, which is 1.5 harmonics, does not have the same phase as before. Expressed in terms of angle, movement for one wavelength λ corresponds to a change in phase of 360 degrees in waveform A and 720 degrees in waveform C, so the folded phase is the same as before movement, but in waveform B, it is 540 degrees. It is not the same because it corresponds to a phase change.

  As described above, since the integral tone multiple of the reference waveform (reference waveform) and other waveforms have different phase distance relationships in different wavelength sections, the distance of movement can be reduced by keeping the moving range within one wavelength. If the movement is not circulated, the relative phase distance cannot always be accurately maintained in waveforms other than the integer overtones of the reference waveform. Accordingly, in order to appropriately preserve the phase distance relationship, it is necessary to cycle the phase shift within one wavelength of the reference waveform. By providing this restriction, the present invention can be applied not only to a waveform having a harmonic structure but also to a general voice in which noise and a plurality of sounds are mixed.

  For the same reason, if there is a waveform having a longer wavelength (lower frequency) than the reference waveform, an appropriate phase distance relationship cannot always be preserved. This is because the distance of the long wavelength waveform may exist in another wavelength section exceeding one wavelength of the reference waveform. For this reason, it is necessary to set the channel as the reference waveform to a channel having the lowest frequency component. The channel B is a channel in which the lowest frequency component exists.

  When the first term on the right side of equation (18) is transformed

It becomes. The parentheses in the equation (19) indicate the phase movement distance in channel B, which is the reference channel, and correspond to the distance ΔP in FIG. In order to save the VPC, the phase may be moved by this distance ΔP in all channels. The calculation of the phase from the distance ΔP can be performed by dividing by the sound velocity ν and then multiplying by the angular velocity ω. The term before the parentheses in the equation (19) is for executing the calculation.

  The first term on the right side of equation (18) is simply the phase change of each channel by multiplying the amount of phase change in channel B (reference waveform) folded in the previous frame by the frequency ratio between channel B and each channel. It may be considered that the amount is calculated. With this term, the VPC up to the immediately preceding frame is stored as described above.

When the second term on the right side of equation (18) is decomposed, from equation (16), (ρ−1) ΔΘ _{i, k} = ρΔΘ _{i, k} −ΔΘ _{i, k} = ΔΘ ′ _{i, k} −ΔΘ _{i, k.} (20)
It can be expressed as. The second term indicates the amount of phase change that occurs between the immediately preceding frames. This section stores the HPC from the previous frame to the current frame. The added value of the term and the first term is the amount of change in phase between the original speech and the synthesized speech up to the current frame. Therefore, if the added value is added to the phase θ of the current frame, the phase θ ′ of the synthesized speech is calculated.

  The calculation of the phase θ ′ by the equation (18) can be performed with reference to values before scaling acquired in the current frame and the immediately preceding frame. For this reason, even if an error in obtaining the phase occurs in any of the channels, the adverse effect of the error on the calculation of the phase θ ′ in the subsequent frames is avoided or reduced. For this reason as well, it is always possible to synthesize better synthesized speech data more reliably.

FIG. 5 is a diagram for explaining the phase relationship between frequency channels in the present embodiment. The phase relationship between the channels in this embodiment for obtaining the phase θ ′ _{i, k} of each channel by the equation (18) is the conventional example in which the phase θ ′ _{i, k} is _obtained by the equation (1) in FIG. The phase relationship between the channels is shown in FIG. 5 (b). The phase relationship between the channels shows the state in the frame after changing from the phase relationship of FIG. 5A, taking the reference waveform and its doubled sound waveform as an example.

In the equation (1), since the phase θ ′ _{i, k} is obtained individually and independently, as shown in FIG. 5B, the distance and direction corresponding to the phase θ′α in the frame of the reference waveform are It does not necessarily match the distance and direction corresponding to the phase θ′β in the double sound wave frame. As a result, the phase shift between the channels is improperly stored according to the phase θ ′ obtained for each channel, and the VPC that is the phase relationship between the channels is not stored.

  On the other hand, in the present embodiment, as shown in FIG. 5C, the phase θ′β in the double sound waveform frame is obtained in accordance with the phase θ′α in the immediately preceding frame of the reference waveform. Thereby, the distance and direction corresponding to those phases are matched. In this way, the phase difference between the original speech and the synthesized speech in the frame is calculated based on the reference waveform, so that the phase θ ′ obtained in each channel stores the appropriate phase relationship, and the VPC is stored. It will be.

  As described above, the voice analysis / synthesis apparatus according to the present embodiment synthesizes synthesized voice data that does not give a sense of phase shift by constantly storing the VPC and HPC, and realizes the sound to be emitted from the speaker 12. Has been. Hereinafter, the operation of the electronic musical instrument that realizes the speech analysis / synthesis apparatus will be described in detail with reference to various flowcharts shown in FIGS. 7 and 8.

  FIG. 7 is a flowchart of the entire process. First, the entire process will be described in detail with reference to FIG. Note that the overall processing is realized by the CPU 1 executing a program stored in the ROM 4 and using resources of the electronic musical instrument.

  First, in step 701, an initialization process is executed when the power is turned on. In the subsequent step 702, a switch process for responding to a user operation on the switches constituting the switch unit 3 is executed. In the switch process, for example, the detection circuit constituting the switch unit 3 detects the state of various switches, receives the detection results, analyzes the detection results, and identifies the type of switch whose state has changed and the change. Done.

  In step 703 following step 702, keyboard processing for responding to user operations on the keyboard 2 is executed. By executing the keyboard process, a musical sound is emitted from the speaker 12 in accordance with a performance operation on the keyboard 2. Step 704 then proceeds.

  In step 704, it is determined whether or not it is a sampling timing at which the original audio data is output from the A / D converter 8. If it is the timing, the determination is YES, and the original sound data is written in the input buffer 21 on the RAM 5 in Step 705, and then the process proceeds to Step 706. Otherwise, the determination is no and the process moves to step 710.

  In step 706, it is determined whether it is frame extraction timing. If the time for sampling the original voice data for the hop size has elapsed since the previous timing, the determination is YES and the process proceeds to step 707. Otherwise, the determination is no and the process moves to step 710.

  In step 707, the original audio data stored in the input buffer 21 is extracted for one frame, and the extracted frame is subjected to LPF processing for removing high frequency components, pitch shift by interpolation or thinning, and FFT sequentially. Do. In the next step 708, a time scaling process for calculating the phase of the frame of the synthesized speech is executed for the frequency component of each channel obtained by FFT. After the execution, the process proceeds to step 709, where IFFT targeting frequency components of each channel subjected to time scaling processing, and synthesized audio data for one frame obtained by the IFFT are stored in the output buffer 29 on the RAM 5. Overlap addition is performed on the synthesized speech data. Step 710 then proceeds.

  The frame extraction unit 22, the LPF 23, the pitch shifter 24, and the FFT unit 25 shown in FIG. 2 are realized by executing the processing in step 707. The time scaling unit 26 is realized by executing the processing in step 708. The IFFT unit 27 and the frame addition unit 28 are realized by executing the processing in step 709.

  In step 710, it is determined whether it is time to output synthesized audio data for one sampling. If it is the timing, the determination is YES, and then the synthesized voice data to be output in step 711 is read from the output buffer 29 and sent to the D / A converter 10 via the tone generator 9 and further in step 712. After executing other processing, the process returns to step 702. Otherwise, the determination is no and the process of step 712 is then executed.

  The synthesized synthesized voice data is sent to the D / A converter 10 via the musical tone generator 9. For this reason, the tone generation unit 9 is equipped with a function for mixing the waveform data of the tone generated by itself with data input from other sources.

  FIG. 8 is a flowchart of the time scaling process executed as step 708 described above. Next, the scaling process will be described in detail with reference to FIG. The frequency component of each frequency channel obtained by performing FFT is passed to the scaling process. As described above, the frequency component is composed of a real part and an imaginary part. The time scaling unit 26 shown in FIG. 2 is realized by executing this scaling process.

First, in step 801, 0 is substituted into a variable k for managing the frequency channel of interest. In the next step 802, the frequency amplitude is calculated from the frequency component of the channel specified by the value of the variable k. In the calculation, when the real part of the frequency component is expressed as real and the imaginary part is expressed as img, the frequency amplitude mag is: mag = (real ² + img ² ) ^1/2 (21)
Is calculated by

In step 803 following step 802, a phase is calculated from the frequency component. When the phase is expressed as phase (= θ), phase = arctan (img / real) (22)
Is calculated by This phase phase is folded. After the calculation, the process proceeds to step 804.

  In step 804, peak detection of the frequency amplitude mag is performed to search for a channel in which a frequency component exists. Since finer peak detection is performed separately, coarse detection is performed here. Specifically, a channel having a frequency amplitude mag larger than the frequency amplitude mag of the four channels before and after is detected as a peak and registered.

  In step 805, which moves to step 804, the phase difference Δθ between the immediately preceding frames (between the current frame and the immediately preceding frame) is calculated by the equation (4). In the following step 806, the phase difference Δθ that has been folded is expanded to obtain the phase difference ΔΘ. The phase expansion is performed according to equation (10). After the phase expansion, the process proceeds to step 807.

  In step 807, the value of the variable k is incremented. In the next step 808, it is determined whether or not the value of the variable k is smaller than the FFT point N. When the frequency amplitude mag and the like are calculated in all frequency channels, the relationship is not satisfied. Therefore, the determination is NO and the process proceeds to step 809. Otherwise, the determination is YES and the above Return to step 802. Accordingly, the processing loop formed in steps 802 to 808 is repeatedly performed until the frequency amplitude mag and the like are calculated for all frequency channels.

  In step 809, peak detection is executed more finely than the processing in step 804, and the peak channel is narrowed down. For example, a channel having the smallest frequency amplitude mag before and after the peak is extracted, and a channel having a frequency amplitude mag of −14 db or less of the frequency amplitude mag of the peak channel is specified as a formal peak. To do. -14db as a reference for the determination is set from the amplitude characteristic of the Hanning window.

The equation (18) is θ ′ _{i, k} = ΔΘ _{i, k} ((θ ′ _{i−1, B} −θ _{i−1, B} ) / ΔΘ _{i, B} + (ρ−1))
+ Θ _{i, k} (23)
And can be transformed.

  All the terms shown as symbols on the right side of equation (23) are prepared when the determination in step 808 is NO. The peak detection in step 809 which shifts to that determination is performed to select channel B. Therefore, in step 810 following step 809, the channel having the lowest frequency among the peaks detected in step 809 is adopted as channel B, and the phase θ ′ of the synthesized speech is obtained for each channel using equation (23). Is calculated.

  The calculation results in steps 803 and 810 are stored at least until the next frame. As a result, all the terms shown as symbols on the right side of equation (23) are prepared when the determination in step 808 is NO.

  In step 709 of FIG. 7 that is shifted to after execution of the time scaling process, IFFT is performed after the frequency component of each frequency channel is operated in accordance with the phase θ ′ obtained in step 810. The operation is performed, for example, by changing the values of the real part real and the imaginary part img without changing the frequency amplitude mag so that the phase obtained from them matches the phase θ ′. By performing such an operation, a synthesized waveform having the phase θ ′ obtained in step 810 is synthesized for each frequency channel.

  In the present embodiment, time scaling is performed together with pitch scaling, but only time scaling may be performed. As the voice, only the synthesized voice based on the synthesized voice data is emitted, but the original voice that is the source may be emitted. Alternatively, they may be emitted together. In such a case, the synthesized voice data is synthesized by shifting the pitch of the original voice, so that the voice added with the harmony effect can be emitted. When a plurality of synthesized sound data having different shift amounts are synthesized, it is possible to emit a sound with a chord constituent sound added. These can be realized, for example, by adding the synthesized voice data stored in the output buffer 29 to the original voice data stored in the input buffer 21 and sending it to the D / A converter 10.

The reference channel B is detected and determined by searching for the lowest frequency channel among the channels extracted as a peak. However, the channel B is determined by using other methods. Also good.
<Second Embodiment>
When pitch shifting is performed by pitch scaling, the position (frequency) of the formant of the synthesized speech moves to a position different from that of the original speech along with the pitch shifting. The movement usually makes the synthesized speech unnatural. Therefore, in the second embodiment, synthesized speech that feels more natural can be generated by performing pitch scaling (pitch shift) while preserving the formant of the original speech.

  The speech analysis / synthesis apparatus according to the second embodiment is mounted on an electronic musical instrument, as in the first embodiment. The configuration of the electronic musical instrument is basically the same as that equipped with the speech analysis / synthesis apparatus according to the first embodiment. The functional configuration of the speech analysis / synthesis apparatus according to the second embodiment is basically the same as that in the first embodiment. For this reason, the same reference numerals as those in the first embodiment are used as they are for the same components as those in the first embodiment, and only different portions from the first embodiment will be described.

FIG. 9 is a functional configuration diagram of the speech analysis / synthesis apparatus according to the second embodiment.
In the second embodiment, the frame from which the high frequency component has been removed by the LPF 23 is input to the FFT unit 25. Thereby, the time scaling unit 26 performs time scaling on the frequency components of each frequency channel obtained by performing the FFT from the frame before shifting the pitch.

  If the value of the pitch scaling factor ρ is a, the frequency is multiplied by a by the pitch shift, and conversely, the audio data (frame size) for one frame is 1 / a times. In the second embodiment, time scaling for multiplying the original audio data for one frame by a is performed so that the audio data (synthesized audio data) for one frame that is finally obtained remains unchanged. This is done before the pitch shift. The frequency component of each frequency channel after time scaling is sent to the formant shift unit 91.

  The formant shift unit 91 preliminarily shifts the formant so as to cancel the shift accompanying the pitch shift in order to compensate the formant shift accompanying the pitch shift. If the value of the pitch scaling factor ρ is a, the formant is shifted by 1 / a. The frequency component of each frequency channel after performing the operation for that is sent to the IFFT unit 27, and is returned to the data on the time coordinate by inverse FFT.

  The audio data thus restored to the data on the time coordinate is different from the original number of data for one frame according to the value of the pitch scaling factor ρ due to the time scaling performed by the time scaling unit 26. . The pitch shifter 24 interpolates or thins out such audio data according to the value of the pitch scaling factor ρ to shift the pitch. For this reason, the audio data for one frame after interpolation or thinning is finally the original, that is, the same as the frame size of the original audio data. The interpolated or thinned audio data is sent to the frame adder 28 as synthesized audio data. In the synthesized voice data, the formant of the original voice data is stored, so that it does not give a sense of phase shift and becomes natural.

FIG. 10 is a flowchart of overall processing in the second embodiment. Next, the entire process will be described in detail with reference to FIG.
In FIG. 10, the same reference numerals are assigned to the processing steps that are basically the same in the processing content as in the first embodiment (FIG. 7). Here, a description will be given in the form of paying attention only to a different part from the first embodiment.

  In the second embodiment, when the determination in step 706 is YES, the process proceeds to step 1001. In step 1001, the original audio data stored in the input buffer 21 is extracted for one frame, and LPF processing for removing high frequency components and FFT are sequentially performed on the extracted frames. After performing the FFT, the process proceeds to step 708, and the time scaling process shown in FIG. 8 is executed. After the execution, the process proceeds to step 1002.

  In step 1002, a formant movement process for moving the formant in order to save the formant of the original voice is executed. In the subsequent step 1003, IFFT targeted for the frequency component of each channel after operation by execution of formant movement processing, interpolation for one frame of audio data obtained by the IFFT, or pitch shift by thinning is performed. The synthesized voice data for one frame obtained by the pitch shift is overlap-added to the synthesized voice data stored in the output buffer 29 on the RAM 5. Thereafter, the process proceeds to step 710.

In the second embodiment, the pitch shifter 24 is realized by executing step 1003. The formant shift unit 91 is realized by executing step 1002.
FIG. 11 is a flowchart of the formant movement process executed as step 1002 above. Next, the movement process will be described in detail with reference to FIG.

First, in step 1101, the tilt component is removed from the frequency amplitude mag (Equation (21)) of each channel.
In general, it is known that the residual signal obtained by removing the influence of the resonance frequency due to formants from an audio signal, that is, the frequency characteristic of the vocal cord sound source signal, has a slope that gradually attenuates as the frequency increases. The inclination is called a tilt component. As a result, the frequency characteristics of the audio signal are such that the characteristics of the resonance frequency due to the tilt component and formants are superimposed. For this reason, when extracting only the formant component, it is necessary to remove the tilt component.

  As described above, the tilt component generally has a slope in which the frequency characteristic of the vocal cord sound source signal is gradually attenuated as the frequency increases. For this reason, the audio data may be passed through a high-pass filter (HPF) having a pass characteristic of about the first order. After performing FFT, it can process as multiplication with respect to the frequency amplitude mag of each channel. As a value to be multiplied by the frequency amplitude mag of each channel, for example, a value assuming a sine wave having a quarter period can be employed.

  When the formant is moved, there is a possibility that the leakage component and noise component from the channel where the frequency component exists are emphasized by the movement of the formant. When such enhancement is performed, a noisy voice or an unnatural voice is synthesized. For this reason, in step 1102 to which the process proceeds after removing the tilt component as described above, the frequency amplitude mag below a certain value is regarded as noise, and an operation for making the amplitude mag smaller is performed.

  In the present embodiment, an operation of further attenuating (reducing) the frequency amplitude mag that is −58 db or less with respect to the maximum value of the frequency amplitude mag is performed by −26 db. Thereby, all frequency amplitudes mag below a certain value are multiplied by 0.05. By performing this operation as pre-processing, it is possible to avoid emphasizing noise and the like even if the formants are shifted, and a good result can be obtained with certainty. The pre-processing is performed in this way, that is, the frequency amplitude mag below a certain value is not all set to 0. If the frequency amplitude mag is all set to 0, the synthesized speech is unnatural. Because it becomes. For this reason, the frequency amplitude mag that should not be emphasized is attenuated so as to cancel the emphasis caused by formants.

  The frequency amplitude mag to be attenuated is determined based on the maximum value, but may be a fixed value. The target range of the frequency amplitude mag may be determined as appropriate. The same applies to the degree (degree) of attenuation of the target frequency amplitude mag.

In step 1103 following step 1102, a formant is extracted from the frequency amplitude mag of each channel subjected to the above preprocessing. The extraction is performed by a moving average filtering process. If the frequency amplitude of the corresponding channel is expressed by subscript, the formants are A and F, respectively, and the order of the moving average filtering process (moving average filter simulated by M) is M, the moving average filtering process for obtaining the formant F _{k of the} channel k is It is expressed as follows.

  By performing the moving average filtering process, a formant outline is extracted for each channel. The reason why the formant is specified by extracting such a rough shape is to further avoid that the frequency amplitude mag that protrudes from others due to noise or the like is extracted as a formant. That is, it is for extracting a formant more appropriately.

  It is necessary to pay attention to the order used in the moving average filter. When the original sound is a high pitch sound, the frequency interval between channels (spectrums) becomes wide. Therefore, the formant outline cannot be appropriately extracted at a low order M. The influence of the original spectrum is strongly reflected in the extracted outline. Therefore, it is necessary to set the order M as high as necessary.

  Conversely, when the original sound is a low pitch sound, the frequency interval between the channels (spectrums) is narrow and dense. In this case, when the degree M is high, the formant outline is crushed, and the outline cannot be extracted properly. Therefore, the order M needs to be lowered to such an extent that the outline is not crushed.

  As the original voice, those having various pitches are inputted by the microphone 7. For this reason, in the present embodiment, the order M is set (updated) as needed, assuming that an appropriate order M for the original voice needs to be set as necessary. Specifically, it is determined based on the peak state of the frequency amplitude mag detected by executing the time scaling process in the immediately preceding step 708. More specifically, when the index of the base channel determined in step 810 is k, an order M obtained by the following equation that has obtained a good result by experiment is set. However, M = 32 is set when M> 32, and M = 8 is set when M <8. “INT” in the formula indicates that the calculation result in parentheses is calculated as an integer.

M = INT (k + 3) (25)
The calculation (setting) of the order M by the equation (25) is performed before the moving average filtering process. As a result, the moving average filtering process is always performed with an appropriate order M corresponding to the pitch of the original speech. Since the moving average filtering process is performed with such order M, the formants can always be appropriately extracted. The order M may be set according to the number of peaks of the frequency amplitude mag, that is, the order M is set lower as the number increases, and the order M is set higher as the number decreases.

  The process moves to step 1104 after the formant (rough shape) is extracted by the moving average filtering process. In step 1104, the frequency amplitude mag of each channel is divided by the extracted formant. The division result corresponds to the frequency domain representation of the residual component in the linear prediction analysis.

  In step 1105 following step 1104, Neville interpolation processing for moving the extracted formants is executed. After the execution, the process proceeds to step 1106, and the shifted formant component is multiplied by the residual component of each channel. The series of processing is finished thereafter.

  By the multiplication, the frequency component of each channel after the operation for moving the formants is obtained. The moved formant is returned to the original position by the pitch shift in step 1003. As a result, the formants are preserved.

FIG. 12 is a flowchart of Neville interpolation processing executed as step 1105 described above. Next, the interpolation process will be described in detail with reference to FIG.
The frequency amplitude (formant component) of each formant channel extracted in step 1103 is assigned to the array variables y and x together with the frequency corresponding to the channel, for example, and stored. The number of formant components (for example, 4) used in the interpolation process is substituted into the variable N. The frequency (channel) to which the formant should be moved is calculated for each formant component from the original frequency and the value of the pitch scaling factor ρ. The formant component at the calculated frequency is calculated with reference to the values assigned to the N elements of the array variables y and x around the calculated frequency. The Neville interpolation process shown in FIG. 12 is an excerpt of the process for obtaining the formant component at the frequency to be moved, focusing on one formant component. A value indicating the frequency to be moved is assigned to the variable t.

  First, in step 1201, 0 is substituted into the variable s1. In the next step 1202, the value of the element y [s1] specified by the value of the variable s1 of the array variable y is substituted for the element w [s1] specified by the value of the variable s1 of the array variable w, and the variable s2 Is substituted with a value obtained by subtracting 1 from the value of the variable s1. Thereafter, the process proceeds to step 1203, and it is determined whether or not the value of the variable s2 is 0 or more. If the value is less than 0, the determination is no and the process moves to step 1206. Otherwise, the determination is yes and the process moves to step 1204.

  In step 1204, a value calculated by the following equation is substituted into element w [s2]. After the substitution, the value of variable s2 is decremented in step 1205, and then the process returns to step 1203.

w [s2] = w [s2 + 1] + (w [s2 + 1] −w [s2]) ×
(Tx [s1]) / (x [s1] -x [s2]) (26)
On the other hand, in step 1206 where the determination in step 1203 is NO and the process proceeds, the value of the variable s1 is incremented. In the next step 1207, it is determined whether or not the value of the variable s1 is less than the value of the variable N. If the value of the variable s1 is less than the value of the variable N, the determination is yes and the process returns to step 1202. Otherwise, the determination is no and the series of processing ends here.

  In this way, the value of the element y [s1] is substituted into the element w [s1] and the value substituted is updated while sequentially incrementing the value of the variable s1. As a result, the formant component at the time of the variable t is finally substituted into the element w [0]. In step 1105, a variable t that matches the frequency corresponding to the channel after the formant movement is obtained, and the above-described series of processing is executed using N formant components around the variable (frequency) t. Then, the value of the variable (frequency) t is sequentially changed in correspondence with each channel, and each time a series of processes as described above are executed, all formant components at the frequency to be moved are obtained. Yes.

  The formant component obtained at the frequency to be moved is basically obtained by performing interpolation or thinning on the extracted formant. Since the formant component does not need to be determined so precisely, linear interpolation may be employed. Instead of Neville interpolation, another interpolation method such as Lagrange interpolation or Newton interpolation may be employed.

  In the second embodiment, pitch shift is performed after time scaling, but they may be performed in the reverse order. However, in this case, since the waveform of the original speech is deformed before performing time scaling, the influence of the deformation extends to the peak detection of the frequency amplitude mag. For this reason, in order to better preserve formants, it is desirable to perform the pitch shift after time scaling.

  The formant movement (shift) is performed in order to preserve the formant even when the pitch is shifted. However, for example, the formant may be moved independently of the pitch shift for the purpose of changing the sound quality. The pitch-shifted synthesized voice may be emitted together with the original voice.

  The speech analysis / synthesis apparatus as described above, or a program that realizes a modification example thereof may be recorded and distributed on a recording medium such as a CD-ROM, DVD, or magneto-optical disk. Alternatively, part or all of the program may be distributed via a transmission medium used in a public network or the like. In such a case, the user can obtain a program and load it into a data processing apparatus such as a computer, thereby realizing a speech analysis / synthesis apparatus to which the present invention is applied using the data processing apparatus. . Therefore, the recording medium may be accessible by a device that distributes the program.

It is a block diagram of the electronic musical instrument carrying the audio | voice analysis synthesis apparatus by 1st Embodiment. 1 is a functional configuration diagram of a speech analysis / synthesis device according to a first embodiment. FIG. It is a figure explaining the phase relationship between frequency components (the 1). It is a figure explaining the phase relationship between frequency components (the 2). It is a figure explaining the phase relationship between the frequency channels in this Embodiment. It is a figure explaining the overlap addition performed with respect to a synthetic speech waveform. It is a flowchart of the whole process. It is a flowchart of a time scaling process. It is a functional block diagram of the speech analysis and synthesis apparatus according to the second embodiment. It is a flowchart of the whole process (2nd Embodiment). It is a flowchart of a formant movement process. It is a flowchart of Neville interpolation processing.

Explanation of symbols

1 CPU
3 Switch part 4 ROM
5 RAM
7 Microphone 8 A / D converter 9 Musical sound generator 10 D / A converter 11 Amplifier 12 Speaker

Claims (11)

In a speech analysis and synthesis device that analyzes a first speech waveform and synthesizes a second speech waveform using the analysis result,
Frequency analysis means for performing frequency analysis of the first speech waveform in units of frames and extracting frequency components for each frequency channel;
A phase difference amount in a predetermined number of previous frames between the first and second audio waveforms is calculated with reference to a predetermined frequency channel in the frequency channel, and the current phase difference amount is used in the form of using the phase difference amount. Phase calculating means for calculating the phase of the second speech waveform in the frame of the frequency with reference to the frequency component extracted by the frequency analyzing means for each frequency channel;
The frequency component extracted by the frequency analyzing unit is converted for each frequency channel according to the phase calculated by the phase component calculating unit, and the second audio waveform is synthesized in units of frames using the converted frequency component. Speech synthesis means to
A speech analysis / synthesis apparatus comprising: In addition to the phase difference amount, the phase calculation means includes the phase difference amount between the first and second speech waveforms generated between the predetermined number of previous frames and the current frame, and the current Calculating the phase of the second speech waveform in the current frame for each frequency channel using the phase of the first speech waveform in a frame;
The speech analysis / synthesis apparatus according to claim 1. The predetermined number of previous frames is the immediately preceding frame, and the predetermined frequency channel is a frequency channel having the lowest frequency among those having the frequency component.
The speech analysis / synthesis apparatus according to claim 1 or 2, The speech synthesizer can synthesize the second speech waveform with an overlap factor different from the overlap factor used in the frequency analyzer.
The speech analysis / synthesis apparatus according to claim 1, 2, or 3. The voice synthesizer can synthesize the second voice waveform obtained by shifting the pitch of the first voice waveform.
The speech analysis / synthesis apparatus according to any one of claims 1 to 4, wherein The speech synthesizer extracts a formant of the first speech waveform from the frequency component extracted for each frequency channel by the frequency analysis unit, shifts the extracted formant by an operation on the frequency component, The second speech waveform can be synthesized using the frequency component after performing in combination with the conversion,
The speech analysis / synthesis apparatus according to claim 1. The speech synthesis means obtains a frequency amplitude for each frequency channel from the frequency component, and extracts a formant of the first speech waveform by a filtering process targeting the frequency amplitude.
The speech analysis / synthesis apparatus according to claim 6. The speech synthesizer changes the order used in the filtering process as necessary based on the state of the frequency amplitude obtained for each frequency channel.
The speech analysis / synthesis apparatus according to claim 7. The speech synthesizer performs an operation for making the frequency amplitude smaller than a predetermined value as a value for the frequency component obtained from the frequency component as an operation for the frequency component,
The speech analysis / synthesis apparatus according to claim 6 or 7, The second speech waveform synthesized by the speech synthesis means can be output together with the first speech waveform,
The speech analysis / synthesis apparatus according to any one of claims 1 to 9. A program to be executed by a speech analysis / synthesis apparatus that analyzes a first speech waveform and synthesizes a second speech waveform using the analysis result,
A function of analyzing the frequency of the first speech waveform in units of frames and extracting a frequency component for each frequency channel;
A phase difference amount in a predetermined number of previous frames between the first and second audio waveforms is calculated with reference to a predetermined frequency channel in the frequency channel, and the current phase difference amount is used in the form of using the phase difference amount. A function of calculating the phase of the second speech waveform in the frame of the reference with reference to the frequency component extracted by the function of extracting for each frequency channel;
The frequency component extracted by the extracting function is converted for each frequency channel according to the phase calculated by the calculating function, and the second audio waveform is synthesized in units of frames using the converted frequency component. Function and
A program to realize

Images