JP2005084661A - Speech analysis generator and program - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To provide a technique for realizing the operation of a Formant position with a lighter load. <P>SOLUTION: The speech data inputted from an A/D converter 8 is extracted in a frame unit, is subjected to fast Fourie transform and is decomposed to a frequency amplitude component and a phase component. The ratio of the frequency components before and after filtering by a moving average filter section 304 is calculated as a frequency amplitude residual and the residual is multiplied by the frequency amplitude scheme obtained by filtering of the frequency amplitude component after a shift, by which the frequency amplitude component is calculated. The pitch is moved by shifting the instantaneous frequency calculated from the phase component. Reverse fast Fourie transform is performed by using the phase component obtained from the frequency amplitude component calculated in such a manner and the instantaneous frequency after the shift, by which the speech data obtained by operating the Formant position and further the pitch is generated. <P>COPYRIGHT: (C)2005,JPO&NCIPI

Description

  The present invention relates to a technique for analyzing a speech waveform and generating (synthesizing) a speech waveform from the analysis result.

A speech analysis / synthesis apparatus that analyzes a speech waveform and synthesizes a speech waveform using the analysis result is also used for applying an acoustic effect such as changing the voice quality of the input speech waveform.
The change in the voice quality is, for example, by manipulating a formant of voice (for example, human voice) or by passing the voice through a bandpass filter (BPF) and specifying an amplitude value for each band, and then changing the voice to a filter configured from the specified amplitude value. It is done by passing. As a speech analysis and generation apparatus that employs the former method, there is one described in Patent Document 1, for example.

  In the conventional speech analysis and generation apparatus described in Patent Document 1, an LPC (Linear Prediction) coefficient extracted by analyzing a speech waveform is converted into an LSP (Line Spectrum Pare) coefficient, and frequency conversion is performed on the LSP coefficient. The formant position was moved by doing. However, in the conversion from the LPC coefficient to the LSP coefficient and the calculation of the filter pole by LPC, the amount of calculation becomes enormous because it is necessary to solve the high-order algebraic equation. For this reason, there was a problem that processing was very heavy.

Due to this problem, it was necessary to prepare a high-performance and expensive processing system in order to bring the processing time to a practical level (this is particularly required when analyzing and generating (synthesizing) in real time. ) For this reason, it has been desired to further reduce the processing load from the viewpoint of reducing the overall manufacturing cost.
JP 2003-66982 A

  The subject of this invention is providing the technique for implement | achieving operation of a formant position with a lighter load.

Both of the speech analysis / generation apparatuses according to the first to third aspects of the present invention are based on the premise that the first speech waveform is analyzed and the second speech waveform is generated using the analysis result. Means.
According to a first aspect of the present invention, there is provided a speech analysis / generation apparatus that analyzes a first speech waveform to extract a first frequency amplitude component and a phase component, a filter unit that filters a frequency amplitude component, a first An instruction means for instructing the amount of formant shift with respect to the frequency amplitude component, a shift means for shifting according to the shift amount instructed by the instruction means, and a filter means for filtering the first frequency amplitude component. Calculating means for calculating a frequency amplitude residual by dividing the first frequency amplitude component by two frequency amplitude components, filtering by the filter means for the first frequency amplitude component shifted by the shift means, and second The third frequency amplitude component obtained by performing one of the shifts by the shift means for the frequency amplitude component is Multiplying means for multiplying several amplitude residuals, and speech waveform generating means for generating a second speech waveform using a fourth frequency amplitude component and a phase component obtained by performing multiplication by the multiplying means. To do.

  The speech analysis generation device according to the second aspect includes Analyzing the first speech waveform to obtain a first frequency amplitude component; And analyzing means for extracting the first phase component; Filter means for filtering frequency amplitude components; First shift means for performing a formant shift on the frequency amplitude component; Residual calculating means for calculating a frequency amplitude residual obtained by dividing the first frequency amplitude component by a second frequency amplitude component obtained by filtering the first frequency amplitude component by the filter means; , Instantaneous frequency calculating means for calculating an instantaneous frequency from the first phase component; Pitch instruction means for instructing the pitch shift amount; According to the shift amount instructed by the pitch instruction means, Instantaneous frequency, And second shifting means for shifting the frequency amplitude residual; Filtering by the filter means for the first frequency amplitude component shifted by the first shift means; And the third frequency amplitude component obtained by performing one of the shifts by the first shift means with respect to the second frequency amplitude component, Amplitude component calculating means for calculating a fourth frequency amplitude component by multiplying the frequency amplitude residual shifted by the second shift means; Phase component calculation means for calculating a second phase component from the instantaneous frequency shifted by the second shift means; A fourth frequency amplitude component; Voice waveform generation means for generating a second voice waveform using the second phase component; It comprises.

  According to a third aspect of the present invention, there is provided a speech analysis / generation apparatus that analyzes a first speech waveform to extract a first frequency amplitude component and a phase component, a filter unit that filters a frequency amplitude component, and a frequency amplitude Shift means for shifting the formant with respect to the component; pitch instruction means for instructing the pitch; sound source waveform generation means for generating a sound source waveform that simulates the vocal cord sound source at the pitch indicated by the pitch instruction means; and Analyzing means for extracting the frequency amplitude component, filtering by the filter means for the first frequency amplitude component shifted by the shift means to the frequency amplitude component extracted from the sound source waveform by the other analysis means, and the filter means Is obtained by performing one of the shifts by the shift means on the filtered first frequency amplitude component. Amplitude component calculating means for multiplying the second frequency amplitude component to calculate the third frequency amplitude component, and voice waveform generating means for generating the second voice waveform using the third frequency amplitude component and the phase component And.

  In the first to third aspects, the analysis unit performs analysis of the first speech waveform using fast Fourier transform, and the speech waveform generation unit uses the inverse fast Fourier transform to perform the second speech. It is desirable to generate a waveform. The filter means desirably functions as a moving average filter.

It is also desirable that the second speech waveform can be superimposed on the first speech waveform and output. The sound source waveform generating means preferably generates a Rosenberg wave as the sound source waveform.
The program of the 1st-3rd aspect of this invention is equipped with the function for implement | achieving the audio | voice analysis production | generation apparatus of the said 1st-3rd aspect, respectively.

  The present invention analyzes the first speech waveform, extracts the first frequency amplitude component and the phase component, and filters the first frequency amplitude component to obtain the first frequency amplitude component. A frequency amplitude residual is calculated by dividing the frequency amplitude component of the second frequency amplitude component, and a third obtained by performing one of filtering on the shifted first frequency amplitude component and shifting of the second frequency amplitude component A frequency amplitude component is multiplied by the frequency amplitude residual to calculate a fourth frequency amplitude component, and a second speech waveform is generated using the fourth frequency amplitude component and the phase component.

  With the shift to the frequency amplitude component performed until the third frequency amplitude component is obtained, the formant position is also shifted. Since no operation is performed on the phase component, the pitch is substantially maintained. Therefore, the formant position can be operated without changing the pitch. Since it is not necessary to perform processing that requires an enormous amount of calculation such as solving higher-order equations for movement of poles or conversion to LSP coefficients by LPC, the entire processing is compared with the case where such processing is performed. The load of can be greatly reduced. By the reduction, a sufficient processing speed can be obtained without employing an expensive high-performance processing system (including a CPU or DSP).

  The instantaneous frequency is calculated from the phase component, the instantaneous frequency and the frequency amplitude residual are shifted, the shifted frequency amplitude residual is multiplied by the third frequency amplitude component, and the fourth frequency amplitude component is calculated. When the second speech waveform is generated using the fourth frequency amplitude component and the phase component obtained from the shifted instantaneous frequency, the pitch operation (shift) is performed simultaneously with the operation of the formant position. Can do. Even in such a case, it is not necessary to perform a process that requires a huge amount of calculation, so that an increase in the load of the entire process can be suppressed.

  The present invention analyzes a first speech waveform, extracts a first frequency amplitude component and a phase component, generates a sound source waveform that simulates a vocal cord sound source at an instructed pitch, and is extracted from the sound source waveform. The frequency amplitude component is filtered by the shifted first frequency amplitude component or is multiplied by the second frequency amplitude component obtained by shifting the filtered first frequency amplitude component to obtain the third frequency amplitude. A component is calculated, and a second speech waveform is generated using the third frequency amplitude component and phase component. For this reason, the operation (shift) of the pitch can be performed simultaneously with the operation of the formant position. Since it is not necessary to perform a process that requires an enormous amount of calculation, the load on the entire process can be greatly reduced as compared with the case where such a process is performed. Thereby, the same effect as the above-mentioned invention can be obtained.

Hereinafter, embodiments of the present invention will be described in detail with reference to the drawings.
<First embodiment>
FIG. 1 is a configuration diagram of an electronic musical instrument equipped with a speech analysis / generation apparatus according to the present embodiment.
As shown in FIG. 1, the electronic musical instrument includes a CPU 1 that controls the entire musical instrument, a keyboard 2 that includes a plurality of keys, a switch unit 3 that includes various switches, a program executed by the CPU 1, and various control applications. The data is input from a ROM 4 storing data, a work RAM 5 of the CPU 1, a display unit 6 including, for example, a liquid crystal display (LCD) and a plurality of LEDs, and a microphone 7 connected to a terminal (not shown). An A / D converter 8 that performs A / D conversion of an analog audio signal and outputs the audio data, a tone generator 9 that generates waveform data for tone generation in accordance with instructions from the CPU 1, and a generator 9 The D / A converter 10 that performs D / A conversion of the generated waveform data and outputs an analog audio signal, the amplifier 11 that amplifies the audio signal, and the amplifier 11 amplifies A speaker 12 for converting the audio signal after the speech, the slider unit 13 provided with various slider, is configured to include a. In these configurations, the CPU 1, the keyboard 2, the switch unit 3, the ROM 4, the RAM 5, the display unit 6, the A / D converter 8, the musical tone generation unit 9, and the slider unit 13 are connected by a bus.

  As shown in FIG. 2, the slider unit 13 includes a pitch slider 21 for instructing a shift of the pitch of the sound input from the microphone 7 and a formant slider 22 for instructing a shift of the formant position (frequency). And. As a result, the voice analysis / generation apparatus according to the present embodiment is realized as an apparatus capable of adding an acoustic effect for manipulating the pitch or formant position of the voice input by the microphone 7.

The slider portion 13 is provided with a detection circuit for detecting the positions of these knobs in addition to the sliders 21 and 22 shown in FIG. The same applies to the switch unit 3.
The analog audio signal output from the microphone 7 is converted into digital audio data by an A / D converter (ADC) 8. The A / D converter 8 performs AD conversion (sampling) at a sampling frequency of 22,052 Hz, for example. Hereinafter, the voice data obtained by AD conversion will be referred to as “original voice data” or “original waveform data” for convenience, and the voice input to the microphone 7 will be referred to as “original voice”. . Audio input may be performed via a storage medium such as a CD-ROM, DVD, or magneto-optical disk, or may be performed via a communication network such as a LAN or a public network.

FIG. 3 is a functional configuration diagram of the speech analysis generation device according to the present embodiment.
The original audio data output from the A / D converter (ADC) 8 is stored in the input buffer 301, which is an area secured in the RAM 5, for example. The frame extraction / windowing unit 302 cuts out original audio data for one frame from the input buffer 301, and multiplies it by a window function, for example, a Hanning Window.

  A fast Fourier transform (FFT) unit 303 performs FFT on a frame after window function multiplication, and calculates a frequency amplitude component and a phase component. The moving average filter unit 304 performs filtering to output an average value of the frequency amplitude components. The optimum number of filter points is 7 at a sampling frequency of 22,052 Hz. In that case, the moving average filter unit 304 performs filtering in units of seven frequency amplitude components, and a value indicating the outline of the frequency amplitude component is output by such filtering. The value corresponds to the formant component.

  The reciprocal computing unit 306 calculates the reciprocal of the frequency amplitude component after filtering (hereinafter referred to as “frequency amplitude outline”). The multiplier 307 multiplies the inverse of the frequency amplitude component before filtering. The multiplication result is a value representing the ratio of frequency amplitude components before and after filtering. Here, this is called a frequency amplitude residual.

The frequency amplitude residual is shifted (shifted in the frequency domain) by the shift unit 310b of the pitch moving unit 310. The frequency amplitude residual after the shift is sent to the multiplier 312.
The shift unit 305a of the formant moving unit 305 shifts the frequency amplitude component received from the FFT unit 303 (shift on the frequency domain). This operation corresponds to an operation for shifting the pitch of the voice, and the formant position of the original voice is also shifted with the shift. However, since the phase component is not manipulated, it is not strictly a pitch shift. Thereby, substantially only the formant position is operated.

The formant moving unit 305 further performs filtering on the frequency amplitude component after the operation by the moving average filter unit 305b. The outline of the frequency amplitude component obtained thereby is sent to the multiplier 312.
The operation panel 311 indicates the formant position and the shift amount of the pitch according to the positions of the knobs of the sliders 21 and 22 shown in FIG. For example, it is realized by the slider unit 13, the CPU 1, the ROM 4, and the RAM 5. In the present embodiment, these shift amounts are indicated by a ratio that represents the formant position and pitch of the original voice as a reference. Hereinafter, the ratios representing the “formant position and pitch shift amount” will be referred to as “formant shift ratio” and “pitch shift ratio”, respectively.

The frequency amplitude component and the phase component are managed by an index. From this, these shifts are performed by multiplying the index value by the ratio to calculate the shifted index value, and changing the arrangement to the calculated index value.
Multiplier 312 multiplies the frequency amplitude component outline from formant moving section 305 by the shifted frequency amplitude residual, and outputs the multiplication result. The frequency / amplitude component calculated as the multiplication result corresponds to a pitch shift and formant shift designated by the user.

The phase component extracted by the FFT unit 303 is sent to the instantaneous frequency calculation unit 308. The calculation unit 308 calculates the instantaneous frequency by using the phase component and the phase component (phase data) of the previous frame by the phase difference measurement method.
The shift unit 310a of the pitch moving unit 310 shifts the instantaneous frequency according to the pitch shift ratio. The frequency phase difference conversion unit 313 converts the instantaneous frequency after software into a phase difference. The phase difference integrating unit 314 generates a phase component by integrating the phase difference.

  The inverse fast Fourier transform (IFFT) unit 315 performs inverse FFT using the phase component from the phase difference accumulation unit 314 and the frequency amplitude component from the multiplier 312. Since the phase component corresponds to that obtained by performing the operation after that of the original sound, the time-domain sound data generated by the inverse FFT is not only the formant position but also the pitch. The windowing frame addition unit 316 multiplies the audio data by a window function in order to add and superimpose the audio data on the audio data of other frames. The multiplication result is sent to the adder 317.

  The adder 317 receives original audio data obtained by the amplifier 318 amplifying the original audio data after the window function multiplication output from the frame extraction / windowing unit 302. From this, the adder 317 adds the original audio data with the audio data from the windowed frame addition unit 316. The audio data after the addition is obtained by adding the original sound obtained by manipulating the formant position or pitch to the original sound, that is, by adding the harmony effect. It is added to the audio data stored in the output buffer 319 and superimposed. The output buffer 319 is, for example, an area secured in the RAM 5, and the sound data read from the output buffer 319 is output to the D / A converter 10 via the musical tone generator 9, so that the harmony effect is obtained from the speaker 12. The added sound is emitted.

  As described above, the voice analysis / generation apparatus according to the present embodiment performs FFT operation on the original voice data to divide the original voice data into the frequency amplitude component and the phase component, and when the shift by the pitch moving unit 310 is not performed, The formant position is moved by doing only. For this reason, compared with the case where a higher order equation is solved for the movement of the pole by LPC and the conversion to the LSP coefficient, the load of the entire process can be greatly reduced. It was confirmed that the processing time was shortened to 1/4 or less by the reduction. In addition to the formant position, the pitch can be manipulated by manipulating the phase component. For this reason, the sound which changed the original sound variously can be generated. An enormous amount of calculation is not required for the operation of the phase component, but if the operation is not performed, the load of the entire process can be further reduced.

Hereinafter, the operation of the electronic musical instrument for realizing the sound conversion device will be described in detail with reference to various flowcharts shown in FIGS.
FIG. 4 is a flowchart of the entire process. First, the entire process will be described in detail with reference to FIG. Note that the overall processing is realized by the CPU 1 executing a program stored in the ROM 4 and using resources of the electronic musical instrument.

  First, in step 401, an initialization process is executed when the power is turned on. In the subsequent step 402, switch processing for responding to the user's operation on the switches constituting the switch unit 3 is executed. In the switch process, for example, the detection circuit constituting the switch unit 3 detects the state of various switches, receives the detection results, analyzes the detection results, and identifies the type of switch whose state has changed and the change. Done.

In step 403 following step 402, keyboard processing for responding to a user operation on the keyboard 2 is executed. By executing the keyboard process, a musical sound is emitted from the speaker 12 in accordance with a performance operation on the keyboard 2. Step 404 then proceeds.
In step 404, slider processing for responding to operations on the sliders 21 and 22 shown in FIG. 2 is executed. In subsequent step 405, the LCD or LED constituting the display unit 6 is driven to perform other processing for realizing information to be provided to the user. After the execution, the process returns to step 402. Thereby, while the power is on, the processing loop formed in steps 402 to 405 is repeatedly executed.

  FIG. 5 is a flowchart of slider processing executed as step 404 described above. 4 shows a flow of processing performed after analyzing a detection result received from a detection circuit constituting the slider unit 13. Next, the slider process will be described in detail with reference to FIG.

  First, in step 501, it is determined whether or not the knob position of the pitch slider 21 has changed. If the user changes the position of the knob, it is determined by analysis, so the determination is YES, and after setting (updating) the pitch shift ratio to be substituted into the variable PitchRatio in step 502, step 503 is performed. Migrate to Otherwise, the determination is no and the process moves to step 503 next.

  In step 503, it is determined whether or not the position of the knob of the formant slider 22 has changed. When the user changes the position of the knob, the determination is similarly YES, and after setting (updating) the formant shift ratio to be substituted in the variable FormatRatio in step 504, the series of processes is terminated. Otherwise, the determination is no and the series of processing ends here.

  In this way, when the user changes the position of the knob of the slider 21 or 22, the pitch shift ratio or formant shift ratio is updated according to the type of the slider that has changed the position of the knob and the position. . Thereby, the user can designate the pitch shift amount and the shift amount of the formant position. The value of the variable is updated with reference to a relationship between a preset knob position and a ratio to be set, for example.

  FIG. 6 is a flowchart of the musical tone timer interrupt process. This is a process executed by an interrupt signal generated at a sampling period, for example, in order to analyze the original voice data and generate (synthesize) the voice data. For example, in the switch process shown in FIG. 4, when it is determined that a switch for instructing generation of audio data has been operated, the interruption (execution) prohibition is released (interrupt is enabled), and the generation prohibition is instructed. Interrupt is prohibited (interrupt is disabled) when it is determined that the switch for operating the switch has been operated. Next, the timer interrupt process will be described in detail with reference to FIG.

  First, in step 601, the original audio data output from the A / D converter 8 is written to the input buffer 301. In the subsequent step 602, it is determined whether or not it is a frame processing timing. If it is the timing, the determination is yes and the process proceeds to step 603. If not, the determination is no and the series of processes ends here.

The generated audio data is added to the audio data of the already generated frame according to the set overlap factor value. For this reason, the processing timing comes in a cycle determined from the sampling frequency and the value of the overlap factor.
In step 603, original audio data of one frame size is extracted from the input buffer 301 and multiplied by a window function (eg, Hanning window). In the next step 604, FFT is performed on the frame after multiplication to divide it into a frequency signal component and a phase component. In the next step 605, frequency signal component filtering processing is performed, and a frequency amplitude residual which is a ratio of frequency amplitude components before and after the filtering processing (= before filtering processing / after filtering processing) is calculated. Step 606 then proceeds.

  In step 606, a formant shift process is executed for realizing the movement of the formant position in accordance with the formant shift ratio assigned to the variable “FormantRatio”. After the execution, a frequency amplitude outline calculation process for calculating the frequency amplitude outline is performed by performing a filtering process on the frequency amplitude component shifted for the realization (step 607).

  In step 608 following step 607, an instantaneous frequency calculation process is performed to calculate an instantaneous frequency using the phase component obtained in step 604 and the phase component of the previous frame. In step 609 which moves after the execution, a pitch shift process for shifting the calculated instantaneous frequency and the frequency amplitude residual calculated in step 605 is executed. After that, the frequency amplitude calculation process of multiplying the frequency amplitude residual shifted in the pitch shift process of step 609 by the frequency amplitude outline calculated in step 607 is executed in step 610 and then the process proceeds to step 611.

  In step 611, a frequency phase difference conversion process for converting the instantaneous frequency shifted in the pitch shift process in step 609 into a phase difference is executed. In the following step 612, a phase difference integration process is performed in which the phase difference is integrated to calculate a phase component. In the next step 613, inverse FFT is performed using the phase component calculated in step 612 and the frequency amplitude component calculated in step 610. After generating the time domain audio data for one frame (audio data whose formant position or pitch has been shifted) by the inverse FFT, the process proceeds to step 614.

  In step 614, the generated voice data is multiplied by a window function, and the original voice data multiplied by the window function in step 603 is added to the multiplication result. In the next step 615, the added audio data (audio data to which the harmony effect is added) is added to the audio data already stored in the output buffer 319 and superimposed. After that, in step 616, the audio data is read from the output buffer 319 and sent to the musical sound generation unit 9, and then the series of processing ends.

  By executing the musical tone timer interrupt process in this way, audio data in which the formant position and further the pitch are manipulated is generated, and the audio data is added to the original audio data. As a result, a harmony effect is added in such a way that the sound whose formant position and further the pitch are manipulated is simultaneously generated in the original sound.

Hereinafter, subroutine processing executed in the timer interrupt processing will be described in detail with reference to the flowcharts shown in FIGS.
FIG. 7 is a flowchart of the formant shift process executed as step 606 described above. First, the shift process will be described in detail with reference to FIG.

  In the musical tone timer interrupt process shown in FIG. 6, the frequency amplitude component obtained by performing the FFT in step 604 is substituted for each element of the one-dimensional array variable Mag. The index value that specifies the element corresponds to the index value. Therefore, for the shift of the frequency amplitude component, a one-dimensional array variable ShiftMag for substituting the shifted frequency amplitude component is prepared, and the array variable ShiftMag to which the frequency amplitude component substituted for the element of the array variable Mag is to be substituted. This is done by specifying and substituting the elements of.

  First, in step 701, 0 is substituted for variable i. In the subsequent step 702, a value obtained by rounding the multiplication result of the value of the variable i and the value of the variable FormatRatio (= INT (i × FormatRatio)) is substituted for the variable ShiftIdx, and 1 is set to the value of the variable i. A value (= INT ((i + 1) × FormantRatio)) obtained by rounding the multiplication result of the added value and the value of the variable FormatRatio is substituted.

  In step 703, it is determined whether or not the value of the variable ShiftIdx is smaller than half the frame size FFT_SIZE. If the value of the variable ShiftIdx is not smaller than the half value, the determination is no and the process moves to step 706. Otherwise, the determination is yes and the process moves to step 704.

The latter half of the frame size of the frequency amplitude component is a return of the first half. This is why the determination in step 703 is performed.
In step 704, the value of the element Shift [Id] specified by the value of the variable i of the array variable Mag is substituted into the element ShiftMag [ShiftIdx] specified by the value of the variable ShiftIdx of the array variable ShiftMag, and the value of the variable ShiftIdx Is incremented. In the next step 705, it is determined whether or not the value of the variable ShiftIdx is smaller than the value of the variable Next. If the former is smaller than the latter, the determination is yes and the process returns to step 703. Otherwise, the determination is no and the process moves to step 706.

  In step 706, the value of variable i is incremented. In the next step 707, it is determined whether or not the value of the variable i is smaller than half the frame size FFT_SIZE. If the value of the variable i is not smaller than the half value, the determination is no, and the series of processing ends here. Otherwise, the determination is yes and the process returns to step 702 above. Thereby, all the frequency components to be substituted into the elements of the array variable ShiftMag are substituted among the frequency amplitude components substituted into the elements of the array variable Mag.

FIG. 8 is a flowchart of the pitch shift process executed as step 609 in the musical tone timer interrupt process shown in FIG. Next, the shift process will be described in detail with reference to FIG.
In the musical tone timer interrupt process shown in FIG. 6, the frequency amplitude residual calculated in step 605 is substituted for each element of the one-dimensional array variable ResMag, and the instantaneous frequency calculated in step 608 is the element of the one-dimensional array variable Freq. Is assigned to The index values specifying these elements correspond to the index values. Therefore, these shifts are performed in the same manner as the shift of the frequency amplitude component in the formant shift process. The frequency amplitude residual and the instantaneous frequency after the shift are assigned to the element of the one-dimensional array variable ShiftResMag and the element of the one-dimensional array variable ShiftFreq, respectively.

  First, in step 801, 0 is substituted for variable i. In the subsequent step 802, a value obtained by rounding the multiplication result of the value of the variable i and the value of the variable PitchRatio (= INT (i × PitchRatio)) is substituted for the variable ShiftIdx, and 1 is assigned to the value of the variable i. A value (= INT ((i + 1) × PitchRatio)) obtained by rounding the multiplication result of the added value and the value of the variable PitchRatio is substituted.

  In step 803, it is determined whether or not the value of the variable ShiftIdx is smaller than a half value of the frame size FFT_SIZE. If the value of the variable ShiftIdx is not smaller than the half value, the determination is no and the process moves to step 808. Otherwise, the determination is yes and the process moves to step 804.

  In step 804, the value of the element ResMag [i] specified by the value of the variable i of the array variable ResMag is substituted for the element ShiftResMag [ShiftIdx] specified by the value of the variable ShiftIdx of the array variable ShiftResMag. In the subsequent step 805, the element ShiftFreq [ShiftIdx] specified by the value of the variable ShiftIdx of the array variable ShiftFreq is multiplied by the value of the element Freq [i] specified by the value of the variable i of the array variable Freq and the value of the variable PitchRatio. Assign the result. Step 806 proceeds after the substitution.

  In step 806, the value of the variable ShiftIdx is incremented. In the next step 807, it is determined whether or not the value of the variable ShiftIdx is smaller than the value of the variable Next. If the former is smaller than the latter, the determination is yes and the process returns to step 803 above. Otherwise, the determination is no and the process moves to step 808.

  In step 808, the value of variable i is incremented. In the subsequent step 809, it is determined whether or not the value of the variable i is smaller than a half value of the frame size FFT_SIZE. If the value of the variable i is not smaller than the half value, the determination is no, and the series of processing ends here. Otherwise, the determination is yes and the process returns to step 802.

  In the present embodiment, the multiplier 312 multiplies the frequency amplitude arithmetic difference from the shift unit 310b by the frequency amplitude outline from the moving average filter unit 305b. The frequency amplitude outline is a moving average. The frequency amplitude outline obtained by filtering by the filter unit 304 may be shifted. The filtering may not be performed by the moving average filter, but may be performed by another low-pass filter.

The shift of the frequency amplitude component or the like is performed without changing the element value of the array variable by paying attention to the index value, but may be performed by higher-order interpolation such as Neville interpolation or Lagrange interpolation. Although only one sound is superimposed on the original sound, a plurality of sounds may be superimposed by changing the formant position and further the pitch.
<Second embodiment>
In the first embodiment, the frequency amplitude residual is shifted in order to shift the pitch. On the other hand, in the second embodiment, the pitch shift is realized by another method.

  The configuration of the electronic musical instrument equipped with the speech analysis / generation apparatus according to the second embodiment is basically the same as that in the first embodiment. The operation is also largely the same or relatively small. For this reason, the same or different ones that are not different from each other will be described by focusing on the different parts from the first embodiment while using the reference numerals in the description of the first embodiment as they are. I will do it.

  FIG. 9 is a functional configuration diagram of the speech analysis generation device according to the second embodiment. First, the functional configuration and the operation of each unit will be described in detail with reference to FIG. In FIG. 9, the same reference numerals are given to the same components as those in the first embodiment or those that are not different enough to be distinguished.

  In the second embodiment, as shown in FIG. 9, instead of calculating the frequency amplitude residual, a Rosenberg wave simulating a vocal cord sound source waveform having a pitch is generated by the Rosenberg wave generation unit 901. . The generation unit 901 generates a Rosenberg wave at a pitch instructed from the operation panel 311.

  The FFT unit 902 performs FFT on the Rosenberg wave generated by the generation unit 901 and sends a frequency amplitude component to the multiplier 312. Thereby, the multiplier 312 multiplies the frequency amplitude component by the frequency amplitude outline from the moving average filter unit 305b of the formant moving unit 305, and sends the multiplication result to the IFFT unit 315. The IFFT unit 315 performs inverse FFT using the frequency amplitude component that is the multiplication result and the phase component from the FFT unit 303 to generate audio data.

  Rosenberg waves can be generated at various pitches. For this reason, By using the frequency amplitude component obtained by generating the Rosenberg wave and multiplying the frequency amplitude component obtained therefrom by the frequency amplitude outline for the inverse FFT, The pitch can be shifted together with the operation of the formant position. Because it is not necessary to handle heavy load to generate the Rosenberg wave, While avoiding a heavy load from the first embodiment, They can be realized. In this second embodiment, You can also use it like vocoder or pitch correct.

  As for the operation of the electronic musical instrument for realizing the sound conversion apparatus according to the second embodiment, the musical tone timer interrupt process (see FIG. 6) is relatively different from the first embodiment. Therefore, only the timer interrupt process will be described in detail with reference to the flowchart shown in FIG. Here, the description of the process of the step which attached the same code | symbol as 1st Example is abbreviate | omitted fundamentally.

  In the second embodiment, when the frequency amplitude outline is calculated in step 607, the process proceeds to step 1001. In step 1001, a Rosenberg wave is generated at a pitch specified by the user using the pitch slider 21. In subsequent step 1002, FFT is performed on the Rosenberg wave to extract a frequency amplitude component. Thereafter, the process proceeds to step 1003.

  In step 1003, the frequency amplitude component is calculated by multiplying the frequency amplitude component extracted in step 1002 by the frequency amplitude outline calculated in step 607. In the next step 1004, inverse FFT is performed using the frequency amplitude component and the phase component extracted in step 604 to generate audio data. Thereafter, the process proceeds to step 614, and the subsequent processing is similarly executed.

  In the present embodiment (first and second embodiments), the present invention is applied to a speech analysis / generation apparatus mounted on an electronic musical instrument. However, a speech analysis / generation apparatus to which the present invention can be applied is like that. However, the present invention is not limited to a voice analysis / generation device. The present invention can be widely applied regardless of the type or use of the device on which the speech analysis / generation device is mounted.

Both the formant position shift amount and the pitch shift amount are specified by the user, but they may be automatically specified. The user may be able to select the designation method, the means for causing designation, and the like.
The voice analysis / generation apparatus as described above, or a program that realizes a modification thereof may be recorded and distributed on a recording medium such as a CD-ROM, DVD, or magneto-optical disk. Alternatively, part or all of the program may be distributed via a transmission medium used in a public network or the like. In such a case, the user can acquire a program and load it into a data processing device such as a computer, thereby realizing a speech analysis generation device to which the present invention is applied using the data processing device. . Therefore, the recording medium may be accessible by a device that distributes the program.

It is a block diagram of the electronic musical instrument carrying the audio | voice analysis production | generation apparatus by a present Example. It is a figure which shows the slider with which the slider part 13 was provided. It is a functional block diagram of the audio | voice analysis production | generation apparatus by a 1st Example. It is a flowchart of the whole process. It is a flowchart of a slider process. It is a flowchart of a musical tone timer interrupt process. It is a flowchart of a formant shift process. It is a flowchart of a pitch shift process. It is a functional block diagram of the audio | voice analysis production | generation apparatus by a 2nd Example. It is a flowchart of a musical tone timer interrupt process (second embodiment).

Explanation of symbols

1 CPU
3 Switch part 4 ROM
5 RAM
7 Microphone 8 A / D converter 9 Musical sound generator 10 D / A converter 11 Amplifier 12 Speaker 13 Slider

Claims (10)

In a speech analysis generation device that analyzes a first speech waveform and generates a second speech waveform using the analysis result,
Analyzing means for analyzing the first speech waveform and extracting a first frequency amplitude component and a phase component;
Filter means for filtering frequency amplitude components;
Indicating means for instructing a formant shift amount with respect to the first frequency amplitude component;
Shift means for shifting according to the shift amount instructed by the instruction means;
Calculating means for calculating a frequency amplitude residual by dividing the first frequency amplitude component by a second frequency amplitude component obtained by filtering the first frequency amplitude component by the filter means;
A third frequency amplitude component obtained by performing one of filtering by the filter unit on the first frequency amplitude component shifted by the shift unit and shifting by the shift unit on the second frequency amplitude component. Multiplying means for multiplying the frequency amplitude residual;
Voice waveform generation means for generating the second voice waveform using the fourth frequency amplitude component obtained by the multiplication by the multiplication means and the phase component;
A speech analysis generation apparatus comprising: In a speech analysis generation device that analyzes a first speech waveform and generates a second speech waveform using the analysis result,
Analyzing means for analyzing the first speech waveform and extracting a first frequency amplitude component and a first phase component;
Filter means for filtering frequency amplitude components;
First shift means for performing a formant shift on the frequency amplitude component;
Residual calculation for calculating a frequency amplitude residual obtained by dividing the first frequency amplitude component by a second frequency amplitude component obtained by filtering the first frequency amplitude component by the filter means. Means,
Instantaneous frequency calculating means for calculating an instantaneous frequency from the first phase component;
Pitch instruction means for instructing the pitch shift amount;
Second shift means for shifting the instantaneous frequency and frequency amplitude residual according to the shift amount instructed by the pitch instruction means;
First obtained by performing one of filtering by the filter means on the first frequency amplitude component shifted by the first shift means and shifting by the first shift means on the second frequency amplitude component. An amplitude component calculating unit that calculates a fourth frequency amplitude component by multiplying the frequency amplitude component of 3 by the frequency amplitude residual shifted by the second shifting unit;
Phase component calculation means for calculating a second phase component from the instantaneous frequency shifted by the second shift means;
Voice waveform generation means for generating the second voice waveform using the fourth frequency amplitude component and the second phase component;
A speech analysis generation apparatus comprising: In a speech analysis generation device that analyzes a first speech waveform and generates a second speech waveform using the analysis result,
Analyzing means for analyzing the first speech waveform and extracting a first frequency amplitude component and a phase component;
Filter means for filtering frequency amplitude components;
Shift means for performing a formant shift on the frequency amplitude component;
Pitch instruction means for indicating the pitch;
Sound source waveform generating means for generating a sound source waveform that simulates a vocal cord sound source at a pitch indicated by the pitch instruction means;
Other analysis means for analyzing the sound source waveform and extracting frequency amplitude components;
Filtering by the filter means on the first frequency amplitude component shifted by the shift means into the frequency amplitude component extracted from the sound source waveform by the other analysis means, and the first frequency amplitude component filtered by the filter means An amplitude component calculating means for calculating a third frequency amplitude component by multiplying the second frequency amplitude component obtained by performing one of the shifts by the shift means,
Voice waveform generating means for generating the second voice waveform using the third frequency amplitude component and the phase component;
A speech analysis generation apparatus comprising: The analysis means performs analysis of the first speech waveform using a fast Fourier transform,
The speech waveform generation means generates the second speech waveform using an inverse fast Fourier transform.
The speech analysis generation apparatus according to claim 1, 2, or 3. The filter means functions as a moving average filter.
The voice analysis generation apparatus according to claim 1, wherein The second voice waveform can be superimposed on the first voice waveform and output.
The voice analysis generation apparatus according to claim 1, wherein The sound source waveform generating means generates a Rosenberg wave as the sound source waveform;
The speech analysis generation apparatus according to claim 3, wherein A program to be executed by a speech analysis generation apparatus that analyzes a first speech waveform and generates a second speech waveform using the analysis result,
An analysis function for analyzing the first speech waveform and extracting a first frequency amplitude component and a phase component;
A filter function for filtering frequency amplitude components;
An instruction function for instructing a formant shift amount with respect to the first frequency amplitude component;
A shift function for shifting according to the shift amount instructed by the instruction function;
A calculation function for calculating a frequency amplitude residual obtained by dividing the first frequency amplitude component by a second frequency amplitude component obtained by filtering the first frequency amplitude component with the filter function;
A third frequency amplitude component obtained by performing one of filtering by the filter function on the first frequency amplitude component shifted by the shift function and shifting by the shift function on the second frequency amplitude component. A multiplication function for multiplying the frequency amplitude residual;
A voice waveform generation function for generating the second voice waveform using the fourth frequency amplitude component obtained by performing multiplication by the multiplication function and the phase component;
A program to realize A program to be executed by a speech analysis generation apparatus that analyzes a first speech waveform and generates a second speech waveform using the analysis result,
An analysis function for analyzing the first speech waveform and extracting a first frequency amplitude component and a first phase component;
A filter function for filtering frequency amplitude components;
A first shift function for performing a formant shift on the frequency amplitude component;
Residual calculation function for calculating a frequency amplitude residual obtained by dividing the first frequency amplitude component by a second frequency amplitude component obtained by filtering the first frequency amplitude component with the filter function When,
An instantaneous frequency calculation function for calculating an instantaneous frequency from the first phase component;
A pitch instruction function for instructing a pitch shift amount;
A second shift function for shifting the instantaneous frequency and the frequency amplitude residual according to the shift amount instructed by the pitch instruction function;
A first obtained by performing one of filtering by the filter function on the first frequency amplitude component shifted by the first shift function and shifting by the first shift function on the second frequency amplitude component. An amplitude component calculation function for calculating a fourth frequency amplitude component by multiplying the frequency amplitude component of 3 by the frequency amplitude residual shifted by the second shift function;
A phase component calculation function for calculating a second phase component from the instantaneous frequency shifted by the second shift function;
A speech waveform generation function for generating the second speech waveform using the fourth frequency amplitude component and the second phase component;
A program to realize A program to be executed by a speech analysis generation apparatus that analyzes a first speech waveform and generates a second speech waveform using the analysis result,
An analysis function for analyzing the first speech waveform and extracting a first frequency amplitude component and a phase component;
A filter function for filtering frequency amplitude components;
A shift function for shifting the formant with respect to the frequency amplitude component;
A pitch instruction function for instructing the pitch;
A sound source waveform generation function for generating a sound source waveform that simulates a vocal cord sound source at a pitch specified by the pitch instruction function;
Other analysis functions for analyzing the sound source waveform and extracting frequency amplitude components;
Filtering by the filter function on the first frequency amplitude component shifted by the shift function into the frequency amplitude component extracted from the sound source waveform by the other analysis function, and the first frequency amplitude component filtered by the filter function An amplitude component calculation function for calculating a third frequency amplitude component by multiplying the second frequency amplitude component obtained by performing one of the shifts by the shift function for
A voice waveform generation function for generating the second voice waveform using the third frequency amplitude component and the phase component;
A program to realize

Images