JP4433734B2 - Speech analysis / synthesis apparatus, speech analysis apparatus, and program - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To provide a technique which makes it possible to synthesize always appropriate speech waveforms by using parameters extracted by analyzing original speech waveforms. <P>SOLUTION: In an analysis phase, the speech data outputted from an A/D converter 8 is analyzed in a frame unit. As the parameters, a PARCOR coefficient, a voiced speech ratio indicating the degree at which the speech represented by the speech data is voiced speech, etc., are extracted. In a synthesis phase, a driving sound source wave is generated in the form of mixing a Rosenberg wave generated at an assigned pitch and a white noise wave not having the pitch according to a voiced speech ratio. A synthesizing filter section 45 generates the speech data for one frame component by using the driving sound source wave and the PARCOR coefficient. <P>COPYRIGHT: (C)2005,JPO&amp;NCIPI

Description

  The present invention relates to a technique for analyzing a speech waveform and synthesizing a speech waveform using the analysis result.

A speech analysis / synthesis apparatus that analyzes a speech waveform and synthesizes a speech waveform using the analysis result is also used for applying an acoustic effect such as changing the voice quality of the input speech waveform.
The change in the voice quality is, for example, by manipulating a formant of voice (for example, human voice) or by passing the voice through a bandpass filter (BPF) and specifying an amplitude value for each band, and then changing the voice to a filter configured from the specified amplitude value. It is done by passing. The latter method is mainly employed in a vocoder (speech analysis / synthesis device) that produces a human voice at a pitch (pitch) designated by a keyboard or the like. As a speech analysis / synthesis apparatus adopting the latter method, there is one described in Patent Document 1, for example.

  In the conventional speech analysis and synthesis apparatus described in Patent Document 1, an input speech waveform is analyzed to extract parameters, a driving sound source waveform corresponding to a designated pitch is generated, and the extracted parameters and driving sound source waveform are generated. Was used to synthesize a speech waveform for output. As the driving sound source waveform, a pulse waveform such as a triangular wave or a waveform having a different pulse width was generated. However, when such a pulse waveform is used, natural human speech cannot be reproduced. Further, it is determined whether the input voice is voiced sound, and either the driving sound source waveform or the white noise waveform for unvoiced sound is selected according to the determination result and used for the synthesis of the voice waveform. When switching, the synthesized speech waveform may change suddenly and become unnatural. Even if they are simply crossfade according to the judgment result, it does not always follow the changes in the original speech waveform appropriately, so it is definitely possible to avoid the synthesized speech waveform from becoming unnatural. Can not.

Sounds that seem unnatural usually give people a sense of incongruity. This is even more so if the voice is from someone who is used to listening. From this, it can be considered that it is very important to make the synthesized speech appropriate so that it always feels natural.
The extraction of the parameters may be performed for speech encoding or the like. In the encoding, it is desired to reproduce the original information more. From this, it can be considered that it is very important to make the synthesized speech always feel natural from the viewpoint of making the original information more reproducible.
JP 09-204185 A

  An object of the present invention is to provide a technique that allows an appropriate speech waveform to be always synthesized using parameters extracted by analyzing an original speech waveform.

Vocoding system of the present invention is to analyze the first speech waveform, assuming to perform the synthesis of the second speech waveform using the analysis result, by analyzing the first speech waveform frame by frame A first analyzing means for extracting a parameter; and a first speech waveform is analyzed in units of frames to calculate an autocorrelation value of a frequency amplitude value of the first speech waveform in each frame; Second analysis means for calculating the standard deviation from the correlation value, and extracting a voiced sound ratio that increases as the standard deviation value decreases, a pitch specifying means for specifying the pitch, and a vocal cord sound source A sound source waveform generating means for generating a sound source waveform simulating a waveform at a pitch specified by the pitch specifying means, another sound source waveform generating means for generating another sound source waveform having no pitch, and extracted The voiced sound ratio value is Mixed with other sound sources waveform as the mixing ratio of the sound source waveform is increased by generating an excitation waveform with increasing listening, speech waveform synthesis to synthesize a second speech waveform using the excitation waveform, and the parameters Means.

  The parameter obtained from the first speech waveform and the voiced sound ratio are stored in a storage unit capable of storing a plurality of data groups including the parameter and the voiced sound ratio, and the speech waveform synthesis unit is stored in the storage unit. It is desirable to synthesize the second speech waveform using one of the stored data groups.

The first analysis means extracts the frequency amplitude value and phase information of the first speech waveform in each frame, calculates an autocorrelation value of the frequency amplitude value, and sets the autocorrelation value to the maximum. It is desirable that the pitch of the first speech waveform is extracted as the parameter from the frequency amplitude value and the phase information.

In addition, the first analysis means continuously repeats a predetermined number of times when a pitch change between a pitch extracted in the current frame and a pitch extracted in a frame before the current frame is equal to or greater than a predetermined value. When continuing, it is preferable that the pitch extracted in the previous frame is adopted as the parameter, and otherwise, the pitch extracted in the current frame is adopted as the parameter .

The speech analysis apparatus of the present invention is based on the assumption that parameters are extracted from a speech waveform, a speech waveform acquisition unit that acquires a speech waveform, and a speech waveform acquired by the speech waveform acquisition unit is analyzed in units of frames. First analysis means for extracting a filter coefficient used for a synthesis filter for waveform synthesis as a parameter, and a speech waveform acquired by the speech waveform acquisition means are analyzed in units of frames, and the first speech waveform in each frame is analyzed. Calculates the autocorrelation value of the frequency amplitude value, calculates its standard deviation from the autocorrelation value of each frame, and inputs the voiced sound ratio that increases as the standard deviation value decreases to the synthesis filter And second analysis means for extracting as a parameter for generating a sound source waveform.

The program of the present invention has a function for realizing a speech analysis / synthesis device and a speech analysis device .

The present invention analyzes the first speech waveform and extracts a voiced sound ratio indicating the degree to which the speech represented by the first speech waveform is voiced sound in addition to parameters such as filter coefficients.
Voiced and unvoiced sounds are not clearly distinguished, and there are cases where the voice is in an intermediate state. Therefore, by mixing the voiced sound source waveform with the unvoiced sound source waveform based on the voiced sound ratio, a more appropriate driving sound source waveform can be generated in reproducing the intermediate state. As a result, more appropriate sound data can be synthesized using the drive sound source waveform, and the sound generated by the sound data can be made more natural.

Hereinafter, embodiments of the present invention will be described in detail with reference to the drawings.
<First embodiment>
FIG. 1 is a block diagram of an electronic musical instrument equipped with a speech analysis / synthesis apparatus according to this embodiment.
As shown in FIG. 1, the electronic musical instrument includes a CPU 1 that controls the entire musical instrument, a keyboard 2 that includes a plurality of keys, a switch unit 3 that includes various switches, a program executed by the CPU 1, and various control applications. The data is input from a ROM 4 storing data, a work RAM 5 of the CPU 1, a display unit 6 including, for example, a liquid crystal display (LCD) and a plurality of LEDs, and a microphone 7 connected to a terminal (not shown). An A / D converter 8 that performs A / D conversion of an analog audio signal and outputs the audio data, a tone generator 9 that generates waveform data for tone generation in accordance with instructions from the CPU 1, and a generator 9 The D / A converter 10 that performs D / A conversion of the generated waveform data and outputs an analog audio signal, the amplifier 11 that amplifies the audio signal, and the amplifier 11 amplifies A speaker 12 for converting the audio signal after the speech, an external storage device 13 to access the example removable storage medium, and is configured with a. In these configurations, the CPU 1, keyboard 2, switch unit 3, ROM 4, RAM 5, display unit 6, A / D converter 8, musical sound generation unit 9, and external storage device 13 are connected by a bus. The external storage device 13 is, for example, a flexible disk device, a CD-ROM device, or a magneto-optical disk device. The switch unit 3 includes, for example, a detection circuit for detecting state changes of various switches in addition to various switches that are operated by the user.

  The CPU 1 realizes automatic performance by processing sequence data (for example, standard MIDI file (SMF)) for performance reproduction stored in the ROM 4 or a storage medium accessible by the external storage device 13. Therefore, as shown in FIG. 2, the switches constituting the switch unit 3 include a start (START) switch 21 for instructing the start of the automatic performance and a stop (STOP) switch 22 for instructing the end thereof. A tempo (TEMPO) switch 23 is provided for instructing change of the tempo value during automatic performance. As the tempo switch 23, an up (UP) switch 23a for instructing an increase in the tempo value and a down (DOWN) switch 23b for instructing the down of the tempo value are provided. In addition, although not particularly illustrated, an analysis switch for instructing analysis of the voice input from the microphone 7, a synthesis switch for performing synthesis of voice using the analysis result, and the keyboard 2 at the time of synthesis There are provided a pitch selection switch for selecting whether or not the pitch specification by the operation is valid, and a file selection switch for selecting one of the stored analysis results to be used for synthesis. ing.

  In the electronic musical instrument having the above-described configuration, the speech analysis / synthesis apparatus according to the present embodiment can impart an acoustic effect for converting the pitch (pitch) to the designated pitch (pitch) with respect to the voice input from the microphone 7. It is realized as. Audio input may be performed via the external storage device 13 or via a communication network such as a LAN or a public network.

FIG. 3 is a functional configuration diagram of the speech analysis / synthesis apparatus according to the present embodiment.
The speech waveform to which the acoustic effect is added is generated by analyzing the original speech waveform and extracting parameters, and using the extracted parameters and the generated sound source waveform. From this, as shown in FIG. 3, the functional configuration of the speech analysis / synthesis apparatus is for the stage of analysis (analysis phase) and for the stage of synthesis of speech waveforms using the analysis results (synthesis phase). And those of The one for the analysis phase is equivalent to one for performing speech encoding or the like, that is, one constituting a speech analysis device, and the one for the synthesis phase is one for performing speech decoding, that is, a speech synthesis device. It corresponds to what constitutes.

First, the configuration for the analysis phase and the operation of each unit will be described.
An A / D converter (ADC) 8 shown in FIG. 3 converts an analog audio signal output from the microphone 7 into digital audio data. For example, AD conversion is performed at a sampling frequency of 22,052 Hz and 16 bits. Hereinafter, the voice data obtained by AD conversion will be referred to as “original voice data” or “original waveform data” for convenience, and the voice input to the microphone 7 will be referred to as “original voice”. .

  The frame extraction unit 31 inputs the original voice data and extracts the frame by cutting out the frame with a predetermined size. The size, that is, the number of audio data is 512, for example, and the frame is cut out by overlapping with an overlap factor that is a value obtained by dividing the frame size by the hop size, as shown in FIG. In the example shown in FIG. 6, the factor value is 8, that is, the hop size is 64 (512/64 = 8). The frames thus extracted (voice data corresponding to the frame size) are sent to the level calculation unit 32, the linear prediction analysis unit 33, and the pitch extraction / voiced sound ratio calculation unit 34.

The level calculation unit 32 calculates a level value (level parameter) indicating a volume level for each transmitted frame. The calculation is performed, for example, by calculating the sum of squares of the original audio data constituting the frame and dividing the sum of squares by the frame size.
The linear prediction analysis unit 33 performs linear prediction analysis on the sent frame and calculates a Percoll coefficient. The order of linear predictive analysis (LPC) is suitably 26th or 28th for speech data with a sampling frequency of 22,050 Hz. In this embodiment, the order is 26th. Thereby, 26 PARCOR coefficients are calculated from the transmitted frame using the Levinson-Durbin algorithm.

The pitch extraction / voiced sound ratio calculation unit 34 calculates the pitch frequency (repetition period) of the original voice and the voiced sound ratio indicating the degree to which the original voice is voiced.
FIG. 4 is a functional configuration diagram of the calculation unit 34. With reference to FIG. 4, the functional configuration of the calculation unit 34 and the operation of each unit will be described in detail.

  The frame sent from the frame extraction unit 31 is input to the high pass filter (HPF) unit 51. The HPF unit 51 outputs the frame from which the low frequency component has been removed to the windowing unit 52. The windowing unit 52 multiplies a frame (audio data for one frame size) by a window function, for example, a Hanning Window.

The fast Fourier transform (FFT) unit 53 performs FFT on the frame after the window function multiplication. The frequency amplitude phase calculation unit 54 obtains a frequency amplitude value and a phase from the real value and the imaginary value calculated thereby. The frequency amplitude autocorrelation calculation unit 55 calculates autocorrelation for the obtained frequency amplitude value and calculates a correlation value. If the calculated frequency amplitude value and x _i, the correlation value r _n is calculated by the following equation. As is apparent from the equation, since the value of the second half of the frame is an index of 256 or more, the first half of the index is folded, so only the total 256 frequency amplitude values x _{i from} 0 to 255 are calculated. Just do it.

By taking the autocorrelation of the frequency amplitude value x _i, the correlation value r _n increases the harmonic structure or voice or a musical tone with harmonics. For this reason, accurate pitch extraction can be performed without the influence of noise or the like. When a human voice is assumed as the original voice, only a range that can be considered as a pitch of the voice needs to be targeted, and therefore, only a correlation value r _{n with} an index value corresponding to 40 to 500 Hz is 1 ≦ n ≦ 13. If you ask for.

The value of _n that maximizes the correlation value r _n is the index value of the frequency amplitude value x _i corresponding to the pitch frequency. However, in reality, high accuracy is often not obtained even if the frequency is calculated from the index value. Therefore, in order to extract the pitch frequency of the original voice with high accuracy, an accurate pitch frequency is calculated by the phase difference measurement method. The calculation is performed by a frequency calculation unit (hereinafter abbreviated as “frequency calculation unit”) 57 using a phase difference measurement method. The calculation unit 57 calculates the pitch frequency with reference to the phase data 56 of the previous frame.

In the pitch extraction method performed by taking the autocorrelation of the frequency amplitude value x _i , the pitch can be extracted with high accuracy if it is a normal spoken word. However, it is not always possible to extract the pitch with a high degree of accuracy when singing or specially pronounced. For this reason, the pitch correction unit 58 corrects the pitch frequency calculated by the frequency calculation unit 57. The corrected pitch frequency is output from the pitch extraction / voiced sound ratio calculation unit 34.

Standard deviation calculating section of the autocorrelation value (hereinafter abbreviated as "standard deviation calculation unit") 59 receives the correlation value r _n from the frequency amplitude autocorrelation calculating unit 55, and calculates the standard deviation. The standard deviation is a value indicating the degree of dispersion (scattering), as is well known. For this reason, as the standard deviation is smaller, the power is concentrated on the specific frequency and the possibility of being a voiced sound increases. On the other hand, the larger it is, the higher the possibility that the sound is unvoiced because the power is dispersed on the average in various frequencies. The standard deviation calculation unit 59 pays attention to this, and calculates a voiced sound ratio mix (0 ≦ mix ≦ 1) indicating the degree to which the original voice is voiced sound from the calculated standard deviation. The voiced sound ratio mix is output from the pitch extraction / voiced sound ratio calculation unit 34.

  In this way, the level calculation unit 32, the linear prediction analysis unit 33, and the pitch extraction / voiced sound ratio calculation unit 34, as shown in FIG. 5, respectively, are a level value, a PARCOR coefficient, a voiced sound ratio mix, and a pitch frequency. Is calculated and output for each frame. These are temporarily stored in the parameter buffer 35 as parameters indicating the analysis results, and are saved as a parameter file 36 as necessary. Note that what stores the parameter buffer 35 and the parameter file 36 is, for example, a storage medium accessible by the RAM 5 or the external storage device 13. Here, for the sake of convenience, the following description will be made on the assumption that the RAM 5 is used.

  The configuration for the analysis phase, that is, the units 31 to 34 that analyze the original voice data output from the A / D converter 8 are realized by, for example, the CPU 1, the switch unit 3, the ROM 4, the RAM 5, and the external storage device 13. . The components 41 to 47 for synthesizing a speech waveform using the configuration for the synthesis phase, that is, the parameters stored in the parameter buffer 35 and outputting the synthesized speech waveform to the D / A converter 10 are, for example, CPU 1, keyboard 2, switch unit 3, This is realized by the ROM 4, the RAM 5, the tone generation unit 9, and the external storage device 13.

Next, the composition for the synthesis phase and the operation of each unit will be described.
In the synthesis of the speech waveform, the parameter buffer 35 reads and stores the parameter file 36 selected by the user.
The changeover switch 41 outputs one of the pitch frequency stored as a parameter in the parameter buffer 35 and the pitch frequency specified by the user operating the keyboard 2 to the Rosenberg wave generation unit 42. The switch 41 selects and outputs one of them according to the operation of the pitch selection switch.

  The Rosenberg wave generation unit 42 generates a Rosenberg wave as shown in FIG. 7 at a pitch input from the changeover switch 41. The Rosenberg wave is a waveform simulating a vocal cord sound source waveform, and an OQ (Open Quotient) parameter for controlling the opening period length of the vocal cord is fixed to 0.5 in this embodiment. The amplitude parameter AV is set to 1. The white noise generation unit 43 generates a white noise wave. As is well known, the noise wave has no pitch unlike the Rosenberg wave.

  The Rosenberg wave generated by the Rosenberg wave generation unit 42 is multiplied by the voiced sound ratio mix stored in the parameter buffer 35 and stored in the driving sound source buffer 44. The white noise wave generated by the white noise generation unit 43 is multiplied by a value (= 1−mix) obtained by subtracting the voiced sound ratio mix from 1 and added to the Rosenberg wave stored in the driving sound source buffer 44. Thereby, a driving sound source waveform obtained by mixing these waveforms in accordance with the voiced sound ratio mix is prepared in the driving sound source buffer 44.

  The Rosenberg wave is a driving sound source waveform for voiced sound, and the white noise wave is a driving sound source waveform for unvoiced sound. However, voiced sound and unvoiced sound are not clearly distinguished, and there are cases where the sound is in an intermediate state between the two. From this, the voiced sound ratio mix is calculated, and the Rosenberg wave and the white noise wave are mixed according to the ratio mix to generate a more appropriate driving sound source waveform for reproducing the intermediate state. Yes. By generating such a driving sound source waveform, it is possible to synthesize a voice waveform that is always felt natural.

  The synthesis filter unit 45 generates a sound waveform (sound data) for one frame from the driving sound source waveform for one frame prepared in the driving sound source buffer 44 and the PARCOR coefficient stored in the parameter buffer 35. The driving sound source waveform is used as an input, and the PARCOR coefficient is used as a filter coefficient. On the other hand, the sum of squares of the entire generated audio data is calculated and divided by the frame size (here, 512), and the level value of the parameter buffer 35 (the value calculated in the same manner with the original audio data) is calculated by the result of the division. Division is performed, and the square root is obtained as a multiplication coefficient and output to the multiplier 46.

  The multiplier 46 multiplies the audio data generated by the synthesis filter unit 45 by a multiplication coefficient. As shown in FIG. 8, the frame adder 47 multiplies the audio data for one frame output from the multiplier 46 by the Hanning window, and uses the audio data obtained thereby for the same overlap factor as in the analysis phase. Overlapping (overlap addition) with other frames. The audio data after such superposition is output to the D / A converter (DAC) 10.

  The frame addition unit 47 is realized by the CPU 1, the ROM 4, and the RAM 5, for example. Superimposition with other frames is performed using an output buffer which is an area provided in the RAM 5, for example. Therefore, the tone generator 9 generates waveform data for generating a tone to be generated in accordance with an instruction from the CPU 1 and outputs it to the D / A converter 10 as well as being read out from the output buffer and sent. The voice data can be output to the converter 10 as it is.

  The speech analysis and synthesis apparatus according to the present embodiment is realized as a system that analyzes original speech data as described above, synthesizes speech data using the analysis result, and emits sound from the speaker 12. Hereinafter, the operation of the electronic musical instrument that realizes the speech analysis / synthesis apparatus will be described in detail with reference to various flowcharts shown in FIGS.

FIG. 9 is a flowchart of the entire process. First, the entire process will be described in detail with reference to FIG. Note that the overall processing is realized by the CPU 1 executing a program stored in the ROM 4 and using resources of the electronic musical instrument.
First, in step 901, an initialization process is executed when the power is turned on. In the subsequent step 902, switch processing for responding to the user's operation on the switches constituting the switch unit 3 is executed. In the switch process, for example, the detection circuit constituting the switch unit 3 detects the state of various switches, receives the detection results, analyzes the detection results, and identifies the type of switch whose state has changed and the change. Done.

In step 903 following step 902, a keyboard process for responding to a user operation on the keyboard 2 is executed. By executing the keyboard process, a musical sound is emitted from the speaker 12 in accordance with a performance operation on the keyboard 2. Step 904 then proceeds.
In step 904, a display process for providing information to be provided to the user by driving the LCD or LED constituting the display unit 6 is executed. After the execution, the process returns to step 902. Thereby, while the power is on, the processing loop formed in steps 902 to 904 is repeatedly executed.

Next, subroutine processing executed in the overall processing will be described in detail.
FIG. 10 is a flowchart of the switch process executed as step 902. 3 shows a flow of processing performed after analyzing a detection result received from a detection circuit constituting the switch unit 3. In the subroutine process executed in the overall process, the switch process will be described in detail with reference to FIG.

  First, in step 1001, it is determined whether or not the analysis switch is turned on (operated). If the switch is operated by the user, the determination is YES, and in step 1002, the value of the variable ana_flg is inverted, that is, 0 if the previous value is 1, and 1 if the previous value is 0. Then, the process proceeds to step 1003. Otherwise, the determination is no and the process moves to step 1006. A value of 1 assigned to this indicates that the original sound input from the microphone 7 is being analyzed.

  In step 1003, it is determined whether or not the value of the variable ana_flg is 1. If the user operates the analysis switch to instruct the analysis of the original voice, 1 is substituted for it, so the determination is YES, the parameter buffer 35 is cleared in step 1004, and the parameter stored at the head of it is saved. After substituting a variable param_buf_adr for a value for designating a region for use (referred to herein as an “address” for convenience), the process proceeds to step 1006. Otherwise, the determination is no, the analysis result stored in the buffer 35 in step 1005 is written and stored in the RAM 5 as the parameter file 36, and the process proceeds to step 1006. In this way, in this embodiment, the original voice is analyzed from when the analysis switch is operated until it is operated again, and the analysis results are sequentially stored in the buffer 35.

  In step 1006, it is determined whether or not the synthesis switch has been turned on (operated). If the switch is operated by the user, the determination is yes, the value of the variable syn_flg is inverted in step 1007, and then the process proceeds to step 1008. Otherwise, the determination is no and the process moves to step 1010.

  In step 1008, it is determined whether or not the value of the variable syn_flg is 1. If 1 is assigned to the variable, the determination is yes, and the parameter file 36 specified by the value of the variable file_num is read into the parameter buffer 35 in step 1009, and then the process proceeds to step 1010. Otherwise, the determination is no and the process moves to step 1010. In this way, in the present embodiment, the parameter file 36 used for synthesis is switched by operating the synthesis switch.

  In step 1010, it is determined whether or not the pitch selection switch is turned on (operated). If the switch is operated by the user, the determination is yes, the value of the variable pitch_sel is inverted in step 1011, and then the process proceeds to step 1012. Otherwise, the determination is no and the process moves directly to step 1012.

  In step 1012, it is determined whether the file selection switch is turned on. If the switch is operated by the user, the determination is YES, the value of the variable file_num is cyclically changed (updated) in step 1013, and other switch processing is performed in step 1014 to correspond to the operation of other switches. After execution, the series of processing is terminated. Otherwise, the determination is no and the process in step 1014 is executed next, and then the series of processes is terminated.

  As described above, the cyclic update of the value of the variable file_num is performed according to, for example, the number of existing parameter files 36 and the file number. Accordingly, the user can arbitrarily select any of the existing parameter files 36 by operating the file selection switch. The parameter file 36 that can be selected in this way is read into the parameter buffer 35 by an operation on the synthesis switch.

FIG. 11 is a flowchart of the keyboard process executed as step 903 in the overall process shown in FIG. Next, the keyboard process will be described in detail with reference to FIG.
First, in step 1101, for example, event data (for example, MIDI data) representing the operation content performed by the user on the keyboard 2 is received, and the operation content is determined. When the event data representing the operation content is not received, the keyboard 2 determines that the state has not changed, and ends a series of processes. If the event data indicates that the key has been pressed, it is determined that the process proceeds to step 1102. If the event data indicates that the key has been released, that is determined. The process proceeds to step 1107.

  In step 1102, it is determined whether or not the value of the variable syn_flg is 1. If the value is not 1, the determination is NO, and processing for sending a command (for example, MIDI data) for instructing the tone generation of the pitch assigned to the pressed key to the tone generator 9 is performed in step 1103. After executing the above, a series of processing ends. Otherwise, the determination is yes and the process moves to step 1104.

  In step 1104, the output buffer constituting the frame adder 47 is cleared, and a value for designating an address (here, an area for storing one audio data) located at the head is substituted into a variable out_buf_adr. A value designating an address (here, an area for storing parameter data) located at the head of the parameter buffer 35 is substituted into a variable param_buf_adr. In the subsequent step 1105, the frame synthesis process for synthesizing the audio data for one frame is executed as described above, and in the subsequent step 1106, 1 is set as a variable indicating that sound is being generated by the synthesized audio data. Substitute into note_on. The series of processing is finished thereafter.

  On the other hand, in step 1107, where it is determined that the event data received from the keyboard 2 in step 1101 indicates that the key has been released, it is determined whether the value of the variable syn_flg is 1. If the value is not 1, the determination is NO, and processing for sending a command (for example, MIDI data) for instructing to mute the musical tone of the pitch assigned to the released key to the musical tone generating unit 9 is performed in step 1108. After executing the above, a series of processing ends. If not, the determination is yes and the process proceeds to step 1109, where the synthesized sound data is read from the output buffer to end the sound generation, and the variable note_on is substituted with a value of 0 indicating that the sound is not sounding. After that, a series of processing ends.

  As described above, when the value of the variable syn_flg is 1, the voice data is synthesized using the parameters stored in the parameter buffer 35, and the voice (musical sound) is generated by the voice data. As a result, between the time when the user operates the synthesis switch and the time when the user operates the synthesis switch again, the synthesized voice is operated according to the operation content to the keyboard 2, that is, the key pressed (pitch), and the period during which the key is pressed. To be pronounced.

FIG. 12 is a flowchart of the frame synthesizing process executed as step 1105. Next, the synthesis process will be described in detail with reference to FIG.
First, in step 1201, it is determined whether or not the value of the variable pitch_sel is zero. If the user operates the pitch selection switch to enable the operation on the keyboard 2 to specify the pitch (pitch) of the voice to be synthesized, 0 is assigned to the variable. The determination is YES, and after setting the pitch assigned to the key pressed as the pitch of the driving sound source waveform in step 1202, the process proceeds to step 1204. Otherwise, the determination is no, and in step 1203, the pitch stored in the parameter buffer 35 in the form of a parameter is set as the pitch, and the process proceeds to step 1204.

  In step 1204, the parameter is read from the address specified by the value of the variable param_buf_adr in the parameter buffer 35, the voice data is synthesized, and the voice data for one frame obtained by the synthesis is added to the other frames already generated. Thus, the newly synthesized audio data for one frame is added to the audio data stored in the output buffer. Specifically, a Rosenberg wave having a pitch set in step 1202 or 1203 and a white noise wave are generated, and the generated waveform is mixed according to the voiced sound ratio mix, thereby generating a driving sound source waveform. Other frames already generated from the address specified by the value of the variable out_buf_adr after the voice data is synthesized from the generated driving sound source waveform and the PARCOR coefficient, the level of the synthesized voice data is adjusted, and the Hanning window is further multiplied. Is added to and superposed, and after the superposition, a value for designating an address at which writing of the next frame is to be started is substituted into the variable out_buf_adr and updated. The series of processing is finished thereafter.

Thus, in this embodiment, when synthesizing voice data, either the pitch of the key pressed on the keyboard 2 or the pitch of the original voice can be selected by operating the pitch selection switch. I am letting.
FIG. 13 is a flowchart of the musical tone timer interrupt process. This is a process executed by an interrupt signal generated at, for example, a sampling period in order to analyze original voice data or synthesize voice data. For example, in the switch processing shown in FIG. 10, when 1 is newly assigned to at least one of the variables ana_flg and syn_flg, the interrupt (execution) prohibition is canceled (interrupt is enabled), and both of these values are 0. When it becomes, interrupt is prohibited (interrupt is disabled). Next, the timer interrupt process will be described in detail with reference to FIG.

  First, in step 1301, an analysis process for analyzing the input original voice data is executed. In the following step 1302, it is determined whether or not the value of the variable syn_flg is 1. If 1 is assigned to the variable, the determination is yes and the process proceeds to step 1303. If not, the determination is no and the series of processes ends here.

  In step 1303, it is determined whether the value of the variable note_on is 1. When the musical sound based on the synthesized voice data is to be generated, since 1 is substituted for it, the determination is YES, and in step 1304, the voice data in the output buffer is converted to D / D via the musical tone generator 9. After executing the process of continuing the tone generation by sending it to the A converter 10, the process proceeds to step 1305.

  In step 1305, it is determined whether it is frame synthesis timing. If it is the timing, the determination is YES, and if the parameter of the frame located at the end of the parameter buffer 35 has not been read, the value of the variable param_buf_adr is updated according to the frame from which the parameter is to be read next (step 1306). After the update, the frame synthesis process shown in FIG. Otherwise, the determination is no and the series of processing ends here.

As described above, when the frame size is 512 and the overlap factor is 8, the frame synthesis timing comes every 64th time when the execution interval of the musical tone timer interrupt process is set as the sampling period.
In this way, when audio data is synthesized, the frame from which the parameter is read out from the parameter buffer 35 is changed as time elapses after the synthesis is started. Thereby, a change in the sound quality of the original voice is reflected in the synthesis of the voice data.

Next, the analysis processing executed as step 1301 will be described in detail with reference to the flowchart shown in FIG.
First, in step 1401, it is determined whether or not the value of the variable ana_flg is 1. If 1 is not assigned to it, the determination is no, and the series of processing ends here. Otherwise, the determination is YES, and the area (hereinafter referred to as “input buffer”) obtained by acquiring the original audio data that has been A / D converted and digitized and output from the A / D converter 8 and secured in the RAM 5, for example. (Step 1402), and the process proceeds to step 1403.

  In step 1403, it is determined whether it is frame analysis timing. The time interval at which the timing arrives is the same as the frame synthesis timing. If it is the timing, the determination is yes and the process proceeds to step 1404. Otherwise, the determination is no and the series of processing ends here.

  In step 1404, the original audio data of the frame size is extracted from the input buffer according to the overlap factor value and multiplied by the Hanning window. In the next step 1405, analysis is performed on the frame after multiplication, and parameter calculation processing for calculating parameters such as a level value, a PARCOR coefficient, a pitch frequency, and a voiced sound ratio mix is executed. After that, the calculated parameter is written to the parameter buffer 35 from the address specified by the value of the variable param_buf_adr, the value is updated according to the writing (step 1406), and the process proceeds to step 1407.

  In step 1407, it is determined whether or not the parameter buffer 35 is full, that is, there is no capacity in the parameter buffer 35 for writing parameters extracted in the next frame. If the capacity does not remain, the determination is YES, the parameter stored in the parameter buffer 35 is saved as the parameter file 36 in step 1408, 0 is substituted into the variable ana_flg, and the series of processing is terminated. Thus, when the parameters that can be written to the parameter buffer 35 are written, the analysis of the original voice data is automatically terminated. Otherwise, the determination is no and the series of processing ends here.

  Hereinafter, the subroutine processing executed in the parameter calculation processing in step 1405 will be described in detail. Here, only those related to the PARCOR coefficient, the pitch frequency, the correction thereof, and the voiced sound ratio mix will be described in detail with reference to various flowcharts shown in FIGS.

FIG. 15 is a flowchart of PARCOR coefficient calculation processing. First, the calculation process will be described in detail with reference to FIG. As described above, in this calculation process, the PARCOR coefficient is calculated using the Levinson-Durbin algorithm.
First, in step 1501, a short-time autocorrelation function is calculated from audio data for one frame. The calculation is calculated by the following formula, where R is the short-time autocorrelation function and y is the voice data. In the figure, the symbol “P” added as a subscript to the symbol “R”, that is, the value of the order P is 26 as described above.

In step 1502 following step 1501, an autocorrelation function R is added to an element (hereinafter referred to as “W ₀ ”, the same applies to other elements) in which the subscript (number in parentheses) of the array variable W is designated as ₀ . ₁ Substitute autocorrelation function R ₀ for element E ₀ of array variable E and 1 for variable n. Thereafter, the process proceeds to step 1503.

In step 1503, a value (= W _n-1 / E _n-1 ) obtained by dividing the value of element W _{n-1 by} the value of element E _n-1 is assigned to element k _n of array variable k, and element E _n is a value obtained by subtracting the value obtained by subtracting the value of the element k _n from 1 to the value of the element E _n−1 (= E _n−1 · (1−k _n ² )). substitute. Value assigned to the element k _n is PARCOR coefficients (partial autocorrelation coefficients).

In step 1504 following step 1503, the element specified by the values of the two variables n of the array variable α ^{(denoted as} “α _n ⁽ⁿ⁾ ” in the figure, and the notation is used hereinafter) is negative of the element k _n . of substituting the value (= -k _n), (the value of the variable n, and elements that are specified by the value of the variable i) elements α _i ⁽ⁿ⁾ to from the value of the element α _i ^(n-1), component k values to the elements α _ni ^(n-1) value obtained by subtracting the value obtained by multiplying the value of ^{_{n (= α i (n-}} 1) -k n α ni (n-1)) is substituted for. The latter is performed for all elements α _i ^{(n) in} which the value of the variable i (index) is within the range of 1 ≦ i ≦ n−1. After such substitution is completed, the routine proceeds to step 1505.

  In step 1506, it is determined whether or not the value of the variable n is equal to the value of the order P. If these values match, the determination is yes, and the series of processing ends, assuming that the calculation of the PARCOR coefficient is complete. Otherwise, the determination is no and the process moves to step 1506.

In step 1506, a value obtained by the following equation is substituted for the element W _n . Thereafter, in step 1507, the value of the variable n is incremented, and the process returns to step 1503.

In this way, by repeatedly executing the processing loop formed in steps 1503 to 1507 until the determination in step 1505 becomes YES, the PARCOR coefficient for the order P is assigned to each element W _n specified by the variable n. Will be.
FIG. 16 is a flowchart of the pitch frequency calculation process. Next, the calculation process will be described in detail with reference to FIG.

  First, in step 1601, FFT is performed on the frame after multiplying by the Hanning window, the frequency amplitude value x and the phase are obtained from the real value and imaginary value obtained thereby, and the above equation (1) is used. Then, the autocorrelation function r of the frequency amplitude value x is calculated. The obtained phase is substituted for each element of the one-dimensional array variable phase. At this time, the value previously assigned to each element of the array variable phase is assigned to the element designated by the same value of the array variable old_phase. The value assigned to the element of the array variable old_phase corresponds to the phase data 56 of the previous frame shown in FIG. Then, the value of the subscript n that maximizes the correlation value r within the frequency range of human speech is specified, and that value is substituted into the variable i. The value substituted for the variable i is an index value of the frequency amplitude value x corresponding to the pitch frequency.

  In Step 1602 following Step 1601, a value obtained by subtracting the value of the element old_phase [i] of the array variable old_phase from the value of the element phase [i] specified by the value of the variable i of the array variable phase to the variable delta_phase. (Phase shift value indicating phase shift) is substituted, and the value of element phase [n] is substituted for element old_phase [n] (1 ≦ n ≦ 13). The reason why the substitution is limited to the range of 1 ≦ n ≦ 13 is that it is only necessary to search the pitch frequency only for the frequency within the range that human voice can take. After such substitution is completed, the process proceeds to step 1603.

  In step 1603, the variable offset_phase includes the FFT grid angular frequency Δω (= 2π × Δf (= sampling frequency FS ÷ FFT points (frame size)), variable i, and time difference Δt (= hop size H ÷ sampling) with the previous frame. A value (= delta_phase−Δω · i · Δt) obtained by subtracting the theoretical phase lead value obtained by multiplying each value of the frequency FS) from the value of the variable delta_phase is calculated and substituted. The value corresponds to the deviation between the frequency grid and the pitch frequency by FFT.

  In the next step 1604, normalization processing is executed so that the value of the variable offset_phase satisfies −π <offset_phase <π. In the subsequent step 1605, a value obtained by dividing the value of the normalized variable offset_phase by the time difference Δt is newly substituted into the variable. Step 1606 proceeds after the substitution. The newly assigned value represents the phase in the dimension of angular frequency.

  In step 1606, a value obtained by dividing the value of the variable offset_phase by the inter-grid angular frequency Δω is substituted for the variable delta. The value thus obtained is a shift component from the FFT frequency grid, that is, a decimal component of the variable i (index value). From this, in the following step 1607, a value obtained by adding the value of the variable delta to the value of the variable i is multiplied by the inter-grid frequency Δf to calculate the pitch frequency (= (i + delta) × Δf), and this is set to the variable freq. substitute. The series of processing is finished thereafter.

  As described above, the correlation value r is large for a sound or musical tone having a harmonic structure, that is, a harmonic component. For this reason, accurate pitch extraction can be performed without the influence of noise or the like. Further, by calculating the frequency that becomes the difference between the FFT frequency grids by the phase difference measurement method performed thereafter, the low frequency resolution of the FFT does not affect the pitch extraction. As a result, highly accurate pitch extraction resistant to noise and the like is realized.

  If the pitch frequency obtained as described above is intended for speech such as ordinary spoken words, high accuracy can be obtained. However, high accuracy is not always obtained with voices such as when singing or when special pronunciations are made. Therefore, in this embodiment, after the pitch frequency calculation process is executed, the pitch correction process shown in FIG. 17 is executed. Next, the pitch correction process will be described in detail with reference to FIG. The symbol “pitch” in FIG. 17 indicates the pitch frequency assigned to the variable freq, and the symbol “pitch_old” indicates the pitch frequency extracted (specified) in the previous frame.

  First, in step 1701, the previous pitch frequency pitch_old is subtracted from the current pitch frequency pitch, and the subtraction result is substituted into a variable x. In the next step 1702, the absolute value of the previous value is substituted for the variable x. Thereafter, the process proceeds to step 1703.

  In step 1703, it is determined whether the value of the variable x is larger than a constant THRESH. The constant THRESH is a constant set for determining whether or not a frequency change that is unlikely to occur in a short time (here, the time difference Δt corresponds) has occurred. Therefore, if such a change occurs in the pitch frequency, the determination is yes and the process moves to step 1705. Otherwise, the determination is no, and in step 1704, 0 is substituted into the variable counter, and the current pitch frequency pitch is set as the pitch frequency pitch_old of the previous frame so that the current pitch frequency pitch is accurate. Then, the series of processing is terminated.

  In step 1705, the value of the variable counter is incremented. In the subsequent step 1706, it is determined whether or not the value of the variable counter is smaller than the value of the constant H. If the former is smaller than the latter, the determination is YES, and setting the pitch frequency pitch_old of the previous frame as the current pitch frequency pitch in step 1707 indicates that the current pitch frequency pitch is likely to be inaccurate. After adopting, the series of processing ends. Otherwise, the determination is no and the process of step 1704 is executed next.

  In this way, in this embodiment, if the change in pitch frequency between frames is relatively large, the change is not adopted unless the change is continued for the number of times set by the constant H. Thereby, since the pitch frequency extracted inappropriately temporarily is invalidated, the pitch frequency can be extracted more appropriately and stably as a whole. It was confirmed that the value of the constant THRESH was about 30 and that the value of the constant H was about 4, but these are appropriately adjusted according to the person who inputs the original voice, the generation method thereof, and the like. It is desirable.

FIG. 18 is a flowchart of the mixing ratio (voiced sound ratio) calculation process. Finally, the calculation process will be described in detail with reference to FIG.
First, in step 1801, the standard deviation deviation of the autocorrelation value r is calculated, and it is determined whether or not the value is equal to or less than a constant CROSS_MIN. The constant CROSS_MIN is a constant set as a reference to be regarded as a complete voiced sound. If the value of the standard deviation deviation is less than that, the determination is YES, and in step 1802, 1 is set as the mixing ratio (voiced sound ratio) mix. After setting, the series of processing is terminated. Otherwise, the determination is no and the process moves to step 1803.

  In step 1803, it is determined whether or not the value of the standard deviation division is equal to or greater than a constant CROSS_MAX. The constant CROSS_MAX is a constant set as a reference for considering it as a complete unvoiced sound. If the value of the standard deviation deviation is more than that, the determination is YES, and 0 is set as the mixing ratio (voiced sound ratio) mix in step 1804. After that, a series of processing is finished. Otherwise, the determination is no, the process proceeds to step 1805, and the value obtained by dividing the subtraction result obtained by subtracting the value of the standard deviation division from the constant CROSS_MAX by the subtraction result obtained by subtracting the constant CROSS_MIN from the constant CROSS_MAX. And the calculated value is set as the mixing ratio mix. A series of processing shifts thereafter.

  In this way, in this embodiment, when it can be regarded as a completely voiced sound or when it can be regarded as a completely unvoiced sound, 1 or 0 is set as the mixing ratio mix, and in other cases, the value of the standard deviation deviation is set. A value that changes depending on the value is set. As a result, the value of the mixing ratio mix is changed within the range of 0 <mix <1 for only voices that cannot be completely regarded as voiced and unvoiced sounds, and the drive sound source waveform is changed to Rosenberg only when necessary. Generated from the wave and the white noise wave. In this way, the driving sound source waveform can be generated more appropriately. Therefore, it is possible to synthesize voice data that always feels natural. It was confirmed that the value of the constant CROSS_MAX was about 0.36 and the value of the constant CROSS_MIN was about 0.14, but these values were appropriately determined depending on the person who inputs the original voice, the generation method, etc. It is desirable to adjust.

  In this embodiment, the parameters extracted from the original voice data are temporarily stored as the parameter file 36, but the extraction (analysis) and synthesis of the voice data using the extracted parameters are performed in real time. You may be able to do it. You may make it output to an external apparatus.

  A plurality of voice data may be synthesized in parallel. It may be made possible by operating parameters. The pitch designation of the audio data to be synthesized may be performed using a stringed instrument, a wind instrument, or the like, or the pitch of a musical sound generated by SMF reproduction may be used.

  The PARCOR coefficient is extracted by an all-pole filter based on linear analysis, but the filter may be a pole-zero filter using a Kalman filter or the like. A lattice filter may be used. A filter coefficient other than the PARCOR coefficient may be extracted.

Although a white noise wave is generated as a sound source waveform for unvoiced sound, a residual signal is calculated during voice analysis, and when the voiced sound ratio mix is low, the residual signal is stored as the sound source waveform, You may make it use at the time of a synthesis | combination. The voice to be analyzed is assumed to be a human voice, but is not limited thereto. However, in order to maintain the accuracy of pitch extraction, it is desirable that the pitch range can be specified in advance.
<Second embodiment>
In the first embodiment, voice data synthesis based on the analysis result of the original voice input from the microphone 7 can be performed in response to an operation on the keyboard 2. On the other hand, in the second embodiment, the synthesis of audio data is performed in synchronization with the reproduction of sequence data (here, SMF).

  The configuration of the electronic musical instrument equipped with the speech analysis / synthesis apparatus according to the second embodiment is basically the same as that in the first embodiment. The operation is also largely the same or relatively small. For this reason, the same or different ones that are not different from each other will be described by focusing on the different parts from the first embodiment while using the reference numerals in the description of the first embodiment as they are. I will do it.

  In playback of sequence data, the tempo can be specified. For this reason, in the second embodiment, a change in tempo is supported. The pitch of the voice data to be synthesized is that of the input voice. As a result, in the parameter file 36, when the one that stores the voice analysis result of a person different from the person who inputs the voice is selected, the voice data is synthesized in the form of voice quality conversion. I am allowed to do it. As a person who inputs voice, a singer who sings is assumed.

  FIG. 19 is a functional configuration diagram for the analysis phase of the speech analysis and synthesis device according to the second embodiment. First, with reference to FIG. 19, the configuration for the analysis phase and the operation of each unit will be described in detail. Note that, as described above, the same reference numerals are assigned to those that are the same as those of the first embodiment or that are not different enough to be distinguished. The same applies to the composition for the synthesis phase described later.

  The tempo instruction means 1901 instructs the singer to specify the tempo. The tempo value set for the tempo is acquired by the tempo acquisition unit 1902 and stored in the parameter buffer 35 as a parameter. The instruction means 1901 may be an external device such as a metronome or a rhythm box.

Unlike the first embodiment, the second embodiment does not extract the pitch of the original speech. Therefore, instead of the pitch extraction / voiced sound ratio calculation unit 34, a voiced sound ratio calculation unit 1903 that calculates only the voiced sound ratio mix is installed. The calculation method is the same as in the first embodiment.
FIG. 20 is a functional configuration diagram for the synthesis phase of the speech analysis and synthesis device according to the second embodiment. Next, the composition for the synthesis phase and the operation of each unit will be specifically described with reference to FIG.

  The SMF 2001 is sequence data to be reproduced, and is configured such that time data indicating the processing timing is added to MIDI (event) data indicating the contents of a performance event to be generated. The sequencer 2002 reproduces the SMF 2001 having such a configuration. The synthesis of the audio data is performed in synchronization with the reproduction of the SMF 2001 by the sequencer 2002.

  The operation unit 2003 changes the tempo in response to a user operation on the tempo switch 23 shown in FIG. As a matter of course, the speed at which the SMF 2001 is reproduced has to be changed as the tempo is changed. From this, the time control unit 2004 realizes a change in the reproduction speed accompanying a change in the tempo. This is achieved by changing the cycle of the clock supplied to the sequencer 2002.

  The pitch extraction unit 2005 performs pitch extraction of the original voice data output from the A / D converter 8 by the same method as in the first embodiment, and outputs the pitch frequency extracted thereby. The drive sound source generation unit 2006 generates at least one of the Rosenberg wave and the white noise wave to be generated according to the value of the voiced sound ratio (mixing ratio) mix in the parameters read from the parameter file 36 into the parameter buffer 35. It is used to generate a driving sound source waveform. The pitch frequency extracted by the pitch extraction unit 2005 is reflected in the generation of the Rosenberg wave. The drive sound source generation unit 2006 includes the Rosenberg wave generation unit 42 and the white noise generation unit 43 in FIG. 3, and sets the value of the voiced sound ratio mix to the waveform generated by them and the value obtained by subtracting it from 1, respectively. A drive sound source waveform is generated by multiplication and mixing, and is stored in the drive sound source buffer 44. The drive sound source waveform stored in the drive sound source buffer 44 is sent to the synthesis filter 45.

  The file control unit 2007 reads parameters from one of the parameter files 36 and stores them in the parameter buffer 35 in accordance with instructions from the time control unit 2004. The stored parameters are sent to the time control unit 2004, the synthesis filter 45, and the output control unit 2008 as needed according to an instruction from the time control unit 2004.

The synthesis filter unit 45 synthesizes audio data for one frame using the parameter (PARCOR coefficient) sent from the file control unit 2007 and the driving sound source waveform, and sends the synthesized audio data to the output control unit 2008.
The output control unit 2008 includes an output buffer, multiplies the audio data for one frame synthesized by the synthesis filter unit 45 by a Hanning window, and superimposes the obtained audio data on other frames with an overlap factor. So that it is stored in the output buffer.

The audio data read from the output buffer is added (mixed) by the adder 2009 and the waveform data output from the sequencer 2002. The audio data after the addition is output to the D / A converter 10.
The waveform data output from the sequencer 2002 is generated by the tone generator 9. The output buffer is an area secured in the RAM 5. For this reason, the tone generator 9 has a function of mixing data generated by itself with data input from others. Thereby, the adder 2009 is realized by the tone generation unit 9.

  The analysis of the speech waveform is performed with a frame size of 512. However, since the playback speed of the SMF 2001 changes as the tempo changes, the synthesis of audio data must also be matched to the change. The use of parameters stored in the parameter file 36 in units of frames must be matched to the playback speed.

  These means that the voice data to be synthesized must be compressed and expanded in the time axis direction while maintaining the specified pitch (here, the pitch of the original voice). Therefore, in this embodiment, the frame size is changed according to the tempo value designated by the operation on the tempo switch 23 (see FIG. 2). The frame size is determined by the time control unit 2004. Specifically, the frame size is determined as follows. This will be specifically described with reference to FIG.

  The time control unit 2004 receives the specified tempo value from the operation unit 2003 and the tempo value stored in the parameter file 36 from the file control unit 2007, respectively. Here, for the sake of convenience, the former tempo value is referred to as a composition tempo value, and the latter tempo value is referred to as an analysis tempo value.

  When the synthesizing tempo value is twice the analyzing tempo value, as shown in FIG. 21, if the sampling frequency is equal, the synthesizing data must be output for one frame in half the time of analysis when synthesizing. For this reason, the frame size is determined to be half of that at the time of analysis. By setting the hop size to half of 32, the overlap factor maintains the value at the time of synthesis. On the other hand, when the tempo value at the time of synthesis is ½ times the tempo value at the time of analysis, audio data for one frame must be output at the time of synthesis twice as long as at the time of analysis. The size is determined to be 1024, which is twice that at the time of analysis, and the hop size is also doubled to 128 to maintain the overlap factor.

  By changing the frame size (including the hop size in this case) according to the tempo value as described above, it is possible to appropriately synthesize audio data using parameters obtained by analysis. For this reason, the pronunciation by the synthesized voice data combined with the reproduction of the SMF 2001 is always natural.

Both frame size and hop size are integers. Depending on the relationship between the analysis tempo value and the composition tempo value, at least one of these values may not be an integer. For this reason, the actual decision is made so that it does not happen.
Regarding the operation of the electronic musical instrument for realizing the sound conversion apparatus according to the second embodiment, in the analysis process (see FIG. 14), a tempo value is acquired at the time of analysis and stored as a parameter, a frame In the synthesizing process (see FIG. 12), the part for synthesizing the audio data for one frame with the determined size is mainly different from the first embodiment. In other processes, the switch process executed as step 902 and the musical tone timer interrupt process (see FIG. 13) in the overall process shown in FIG. 9 are relatively different from those in the first embodiment. Therefore, in the second embodiment, only the switch process and the timer interrupt process will be described.

  FIG. 22 is a flowchart of the switch process in the second embodiment. First, the switch process will be described in detail with reference to FIG. Here, the same reference numerals are given to the processes of steps that are the same as those in the first embodiment or need not be distinguished from each other, and the description thereof will be omitted.

  In step 2201, which is determined as NO in step 1001 or shifted by executing the processing in step 1004 or 1005, it is determined whether the start switch 21 is turned on and the value of the variable play_flg is zero. The variable play_flg is a variable for managing the playback of the SMF 2001. 0 indicates unplayed and 1 indicates that playback is in progress. Therefore, if the user operates the start switch 21 while the SMF 2001 is not being played back, the determination is YES, the parameter file 36 specified by the value of the variable file_num is read into the parameter buffer 35, and the playback of the SMF 2001 is progressed. The execution prohibition of the timer interrupt processing for unit time counting for management is canceled (indicated as “synchronous clock start” in the drawing), and further, a process of substituting 1 into the variable play_flg is executed in step 2202, and then step 2203 is executed. Migrate to Otherwise, the determination is no and the process of step 2203 is executed next.

  Time data constituting the SMF 2001 is expressed in a predetermined unit time (a time corresponding to 1/24 of a quarter note in MIDI). The timer interrupt process (hereinafter referred to as “timer timer interrupt process” for convenience) is a process executed by an interrupt signal generated every unit time, and is updated by executing the process. The arrival of the processing timing represented by the time data is determined by referring to the value of. The unit time is changed according to the tempo value, so that the SMF 2001 can be reproduced at a speed matching the set tempo value.

  In step 2203, it is determined whether or not the stop switch 22 is turned on and the value of the variable play_flg is 1. If 1 of the variable play_flg is substituted, that is, if the user operates the stop switch 22 during playback of the SMF 2001, the determination is YES, the output buffer is cleared, and the sound generation by the sound data stored therein is terminated, and the timer for timekeeping is completed. Execution of the interrupt process is prohibited, and a process of substituting 0 into the variable play_flg is executed in step 2204, and then the process proceeds to step 2205. Otherwise, the determination is no and the process of step 2205 is executed next.

  In step 2205, it is determined whether or not the up switch 23a for instructing to increase the tempo value is turned on. If the switch 23a is operated by the user, the determination is YES, and after the variable tempo_user value substituted with the user-specified tempo value is incremented in step 2206, and further, the frame correction process for correcting the frame size is executed in step 2207. The process proceeds to step 2208. Otherwise, the determination is no and the process moves to step 2208 without executing the process of other steps.

  In step 2208, it is determined whether or not the down switch 23b for instructing to lower the tempo value is turned on. If the switch 23b is operated by the user, the determination is yes, the value of the variable tempo_user is decremented in step 2209, the frame correction process is further executed in step 2210, and the process proceeds to step 1012. Otherwise, the determination is no and the process moves to step 1012 without executing the process of other steps. A description of the processing after step 1012 is omitted.

  As described above, in the second embodiment, every time the user operates the up switch 23a or the down switch 23b, the frame size at the time of synthesis is corrected and updated. Thereby, it is possible to always synthesize audio data with an appropriate frame size.

FIG. 23 is a flowchart of the frame correction process executed as step 2207 or 2210. Next, the correction process will be described in detail with reference to FIG.
First, in step 2301, the tempo value tempo_org at the time of analysis read from the parameter file 36 is divided by the value of the variable tempo_user, and the result obtained by multiplying the division result by the hop size hop_size at the time of analysis is rounded to an integer value ( = INT (hop_size × tempo_org / tempo_user)) is substituted into the variable new_hop_size. In the subsequent step 2302, a value obtained by multiplying the value of the variable new_hop_size by the value of the overlap factor OLF (represented by “8” here) is substituted for the variable new_frame_size. In the next step 2303, the tempo value tempo_org is multiplied by the value of the frame size frame_size at the time of analysis, and the multiplication result is divided by the value of the variable new_frame_size, which is a rounded integer value (= INT (tempo_org × frame_size). / New_frame_size)) is substituted into the variable tempo. The series of processing is finished thereafter.

  As described above, in the second embodiment, when the user changes the tempo value, first, the hop size corresponding to the changed tempo value is obtained as an integer value, and the frame size to be set next is determined from the obtained hop size. The tempo value actually set is obtained from the obtained frame size. As a result, the hop size, frame size, and actual tempo value are all determined as integer values.

  FIG. 24 is a flowchart of the musical tone timer interrupt process in the second embodiment. This is a process executed by an interrupt signal generated at a sampling period, for example. For example, in the switch process shown in FIG. 22, when 1 is newly assigned to at least one of the variables ana_flg and play_flg, the interrupt (execution) prohibition is canceled (interrupt is enabled), and both of these values are 0. When it becomes, interrupt is prohibited (interrupt is disabled). Finally, the timer interrupt process will be described in detail with reference to FIG.

  First, in step 2401, an analysis process for analyzing the input original voice data is executed. In the following step 2402, it is determined whether or not the value of the variable play_flg is 1. If 1 is assigned to the variable, the determination is yes and the process proceeds to step 2303. If not, the determination is no and the series of processes ends here.

  In step 2403, referring to the value of the variable updated by the execution of the timekeeping timer interrupt process, the SMF regeneration process for proceeding with the regeneration of the SMF 2001 is executed. The reproduction is performed by sequentially sending the MIDI data at the processing timing to the musical sound generating unit 9. After the reproduction process is executed, the process proceeds to step 2404.

  In step 2404, it is determined whether there is a voice input. If the original sound having a volume higher than the predetermined volume is not input to the microphone 7, the determination is NO because there is no sound input, the sound generation by the sound data stored in the output buffer is terminated, and 0 is substituted for the variable note_on. After executing the processing to be performed in step 2405, the series of processing is terminated. Otherwise, the determination is yes and the process moves to step 2406. The variable note_on is a local variable, and 1 is substituted while the voice input continues, and 0 is substituted otherwise.

  In step 2406, it is determined whether or not the value of the variable note_on is 1. If the voice input has continued, 1 is assigned to it, so the determination is yes and the process moves to step 2408. Otherwise, the determination is no, the output buffer is cleared, a value specifying the address located at the head of the output buffer (here, an area for storing one audio data) is substituted into the variable out_buf_adr, and the parameter After performing a process of substituting a value specifying an address (here, an area for storing parameter data) located at the head of the buffer 35 into a variable param_buf_adr and further substituting 1 into a variable note_on in step 2407, Control goes to step 2411.

  In step 2408, a process for continuing the sound generation of the musical sound by sending the audio data in the output buffer to the D / A converter 10 via the musical sound generating unit 9 is executed. At this time, if the musical sound instructed by the MIDI data is being generated, the musical sound generating unit 9 mixes the sound data with the waveform data generated for the musical sound generation.

  In step 2409 following step 2408, it is determined whether or not it is a frame synthesis timing. If the timing (the cycle varies depending on the frame size), the determination is YES, and if the parameter of the frame located at the end of the parameter buffer 35 has not been read, the variable param_buf_adr is read according to the frame from which the parameter is to be read next. Is updated (step 2410). In step 2411 to which the process proceeds after the update, the parameters are read from the address specified by the value of the variable param_buf_adr in the parameter buffer 35, the voice data is synthesized, and the voice data for one frame obtained by the synthesis has already been generated. By adding to other frames, the newly synthesized audio data for one frame is added to the audio data stored in the output buffer. Specifically, a Rosenberg wave with a pitch extracted from the original voice data and a white noise wave are generated, and the generated waveform is generated by mixing the generated waveforms according to the voiced sound ratio mix, and generated. After synthesizing the audio data of the frame size that changes according to the tempo value from the driving sound source waveform and the PARCOR coefficient, the synthesized audio data is multiplied by the Hanning window, and then generated from the address specified by the value of the variable out_buf_adr Add to other frames and superimpose, and after superimposing, determine the value that specifies the address to start writing the next frame in consideration of the hop size that changes according to the tempo value, and assign it to variable out_buf_adr to update To do. The series of processing is finished thereafter.

  In this manner, in the second embodiment, the voice data is synthesized only when it is determined that there is a voice input. Thereby, the period when the voice data is synthesized, the timing, and the pitch (pitch) thereof can be controlled by the singer through the pronunciation of the voice.

  In the second embodiment, the present invention is applied to an electronic musical instrument equipped with a sequencer. However, the present invention may be applied to an apparatus not equipped with a sequencer. Even in such a case, by using a synchronization method such as SMPTE, it is possible to synchronize with reproduction of sequence data (SMF, etc.) by a sequencer.

  Although the frame size is set based on the analysis tempo value stored in the parameter file 36, the reference tempo value is set to the tempo value set in the SMF 2001 or the like, or set at the time of playback. The tempo value that has been used may be adopted. The tempo value may be arbitrarily set by the user. For this reason, it is not always necessary to save the tempo value as a parameter during analysis.

  In this embodiment (first and second embodiments), both voice analysis and voice synthesis can be performed, but they may be realized as separate devices. That is, the speech analysis apparatus and speech synthesis apparatus to which the present invention is applied are not necessarily installed in the same apparatus. Even in such a case, the extracted parameters can be exchanged between various people as a data file.

  The above-described speech analysis / synthesis device, speech analysis device, speech synthesis device, or a program that realizes a modification thereof is recorded on a recording medium such as a CD-ROM, DVD, or magneto-optical disk and distributed. Also good. Alternatively, part or all of the program may be distributed via a transmission medium used in a public network or the like. In such a case, the user obtains the program and loads it into a data processing device such as a computer, so that the speech analysis / synthesis device, speech analysis device, or A speech synthesizer can be realized. Therefore, the recording medium may be accessible by a device that distributes the program.

It is a block diagram of the electronic musical instrument carrying the audio | voice analysis synthesis apparatus by a present Example. It is a figure which shows arrangement | positioning of the one part switch which comprises a switch part. 1 is a functional configuration diagram of a speech analysis / synthesis device according to a first embodiment. FIG. It is a functional block diagram of a pitch extraction / voiced sound ratio calculation part. It is a figure explaining the parameter stored in a parameter buffer. It is a figure explaining the cutting-out method of a frame. It is a figure explaining a Rosenberg wave. It is a figure explaining the addition method between the frames | frames of the audio | voice data synthesize | combined per frame. It is a flowchart of the whole process. It is a flowchart of a switch process. It is a flowchart of a keyboard process. It is a flowchart of a frame composition process. It is a flowchart of a musical tone timer interrupt process. It is a flowchart of an analysis process. It is a flowchart of a PARCOR coefficient calculation process. It is a flowchart of a pitch frequency calculation process. It is a flowchart of a pitch correction process. It is a flowchart of a mixing ratio calculation process. It is a function block diagram for the analysis phase of the speech analysis and synthesis apparatus according to the second embodiment. It is a functional block diagram for the synthesis | combination phase of the speech analysis synthesizer by 2nd Example. It is a figure explaining the change method of a frame size. It is a flowchart of a switch process (2nd Example). It is a flowchart of a frame correction process (2nd Example). It is a flowchart of a musical tone timer interrupt process (second embodiment).

Explanation of symbols

1 CPU
3 Switch part 4 ROM
5 RAM
7 Microphone 8 A / D Converter 9 Musical Sound Generation Unit 10 D / A Converter 11 Amplifier 12 Speaker 13 External Storage Device

Claims (7)

In a speech analysis and synthesis device that analyzes a first speech waveform and synthesizes a second speech waveform using the analysis result,
First analysis means for analyzing the first speech waveform in units of frames and extracting parameters;
The first speech waveform is analyzed on a frame basis to calculate the autocorrelation value of the frequency amplitude value of the first speech waveform in each frame, and the standard deviation is calculated from the autocorrelation value of each frame. A second analyzing means for extracting a voiced sound ratio whose value increases as the standard deviation value decreases ;
A pitch designating means for designating the pitch;
A sound source waveform generating means for generating a sound source waveform simulating a vocal cord sound source waveform at a pitch specified by the pitch specifying means;
Other sound source waveform generating means for generating another sound source waveform having no pitch,
A drive sound source waveform is generated by mixing with the other sound source waveform so that the mixing ratio of the sound source waveform increases as the extracted voiced sound ratio value increases , and the drive sound source waveform and the parameter are used. Voice waveform synthesis means for synthesizing the second voice waveform;
A speech analysis / synthesis apparatus comprising: The parameter obtained from the first speech waveform and the voiced sound ratio are stored in a storage unit capable of storing a plurality of data groups including the parameter and the voiced sound ratio,
The speech waveform synthesis means synthesizes the second speech waveform using one of the data groups stored in the storage means;
The speech analysis / synthesis apparatus according to claim 1. The first analysis means extracts the frequency amplitude value and phase information of the first speech waveform in each frame, calculates an autocorrelation value of the frequency amplitude value, and maximizes the autocorrelation value. Extracting the pitch of the first speech waveform from the frequency amplitude value and the phase information as the parameter ,
The speech analysis / synthesis apparatus according to claim 1. The first analyzing means continuously repeats a predetermined number of times when a pitch change between a pitch extracted in the current frame and a pitch extracted in a frame before the current frame is a predetermined value or more. When followed, the pitch extracted in the previous frame is adopted as the parameter, otherwise the pitch extracted in the current frame is adopted as the parameter .
The speech analysis / synthesis apparatus according to claim 3. In a speech analyzer that extracts parameters from speech waveforms,
Voice waveform acquisition means for acquiring the voice waveform;
A first analysis unit that analyzes the speech waveform acquired by the speech waveform acquisition unit in units of frames and extracts filter coefficients used in a synthesis filter for synthesis of the speech waveform as a parameter;
The speech waveform acquired by the speech waveform acquisition means is analyzed in units of frames to calculate the autocorrelation value of the frequency amplitude value of the first speech waveform in each frame, and the standard from the autocorrelation value of each frame Second analysis means for calculating a deviation, and extracting a voiced sound ratio whose value increases as the value of the standard deviation decreases as a parameter for generating a sound source waveform input to the synthesis filter;
A voice analysis device comprising: A program to be executed by a speech analysis / synthesis apparatus that analyzes a first speech waveform and synthesizes a second speech waveform using the analysis result,
A first analysis function for analyzing the first speech waveform in units of frames and extracting parameters;
The first speech waveform is analyzed in units of frames to calculate the autocorrelation value of the frequency amplitude value of the first speech waveform in each frame, and the standard deviation is calculated from the autocorrelation values of each frame. A second analysis function for extracting a voiced sound ratio that increases as the standard deviation value decreases ;
A pitch specification function for specifying the pitch,
A sound source waveform generating function for generating a sound source waveform simulating a vocal cord sound source waveform at a pitch specified by the pitch specifying function;
Other sound source waveform generation functions for generating other sound source waveforms having no pitch,
A drive sound source waveform is generated by mixing with the other sound source waveform so that a mixing ratio of the sound source waveform increases as the extracted voiced sound ratio value increases , and the drive sound source waveform and the parameter are used. A voice waveform synthesis function for synthesizing the second voice waveform;
A program to realize A program to be executed by a speech analyzer that extracts parameters from a speech waveform,
A voice waveform acquisition function for acquiring the voice waveform;
A first analysis function for analyzing a voice waveform acquired by the voice waveform acquisition function in units of frames and extracting filter coefficients used for a synthesis filter for synthesis of the voice waveform as a parameter;
The speech waveform acquired by the speech waveform acquisition function is analyzed in units of frames to calculate an autocorrelation value, and the standard deviation is calculated from the autocorrelation value of each frame, and the value of the standard deviation becomes smaller A second analysis function for extracting a voiced sound ratio having a larger value as a parameter for generating a sound source waveform input to the synthesis filter;
A program to realize

Images