JP2005099509A - Voice processing device, voice processing method, and voice processing program - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To provide a voice processing device, a voice processing method, and a voice processing program, which give vibrato accompanied with natural tone variation. <P>SOLUTION: A template creation part 900 creates a template set TS concerned with a target voice which is supplied from a target input part 950 and to which vibrato is applied. A pitch, a gain, and an inclination of a spectrum (slope) of each frame of the target voice are recorded in the template set TS created by the template creation part 900. An vibrato applying part 400 changes pitches, gains, and slopes of a singing voice (input voice) of an amateur singer or the like under the control of a vibrato application control part 700 and performs inverse FFT of spectrums obtained by the changed pitches, gains, and slopes and supplies a voice obtained by inverse FFT (namely, a voice having the changed pitches, gains, and slopes) to a voice output part 500. <P>COPYRIGHT: (C)2005,JPO&NCIPI

Description

  The present invention relates to a voice processing device, a voice processing method, and a voice processing program that can add vibrato to an input singing voice or the like.

Vibrato, one of the singing techniques, is a technique that gives periodic pitch and amplitude fluctuations to the singing voice. By applying vibrato to a long stretched sound, it can give a rich expression (change in sound). it can. Conversely, if vibrato is not applied when singing a long sound, the change in sound will be scarce and the singing will tend to be monotonous. However, because vibrato is an advanced singing technique, there is a problem that it is difficult for an amateur singer who is not supposed to sing to sing with a beautiful vibrato.
Under such a background, various devices for automatically adding vibrato to the singing voice of an amateur singer have been proposed. For example, the following Patent Document 1 does not simply mechanically add a certain amount of vibrato, but modulates the signal according to the state of the pitch, volume, duration of the same sound, etc. of the input singing voice signal. , And a device for adding vibrato by modulating the pitch and amplitude of the singing voice signal using this modulated signal.

Japanese Patent Laid-Open No. 9-044158

  However, in the apparatus disclosed in Patent Document 1, a modulation signal is generated based on a composite signal such as a sine wave or a triangular wave generated by an LFO (Low Frequency Oscillator), so that a song to be imitated The subtle pitch and amplitude fluctuations of the vibrato sung by the musician (the target singer) cannot be reproduced, nor can it be accompanied by a natural change in timbre. There is a conventional example that uses a sample of the actual vibrato waveform of the target singer instead of the sine wave, but reproduces the natural pitch, amplitude, and timbre of the singing voice from one vibrato waveform. I can't.

  The present invention has been made in view of the circumstances described above, and an object thereof is to provide an audio processing device, an audio processing method, and an audio processing program capable of providing vibrato accompanied by natural timbre changes.

  In order to solve the above-described problem, a speech processing apparatus according to the present invention includes a storage unit that stores a pitch and a spectrum inclination of a target speech to which vibrato has been applied, an input unit that inputs speech, and the input speech. A pitch extracting means for extracting a pitch; a pitch changing means for changing the pitch of the extracted voice according to the pitch of the target voice; and a spectrum of the voice obtained by spectrally analyzing the inputted voice. Spectral slope changing means for changing the slope according to the slope of the spectrum of the target voice; and output means for outputting a voice having the pitch of the voice after the change and the slope of the spectrum of the voice after the change. It is characterized by comprising.

  According to this configuration, for example, when an amateur singer's voice or the like is input, the pitch of the input voice is changed according to the pitch of the target voice with vibrato previously stored in the storage means, The slope of the input speech spectrum is changed according to the slope of the target speech spectrum. By changing the slope of this spectrum, it is possible to change only the timbre without changing the overall volume (see FIG. 10 and FIG. 5 and FIG. 11 and FIG. 5 for comparison), as if the target singer is It becomes possible to add a vibrato showing a natural timbre change as if singing to the voice of an amateur singer.

  Here, the storage means stores gain of the target voice in addition to the pitch and spectrum inclination of the target voice, and extracts the gain of the inputted voice, and the extracted voice Gain changing means for changing the gain of the voice according to the gain of the target voice, and the output means includes the pitch of the voice after the change, the slope of the spectrum of the voice after the change, and the voice after the change. It is good also as a structure which outputs the audio | voice which has these gains.

  Further, the storage means stores a plurality of types of pitches of the target sound subjected to vibrato and a spectrum inclination corresponding to each pitch, and the pitch changing means is configured to select from among the plurality of types of pitches of the target sounds. Selecting the pitch closest to the pitch of the extracted voice, changing the pitch of the extracted voice according to the pitch of the selected target voice, and the spectrum slope changing means A spectrum slope corresponding to the pitch of the target speech selected by the pitch changing means is selected from the types of spectrum slopes, and the slope of the speech spectrum is selected according to the slope of the spectrum of the selected target speech. It is good also as a structure to change.

  Furthermore, the storage means may be configured to store the pitch and the slope of the spectrum in each region when the target sound subjected to vibrato is divided into at least a vibrato attack region and a vibrato body region.

  As described above, according to the present invention, it is possible not only to reproduce the subtle pitch and gain fluctuations of the vibrato added by the target singer, but also to faithfully reproduce the natural changes in the timbre. Become.

Embodiments according to the present invention will be described below with reference to the drawings.
A. Embodiment A-1. Overall Configuration FIG. 1 is a diagram showing a configuration of a speech processing apparatus 100 according to the present embodiment.
The voice input unit (input unit) 200 is configured by a microphone or the like, and inputs voice of an amateur singer or the like into the voice processing apparatus 100. In the following description, the voice of an amateur singer or the like input via the voice input unit 200 is simply referred to as input voice for convenience.
A spectrum / pitch analysis unit (pitch extraction means, gain extraction means) 300 is a spectrum analysis based on FFT (Fast Fourier Transform) in units of frames (about 5 to 10 ms) for input speech supplied from the speech input unit 200. To extract the pitch, spectrum, etc. for each frame. Note that the pitch extraction for each frame is not limited to spectrum analysis, and a known method may be used.
The parameter input unit 600 inputs a parameter (for example, a desired vibrato rate) for controlling the vibrato added to the input speech into the speech processing apparatus 100. Parameters for controlling the vibrato (hereinafter referred to as Vib parameters) are input by an amateur singer or the like by appropriately operating operation buttons or the like (not shown).

The target input unit (target input means) 950 inputs a sound (hereinafter referred to as a target sound) when the target singer sings with vibrato, and supplies this to the template creation unit 900.
The template creation unit (target pitch extraction means, target spectrum / slope calculation means) 900 creates a template set TS related to the target voice with vibrato supplied from the target input unit 950 and stores it in the vibrato database 800. . Details regarding creation of the template set TS will be described later.
The vibrato database (storage means) 800 stores a plurality of template sets TS related to voices when the target singer sings with vibrato at various pitches. By adopting such a configuration, when adding vibrato to the input voice, it is possible to select and use the template set TS closest to the pitch of the input voice, thereby adding realistic vibrato. It becomes possible.

A vibrato addition control unit (pitch changing means, spectrum / slope changing means, gain changing means) 700 selects a template set TS closest to the pitch of the input voice from the vibrato database 800, and selects the selected template set TS and the Vib parameter. Based on the above, vibrato added to the input voice is controlled (details will be described later).
A vibrato adding section (pitch changing means, spectrum slope changing means, gain changing means) 400 is a gain obtained by analyzing the pitch of the input voice and the spectrum of the input voice under the control of the vibrato addition control section 700. The spectrum slope (hereinafter referred to as “slope” as appropriate) is changed, and the spectrum obtained from the changed pitch, gain, and slope is subjected to inverse FFT (details will be described later). The vibrato adding unit 400 supplies the audio obtained by the inverse FFT (that is, the audio having the changed pitch, gain, and slope) to the audio output unit 500.
The audio output unit (audio output unit) 500 includes a speaker or the like and outputs the audio supplied from the vibrato adding unit 400 to the outside. By using the voice processing device 100 described above, the voice of an amateur singer without vibrato can be converted into a voice that shows natural timbre changes as if the target singer is singing. It becomes possible.

A-2. Creation of Template Set TS When the template creation unit 900 receives the target voice subjected to vibrato from the target input unit 950, the target voice is subjected to spectrum analysis based on FFT for each frame, and the pitch and gain for each frame. Find the slope. When obtaining the pitch for each frame, a well-known method may be used in addition to the spectrum analysis as in the case of the input speech.

  FIG. 2 is a diagram showing a pitch waveform which is a waveform having the pitch of the target sound to which vibrato is applied. The template creation unit 900 divides one audio waveform with vibrato into a vibrato attack area (VA area), a vibrato body area (VB area), and a vibrato release area (VR area) as shown in FIG. Specifically, the part where the vibrato is first applied (from the point where the pitch starts to change vibrato to just before the periodical change) is set as the VA area, and the part where the pitch changes periodically following the vibrato attack is set as the VB area. And the VR area is the part of the vibrato body until the sound is muted. In addition, regarding the boundary of each region, the maximum value portion (time point) of the peak of the pitch is used as the boundary in consideration of the change state of the vibrato period and the like.

  Although it is possible to add vibrato by simply using each data (ie, pitch, gain, slope) in the VB area, in the present embodiment, in order to add more realistic vibrato, the VA area , VB area, and VR area data are used. When the template creation unit 900 divides one target speech waveform into regions as described above, a template is created for each of these regions. In the following description, a template created for the VA area is called a VA template, a template created for the VB area is called a VB template, a template created for the VR area is called a VR template, A group of area templates is called a template set TS.

A-2-1. Extraction of Pitch FIG. 3 is a diagram showing a pitch waveform in the VA region. First, the template creation unit 900 sequentially extracts the pitch waveform of each frame in the VA area shown in FIG. 3, and sequentially extracts the pitch of each frame of the target speech by analyzing each pitch waveform. At this time, the template creation unit 900 generates additional information as shown below together with the pitch of each frame, and records these in the VA template. Such additional information includes start vibrato depth (mBeginDepth [cent]), end vibrato depth (mEndDepth [cent]), start vibrato rate (mBeginRate [Hz]), end vibrato rate (mEndRate [Hz]), template section length ( mDuration [s]), start pitch (mPitch [cent]), and the like.

  As shown in FIG. 3, the start vibrato depth (mBeginDepth [cent]) is the difference between the maximum value and the minimum value of the pitch of the start vibrato period, and the end vibrato depth (mEndDepth [cent]) is the pitch of the end vibrato period. It is the difference between the maximum and minimum values. The vibrato period refers to, for example, the time (s) from the maximum value of the pitch to the next maximum value. In FIG. 3, the pitch waveform of the VA region is composed of a start vibrato cycle and an end vibrato cycle (that is, the VA region has a length of two cycles). In some cases, the VA region has a length of three cycles or more. As described above, although the VA area has a length corresponding to the frequency of the VA area, the VA area does not change in terms of the area up to the VB area where the vibrato periodically changes.

  The start vibrato rate (mBeginRate [Hz]) is the reciprocal of the start vibrato cycle (= 1 / start vibrato cycle), and the end vibrato rate (mEndRate [Hz]) is the reciprocal of the end vibrato cycle (= 1 / end vibrato cycle). ). The template section length (mDuration [s]) is the temporal length of the template, and the start pitch (mPitch [cent]) is the pitch of the first frame (vibrato period) in the VA area. The additional information is used when the template is modified to obtain a desired vibrato rate or the like (described later).

  Similarly, FIG. 4 shows a pitch waveform in the VB region. As described above, the VB region is a portion where the pitch periodically varies following the vibrato attack. By using each data in this VB area in a loop according to the length of vibrato addition, vibrato longer than the template section length is realized. Note that the information recorded in the VB template by the template creation unit 900 is the same as the information recorded in the VA template, and the description thereof will be omitted. The VR region can be described in the same manner as the VA region and VB region, and will not be described.

A-2-2. Extraction of Gain and Slope FIG. 5 is a diagram illustrating a spectrum of a frame having a VA region, and FIG. 6 is a diagram for explaining a curve representing the slope (slope) of the spectrum shown in FIG. In each figure, the frequency f [Hz] is taken on the horizontal axis, and the amplitude value Magnitude [dB] is taken on the vertical axis. In FIG. 5, the spectrum envelope is indicated by a solid line, and a curve representing a slope (hereinafter referred to as “Ecurve”) is indicated by a broken line. First, EcurveMag, which is the amplitude value of Ecurve, can be expressed by the following formula (1).
EcurveMag (f) = Gain + 100 * (e−Slope * f−1) (1)
Gain; Gain Slope of the frame; Slope of the frame

  Furthermore, if this equation is bitten a little more, it will be as shown in FIG. That is, EC curve starts from Gain [dB] at a frequency f = 0 [Hz], and approaches an asymptotic line of (Gain-100) [dB] as the frequency f increases. The inclination of the Ecurve changes depending on the slope. From this equation (1), it can be seen that the magnitude of the signal can be changed purely by the gain, and the frequency characteristic (change in timbre) can be controlled by the slope (described later).

  The template creation unit 900 calculates the gain and slope for each frame by analyzing the spectrum of each frame in the VA region as described above. At this time, the template creation unit 900 includes a start gain (mGain [dB]), a start slope (mSlope), a start tremolo depth (mBegin Tremolo Depth [dB]), and an end tremolo depth (mEnd Tremolo Depth [dB] along with the gain and slope for each frame. ]) Are generated and recorded in the VA template. Here, the start gain (mGain [dB]) is the gain of the first frame in the VA region, and the start slope (mSlope) is the slope of the spectrum of the first frame in the VA region. The start tremolo depth (mBeginTremoloDepth [dB]) is the difference between the maximum value and the minimum value of the gain of the start vibrato period described above, and the end tremolo depth (mEndTremoloDepth [dB]) is the maximum value of the gain of the end vibrato period. And the difference between the minimum values.

  The additional information is used when a desired tremolo depth or the like is obtained by deforming the template in the same manner as the additional information obtained from the pitch waveform (described later). The gain, slope, and additional information for each frame in the VB area are generated in the same manner as the gain, slope, and additional information for each frame in the VA area, and are recorded in the VB template. Furthermore, the gain, slope, and additional information for each frame in the VR area are similarly generated and recorded in the VR template.

  The template creation unit 900 stores the template set TS including the VA template, VB template, and VR template created in this way in the vibrato database 800. As a result, the vibrato database 800 stores data indicating the pitch, the slope of the spectrum, and the gain in each region when the target audio subjected to vibrato is divided into the VA region, the VB region, and the VR region.

A-3. Method of adding vibrato As a method of adding vibrato to the input sound using the template set TS, the start pitch (mPitch [cent]), the start gain (mGain [dB]), and the start slope (mSlope) of the VA region are used as a reference. Based on each delta value (that is, ΔPitch [cent], ΔGain [dB], ΔSlope), the spectrum of the input speech frame is basically modified.

  As described above, since each delta value is obtained based on each data value in the VA area, discontinuity does not occur when each of the vibrato attack, the vibrato body, and the vibrato release is connected. At the start of vibrato, each data in the VA area is used only once, and then each data in the VB area is used. For each data in the VB area, a vibrato longer than the template section length of the VB area is realized by using the loop as described above. At the end of the vibrato, each data in the VR area is used only once after each data in the VB area is used. However, each data in the VR area is not necessarily used, and each data in the VB area may be repeatedly used until the end of the vibrato instead of using each data in the VR area. Hereinafter, pitch change, gain change, and slope change when vibrato is added to the input sound using the template set TS will be described. In the following, it is assumed that vibrato is added using each data in the VB area.

A-3-1. Regarding the pitch change First, when the current time relative to the start time of the vibrato is represented by Time [s] and the pitch of the VB area in this Time [s] is represented by DBPitch (Time) [cent], The delta value (ΔPitch [cent]) is expressed by the following equation (2). In the following formula (2), mPitch [cent] represents the start pitch of the VA region as described above.
ΔPitch = DBPitch (Time) −mPitch (2)

As is apparent from this equation, ΔPitch represents the amount of deviation between the current pitch of the VB area (DBPitch (Time) [cent]) and the start pitch of the VA area (mPitch [cent]). . As described above, ΔPitch obtained using the VB template is finally added to the pitch of the input voice (input Pitch [cent]) by the following equation (3), thereby adding vibrato. The pitch of the output voice (output Pitch [cent]) is generated.
Output Pitch = Input Pitch + ΔPitch (3)

A-3-2. Regarding Gain Change As described above, if the gain of the VB region in Time [s] is expressed as DBGain (Time), the gain delta value (ΔGain [dB]) is expressed by the following equation (4). Note that in the following formula (4), mGain [dB] represents the start gain of the VA region, as described above.
ΔGain = DBGain (Time) −mGain (4)
As described above, ΔGain obtained by using the VB template is finally added to the gain of the input sound (input Gain [dB]) by the following equation (5), thereby adding vibrato. Of the output voice (output Gain [dB]) is generated.
Output Gain = Input Gain + ΔGain (5)

Here, FIG. 7 and FIG. 8 are diagrams showing changes in the spectral envelope and EC curve when the magnitude of Gain is changed, and FIG. 7 is a diagram when Gain is increased (ΔGain = positive value). FIG. 8 shows a change when Gain is reduced (ΔGain = negative value). Note that the spectrum envelope and EC curve shown in FIGS. 7 and 8 are all based on the spectrum envelope and EC curve shown in FIG.
As is clear from comparison between FIGS. 7 and 5 and FIGS. 8 and 5, when Gain (in each figure, EC curve intercept) is changed, EC curve and spectrum envelope shift up and down while maintaining their shapes. The shape of the spectral envelope itself does not change. That is, even if the magnitude of Gain is changed, the overall volume changes only (the overall volume increases as Gain increases, and the overall volume decreases as Gain decreases), and the timbre changes. do not do.

A-3-3. About Slope Change As described above, if the slope of the VB region in Time [s] is expressed as DBSlope (Time), the slope delta value (ΔSlope) is expressed by the following equation (6). In addition, in the following formula (6), mSlope represents the start slope of the VA region as described above.
ΔSlope = DBSlope (Time) / mSlope (6)
ΔSlope and mslope thus obtained are further substituted into the following equation (7) to obtain Diff [dB] for each frequency Freq [Hz] of the input voice.
Diff = 100 * (exp (mslope * ΔSlope * Freq) −1.0) −100 * (exp (mslope * Freq) −1.0) (7)

Then, the output bin Magnitude [dB] is obtained by substituting Diff thus obtained into the following equation (8). Note that in the following equation (8), input bin Magnitude [dB] represents the magnitude of the component of frequency freq [Hz] in the spectrum of the input speech (see FIG. 9).
Output bin Magnitude = Input bin Magnitude + Diff (8)
Such a calculation is performed for all bins (frequency), and an output bin Magnitude [dB] in each bin is obtained. By this operation, the slope (slope) of the spectrum obtained by the spectrum analysis of the input speech changes with time.

FIG. 10 and FIG. 11 are diagrams showing how the spectral envelope changes when the Evolve Slope is changed by the above operation. FIG. 10 shows the change in the spectral envelope when the Slope is increased. , The change in the spectral envelope when the Slope is reduced. Note that the spectrum envelope and EC curve shown in FIGS. 10 and 11 are all based on the spectrum envelope and EC curve shown in FIG.
As is obvious from comparison between FIGS. 10 and 5 and FIGS. 11 and 5, when the slope is changed, the gain (the EC curve intercept in each figure) representing the overall volume does not change, but the EC curve slope is changed. With the change, the shape of the spectrum envelope changes, thereby changing the timbre. More specifically, as shown in FIG. 10, when the Slope is increased, a high-frequency spectrum is not generated, so that the tone becomes muffled. On the other hand, when the Slope is made small as shown in FIG. 11, a spectrum appears uniformly from the low range to the high range, so that a bright tone is obtained.

  Next, a method of using this VB template to obtain a desired vibrato rate, a desired pitch depth (pitch wave depth), and a desired tremolo depth (gain wave depth) will be described.

A-4. When obtaining a desired vibrato rate To obtain a desired vibrato rate, first, the reading time (speed) of the template is changed by the following formulas (9) and (10). In the following formulas (9) and (10), the desired vibrato rate is represented by VibRate [Hz], and the template start vibrato rate and the template end vibrato rate are represented by mBeginRate [Hz] and mEndRate [Hz], respectively. . Also, the time when the template reading start time is set to 0 is represented by Time [s], and the reading time of the VB template obtained by Expression (10) is represented by NewTime [s].
VibRateFactor = VibRate / {(mBeginRate + mEndRate) / 2} (9)
NewTime = Time * VibRateFactor (10)
In this way, a desired pitch change is realized by changing the reading time (speed) of the template.

A-5. Case of Obtaining Desired Pitch Depth Next, regarding pitch depth, a new ΔPitch is obtained by the following equation (11), thereby realizing a desired pitch depth. In the following formula (11), the desired pitch depth is represented by Pitch Depth [cent], and the start vibrato (pitch) depth of the VB template and the end vibrato (pitch) depth of the VB template are respectively mBeginDepth [cent] and mEndDepth [cent]. Represented by Also, the time when the template reading start time is 0 (template reading time) is represented by Time [s], and the pitch delta value at Time [s] is represented by ΔPitch (Time) [cent].
ΔPitch = ΔPitch (Time) * PitchDepth / {mBeginDepth + mEndDepth) / 2} (11)

However, when the template reading time is changed to NewTime (s) as described above, a new ΔPitch may be obtained by the following equation (11) ′.
ΔPitch = ΔPitch (NewTime) * PitchDepth / {mBeginDepth + mEndDepth) / 2} (11) ′

A-6. Case of Obtaining Desired Tremolo Depth Next, for tremolo depth, a new ΔGain [dB] is obtained by the following equation (12), thereby realizing a desired tremolo depth. In the following formula (12), the desired tremolo depth is represented by Tremolo Depth [dB], the start tremolo depth of the VB template and the end tremolo depth of the VB template are represented by mBeginTremoDepth [dB] and mEndTremoDepth [dB], respectively. Also, the time when the template reading start time is 0 (template reading time) is represented by Time [s], and the gain delta value at Time [s] is represented by ΔGain (Time) [dB].
ΔGain = ΔGain (Time) * TremoloDepth / {(mBeginTremoloDepth + mEndTremoDepth) / 2} (12)

However, when the template reading time is changed to NewTime (s) as described above, a new ΔGain may be obtained by the following equation (12) ′.
ΔGain = ΔGain (NewTime) * TremoloDepth / {(mBeginTremoloDepth + mEndTremoDepth) / 2} (12) ′

Further, since the change in timbre is considered to be mainly linked to the volume of the sound, a new Diff [dB] is obtained by the following equation (13). In addition, Diff [dB] shown on the right side of the following formula (13) indicates Diff [dB] obtained by the above formula (7).
Diff = Diff * TremoDepth / {(mBeginTremoloDepth + mEndTremoDepth) / 2} (13)

However, when the reading time of the template is changed to NewTime [s] as described above, Diff [dB] in NewTime [s] is obtained using the above formulas (6) and (7), and this Diff [ Substituting [dB] into the right side of equation (13), a new Diff [dB] may be obtained.
In the above, the case of adding vibrato using a VB template has been described as an example, but the case of adding vibrato using a VA template or a VR template can also be described in the same way, so further explanation is omitted. To do.

A-7. Operation of Embodiment Hereinafter, an operation in the case of adding vibrato using the audio processing device 100 according to the present embodiment will be described. In the following description, it is assumed that vibrato is added without using each data in the VR area.
12 and 13 are flowcharts showing a vibrato addition process executed by the vibrato addition control unit 700. FIG. When the pitch of the input speech is detected and determined to be voiced, the following processing for adding vibrato to the speech is started. The vibrato addition control unit 700 receives a Vib parameter input by an amateur singer or the like from the parameter input unit 600 in order to control vibrato (step S1). Here, as the Vib parameter input to the parameter input unit 600, for example, a vibrato delay (VibDelay [s]) indicating a time length from when a voiced sound is detected until the actual addition of vibrato is started, Vibrato period (VibRate [Hz]) representing a period, Pitch depth (PitchDepth [cent]) representing the depth of pitch fluctuation in vibrato, Tremolo Depth (TremoloDepth [DB]) representing the depth of fluctuation in volume in vibrato Etc. Note that the timing of starting the vibrato application (that is, starting the application of the vibrato attack part) is triggered by the detection of a voiced sound as described above, or by the detection of a pitch change. good. According to this aspect, for example, when voiced sounds having different pitches are continuously input, the vibrato attack unit is applied only to the first pitch sound, or both the first pitch sound and the next pitch sound are applied. The vibrato attack part can be applied to Of course, the present invention is not limited to these, and it is possible to appropriately set how the detection result of the pitch change is used to start the addition of vibrato.

  Next, the vibrato addition control unit 700 initializes an algorithm for adding vibrato. Here, for example, the flag VibAttackFlag and the flag VibBodyFlag are set to “1” (step S2). Each of these flags is a flag for determining whether or not to use a VA template or a VB template. When the flag VibAttackFlag is set to “1”, the VA template is used, and the flag VibAttackFlag is “0”. When the flag VibBodyFlag is set to “1”, the VB template is used, and when both the flag VibAttackFlag and the flag VibBodyFlag are set to “0”, it means that the addition of the vibrato is completed. In addition, as described above, the addition of vibrato is started after the time indicated by the vibrato delay (VibDelay [s]) of the Vib parameter has elapsed since the input sound was detected as a voiced sound.

  When the vibrato addition control unit 700 proceeds to step S3, the pitch detected in step S1 of the input voice input via the voice input unit 200 from the plurality of types of template sets TS stored in the vibrato database 800. (In this operation example, the template set TS composed of the VA template and the VB template) is selected (step S3). Since the details of the creation of the template set TS are clarified in A-2 above, description thereof is omitted here.

  Then, the vibrato addition control unit 700 acquires the time length of the selected VA template (VibAttackDuration [s]) and the time length of the VB template (VibBodyDuration [s]) (Step S4), and proceeds to Step S5.

  In step S5, the vibrato addition control unit 700 checks the value of the flag VibAttackFlag. If the value of the flag VibAttackFlag is “1”, the process proceeds to step S6. If the value of the flag VibAttackFlag is “0”, the process proceeds to Step S10.

  In step S6, the vibrato addition control unit 700 reads the VA template from the template set TS selected in step S3, and sets this as DBdata (DBPitch, DBGain, DBSlope). Also, VibRateFactor is calculated by the above equation (9).

  In step S7, the vibrato addition control unit 700 calculates the template reading time from the calculated VibRateFactor according to the above equation (10) to obtain NewTime [s].

  In step S8, the vibrato addition control unit 700 determines whether the template reading time NewTime [s] obtained as described above exceeds the time length VibattDuration [s] of the VA template (see step S4). To do. If the vibrato addition control unit 700 determines that the template reading time NewTime [s] exceeds the time length VibattDuration [s] of the VA template (that is, the VA template has been used to the end) (step S8; YES). The process proceeds to step S9. On the other hand, if the vibrato addition control unit 700 determines that the template reading time NewTime [s] does not exceed VibAttackDuration (step S8; NO), the process proceeds to step S15 to add vibrato using the VA template.

A-7-1. When proceeding to step S15 (step S8; NO)
When the process proceeds to step S15, the vibrato addition control unit 700 obtains the value of each parameter (Pitch, Gain, Slope) at time NewTime [s] from the DB data of the VA template. At this time, when the time NewTime [s] is located in the middle of a certain frame time of the actual data in the VA template, the values of the parameters in the frames before and after NewTime [s] are interpolated (for example, linear interpolation). The values of the above parameters are obtained.

  And the vibrato addition control part 700 calculates | requires each delta value ((DELTA) Pitch, (DELTA) Gain, (DELTA) Slope) of each parameter (step S16). However, when calculating each delta value, as described in the above A-5 and A-6, the values of Pitch Depth [cent] and Tremolo Depth [dB] are reflected (the above formulas (11) to (13)). reference).

  When the process proceeds to step S17, the vibrato addition control unit 700 obtains a coefficient Muldelta as shown in FIG. The coefficient Muldelta is used to converge the vibrato by gradually decreasing the Δ value of each parameter from the time when the end condition of the vibrato (for example, when the pitch of the input speech is not detected; details will be described later) is satisfied. Is the coefficient. If the vibrato is stopped at the same time as the time when the vibrato is to be applied without performing such processing, a problem arises in that the pitch or the like changes suddenly at that time. In this embodiment, in order to avoid such a problem, a coefficient Muldelta for converging the vibrato is obtained. As is clear from the graph shown in FIG. 14, when the time when the end condition of the vibrato is not reached, the coefficient Muldelta is set to “1”, and the process of converging the vibrato is not performed.

  When the process proceeds to step S18, the vibrato addition control unit 700 multiplies each delta value (ΔPitch, ΔGain, ΔSlope) of each parameter obtained in step S16 by the coefficient Muldelta obtained in step S17. Then, the vibrato addition controller 700 supplies the multiplied result to the vibrato adder 400, and the process returns to step S7. When the vibrato adding unit 400 receives the multiplication result from the vibrato adding control unit 700, the pitch, gain, and slope of the input speech obtained previously are set to each delta value (ΔPitch, ΔGain, ΔSlope) indicated in the multiplication result. Perform the corresponding spectral conversion. That is, the vibrato adding unit 400 calculates the output Pitch [cent] using the above-described equation (3), calculates the output Gain [dB] using the above-described equation (5), and the above-described equation ( 7) and (8) are used to calculate the output bin Magnitude [dB], and a spectrum in which the pitch, gain, and slope are changed based on these is obtained. Then, the vibrato adding unit 400 performs inverse FFT on the spectrum thus obtained and supplies the audio obtained by the inverse FFT to the audio output unit 500. As a result, the voice output unit 500 outputs a voice indicating a natural timbre change as if the target singer is singing.

A-7-2. When proceeding to step S9 (step S8; YES)
On the other hand, if the vibrato addition control unit 700 determines in step S8 that the template reading time NewTime [s] exceeds the time length VibattDuration [s] of the VA template, the process proceeds to step S9 and sets the flag VibAttackFlag to “0”. To end the vibrato attack, and set the time Time [s] at that time as VibAttackEndTime [s].
Then, the vibrato addition control unit 700 checks the flag VibBodyFlag and determines whether or not to use the VB template (step S10). When the value of the flag VibBodyFlag is “0” (step S10; NO), the vibrato addition control unit 700 ends the vibrato addition process, while when the value of the flag VibBodyFlag is “1” (step S10; YES). The process proceeds to step S11.

  When it is confirmed that the value of the flag VibAttackFlag is set to “0” and the value of the flag VibBodyFlag is set to “1”, the vibrato addition control unit 700 reads the VB template and reads the DBdata (DBPitch, DBGain, DBSlope).

  In step S12, the vibrato addition control unit 700 calculates VibRateFactor according to the above equation (9) and calculates the template reading time according to the above equation (10) to obtain NewTime [s]. However, care must be taken when calculating NewTime [s] when the VB template is looped. When the vibrato addition control unit 700 obtains NewTime [s] in this way, it determines whether or not to end the vibrato (step S13). Specifically, the vibrato addition control unit 700 is instructed to end the vibrato via an operation unit (not shown) when the pitch of the input voice is no longer detected (such as when no voiced sound is detected). In such a case, it is determined that the vibrato should be terminated when the volume of the input sound is equal to or lower than the threshold value.

  If the vibrato addition control unit 700 determines that the vibrato should not be terminated (step S13; NO), the process proceeds to step S15 '. On the other hand, when the vibrato addition control unit 700 determines that the vibrato should be terminated (step S13; YES), the process proceeds to step S14, and a flag VibConFlag for determining whether or not the vibrato should be converged is “1”. Set to. Then, the vibrato addition control unit 700 proceeds to step S15 '. When the value of this flag VibConFlag is “0”, the vibrato is not converged. When the value of this flag VibConFlag is “1”, the vibrato is converged (refer to step S17 for the convergence of the vibrato).

When the process proceeds to step S15 ′, the vibrato addition control unit 700 obtains the value of each parameter (Pitch, Gain, Slope) at time NewTime [s] from the DB data of the VB template. At this time, when the time NewTime [s] is located in the middle of a certain frame time of the actual data in the VB template, the value of each parameter in the frame before and after NewTime [s] is interpolated (for example, linear interpolation). The values of the above parameters are obtained. And
The vibrato addition control unit 700 obtains each delta value (ΔPitch, ΔGain, ΔSlope) of each parameter (step S16 ′). However, when calculating each delta value, as described in the above A-5 and A-6, the values of Pitch Depth [cent] and Tremolo Depth [dB] are reflected (the above formulas (11) to (13)). reference).

  When the process proceeds to step S17 ', the vibrato addition control unit 700 refers to the value of the flag VibConFlag and determines whether or not the time when the vibrato end condition is satisfied has been reached. If the vibrato addition control unit 700 determines that the value of the flag VibConFlag is “0” and has not yet reached the time when the end condition of the vibrato has been satisfied, the process proceeds to step S18 ′, where the coefficient Muldelta = “1” (ie, Get the value that does not converge the vibrato). Then, the vibrato addition control unit 700 proceeds to step S19 ′, and multiplies each delta value (ΔPitch, ΔGain, ΔSlope) obtained in step S16 ′ by the coefficient Muldelta = “1” obtained in step S18 ′. To do. Then, the vibrato addition controller 700 supplies the multiplied result to the vibrato adder 400, and the process returns to step S12.

  On the other hand, when the vibrato addition control unit 700 determines in step S17 ′ that the value of the flag VibConFlag is “1” and the time when the end condition of the vibrato has been satisfied, the process proceeds to step S18 ″. Each Δ value is gradually reduced to obtain a coefficient Muldelta that converges the vibrato (see Vib end time shown in FIG. 14). Then, the vibrato addition control unit 700 proceeds to step S19 ″, and multiplies each delta value (ΔPitch, ΔGain, ΔSlope) of each parameter obtained in step S16 ′ by the value of the coefficient Muldelta obtained in step S18 ″. To do. Then, the vibrato addition controller 700 supplies the multiplication result to the vibrato adder 400. When the process proceeds to step S20 '', the vibrato addition control unit 700 determines whether or not the coefficient Muldelta acquired in step S18 '' has converged to "0". When the vibrato addition control unit 700 determines that the obtained coefficient Muldelta has not yet converged to “0” (step S20 ″; NO), the vibrato addition control unit 700 returns to step S12 and repeatedly executes the above-described processing. If the vibrato control unit 700 determines that the coefficient Muldelta acquired in step S18 ″ has converged to “0” (step S20 ″; YES) while repeatedly executing such processing, the value of the flag VibBodyFlag Is set to “0” (step S21 ″), and the above-described processing is terminated. The operation after the multiplication result is supplied from the vibrato addition control unit 700 to the vibrato addition unit 400 can be described in the same manner as the case where the vibrato is added using the VA template. To do.

  As described above, the case where the vibrato is added without using each data in the VR area has been described. However, the vibrato may be added using each data in the VR area (that is, using the VR template). Of course. In such a case, when the above-described vibrato termination condition is satisfied, control may be performed so as to add vibrato using the VR template. In addition, when adding vibrato using this VR template, it is not necessary to obtain the coefficient Muldelta for converging the vibrato by calculation, and the processing burden on the control unit of the speech processing apparatus 100 can be reduced. it can.

As described above, according to the present embodiment, not only the subtle pitch and gain fluctuations of the vibrato added by the target singer can be reproduced, but also the natural change in tone can be reproduced faithfully. . This makes it possible to add a more realistic vibrato that cannot be achieved with a conventional karaoke apparatus or the like to the singing voice of an amateur singer.
In addition, according to the present embodiment, each data obtained by analyzing the voice of the target singer subjected to vibrato is divided and held in the vibrato attack area, the vibrato body area, and the vibrato release area. Real vibrato can be added as compared with a karaoke apparatus or the like that holds only each data of the area.

Further, according to the present embodiment, even when the vibrato release region template is not used when adding the vibrato, the vibrato is gradually attenuated by using the coefficient Muldelta. This makes it possible to end natural vibrato.
Further, according to the present embodiment, a plurality of types of template sets TS obtained by analyzing voices when the target singer sings with vibrato at various pitches are stored in the vibrato database. By adopting such a configuration, when adding vibrato to the input voice, it is possible to select and use the template set TS closest to the pitch of the input voice, thereby adding realistic vibrato. It becomes possible.

  In the present embodiment, the singing voice has been described as an example, but the present invention can be applied to, for example, a normal conversation voice or a musical instrument sound. Further, since the functions of the respective units of the voice processing apparatus 100 described above are realized by a program stored in a ROM or the like, the program is recorded on a recording medium such as a CD-ROM and distributed, or the Internet or the like. You may distribute via a communication network. Of course, the function of each unit of the speech processing apparatus 100 may be realized by hardware.

It is a figure which shows the structure of the audio processing apparatus which concerns on this embodiment. It is a figure which shows the pitch waveform of the target audio | voice which applied the vibrato which concerns on the same embodiment. It is a figure which shows the pitch waveform of the VA area | region which concerns on the same embodiment. It is a figure which shows the pitch waveform of the VB area | region which concerns on the same embodiment. It is the figure which illustrated the spectrum of a certain frame with the VA field concerning the embodiment. It is a figure for demonstrating the curve showing the inclination of the spectrum shown in FIG. 5 which concerns on the same embodiment. It is a figure which shows the mode of the change of a spectrum envelope and ECurve when the magnitude | size of Gain which concerns on the embodiment is changed. It is a figure which shows the mode of the change of a spectrum envelope and ECurve when the magnitude | size of Gain which concerns on the embodiment is changed. It is the figure which illustrated input bin Magnitude concerning the embodiment. It is a figure which shows the mode of a change of the spectrum envelope when the slope of ECurve concerning the embodiment is changed. It is a figure which shows the mode of a change of the spectrum envelope when the slope of ECurve concerning the embodiment is changed. It is a flowchart which shows the vibrato addition process which concerns on the same embodiment. It is a flowchart which shows the vibrato addition process which concerns on the same embodiment. It is a figure which shows the relationship between the coefficient Muldelta which concerns on the embodiment, and vibrato time.

Explanation of symbols

DESCRIPTION OF SYMBOLS 100 ... Voice processing apparatus, 200 ... Voice input part, 300 ... Spectrum / pitch analysis part, 400 ... Vibrato addition part, 500 ... Voice output part, 600 ... Parameter input part, 700 ... Vibrato addition control unit, 800 ... Vibrato database, 900 ... Template creation unit, 950 ... Target input unit, TS ... Template set.

Claims (7)

Storage means for storing the pitch of the target voice subjected to vibrato and the inclination of the spectrum;
An input means for inputting voice;
Pitch extraction means for extracting the pitch of the input voice;
Pitch changing means for changing the pitch of the extracted voice according to the pitch of the target voice;
Spectrum slope changing means for changing the slope of the spectrum of the voice obtained by performing spectrum analysis on the inputted voice according to the slope of the spectrum of the target voice;
An audio processing apparatus comprising: output means for outputting a voice having the pitch of the voice after change and the slope of the spectrum of the voice after change. The storage means stores a gain of the target sound in addition to the pitch and spectrum inclination of the target sound,
Gain extraction means for extracting the gain of the input voice, and gain change means for changing the gain of the extracted voice according to the gain of the target voice;
2. The audio processing according to claim 1, wherein the output unit outputs a sound having a pitch of the sound after change, a slope of a spectrum of the sound after change, and a gain of the sound after change. apparatus. The storage means stores a plurality of types of pitches of the target sound subjected to vibrato and a spectrum inclination corresponding to each pitch,
The pitch changing means selects a pitch closest to the pitch of the extracted voice from a plurality of types of pitches of the target voice, and the pitch of the extracted voice according to the pitch of the selected target voice Change
The spectrum slope changing means selects a slope of the spectrum corresponding to the pitch of the target sound selected by the pitch changing means from a plurality of kinds of spectrum inclinations of the target sound, and selects the selected target sound. The speech processing apparatus according to claim 1, wherein a slope of a spectrum of the speech is changed according to a slope of the spectrum.   The pitch and spectrum inclination in each area when the target sound subjected to vibrato is divided into at least a vibrato attack area and a vibrato body area are stored in the storage means. Voice processing device. Target input means for inputting the target audio with vibrato,
Target pitch extraction means for extracting the pitch of the input target speech and storing it in the storage means;
2. A target spectrum / slope calculation unit that calculates a slope of a spectrum of the target speech by performing spectrum analysis on the input target speech and stores the spectrum in the storage unit. The speech processing apparatus according to any one of claims 3 and 4. A memorization process for memorizing the pitch and spectrum inclination of the target sound with vibrato applied;
Input process of inputting voice,
A pitch extraction process for extracting the pitch of the input voice;
A pitch changing process of changing the pitch of the extracted voice according to the pitch of the target voice;
A spectrum slope changing process for changing the slope of the spectrum of the voice obtained by performing spectrum analysis on the input voice according to the slope of the spectrum of the target voice;
An output process for outputting a sound having the pitch of the sound after the change and the slope of the spectrum of the sound after the change. Computer
Storage means for storing the pitch of the target voice subjected to vibrato and the inclination of the spectrum;
An input means for inputting voice;
Pitch extraction means for extracting the pitch of the input voice;
Pitch changing means for changing the pitch of the extracted voice according to the pitch of the target voice;
Spectrum slope changing means for changing the slope of the spectrum of the voice obtained by performing spectrum analysis on the inputted voice according to the slope of the spectrum of the target voice;
A speech processing program for functioning as output means for outputting speech having the pitch of the speech after change and the slope of the spectrum of the speech after change.

Images