JP3447220B2 - Voice conversion device and voice conversion method - Google Patents

Description

Detailed Description of the Invention

[0001]

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention relates to a voice converting apparatus and a voice converting method for converting an input voice into another voice and outputting the same, and more particularly to a voice converting apparatus and a voice converting method suitable for use in a karaoke apparatus. .

[0002]

2. Description of the Related Art Various types of voice conversion devices have been developed for changing the frequency characteristics of input voice and outputting the same. For example, some karaoke devices convert the pitch of the singing voice of a singer to a male player. There is one that converts the voice of a woman into the voice of a woman and vice versa (for example, in Japanese Patent Publication No. 8-5085).
No. 81).

[0003]

However, in the conventional voice conversion device, there is a problem in that a audible natural voice cannot be obtained because the pitch of the singing voice is simply converted. Therefore, it is an object of the present invention to provide a voice conversion device and a voice conversion method that can easily obtain a audibly natural voice when performing voice conversion.

[0004]

In order to solve the above-mentioned problems, the structure according to claim 1 has a spectral shape extracting means for extracting a spectral shape on a frequency axis from an input audio signal, and a resonance of the spectral shape. a resonance point detecting means for detecting the point, the gate of the resonance point
Wherein the reference gain <br/> level setting means for setting a reference gain level based on in, and the reference gain level spectrum
The modified spectral shape generating means for generating a modified spectral shape by changing the bandwidth of the resonance point of the spectral shape in response to the difference between the gain level of the traral shape and the modified spectral shape. A voice generating means for generating and outputting a converted voice signal based on the shape.

According to a second aspect of the present invention, there is provided a spectral shape extracting means for extracting a spectral shape on the frequency axis from an input voice signal, and the spectral shape extracting means.
A resonance point detecting means for detecting the resonance point of the shape, the reference gain level setting means for setting a reference gain level based on the gain of the resonance point, the reference gain level before
Note the difference between the spectral level and the gain level.
In response to changing the gain of the dip portion of the spectral shape, a changed spectral shape generating means for generating a changed spectral shape, and a voice generation for generating and outputting a converted audio signal based on the changed spectral shape And means are provided.

According to a third aspect of the present invention, there is provided a spectral shape extracting means for extracting a spectral shape on the frequency axis from an input audio signal, and the spectral shape extracting means.
Dip portion detecting means for detecting the dip portion of the shape, reference gain level setting means for setting a reference gain level based on the gain of the dip portion, the reference gain level and the gain level of the spectral shape.
Change peak shape of the spectral shape corresponding to the difference between the spectral shape and the modified spectral shape generating means to generate a modified spectral shape, and a converted audio signal is generated based on the modified spectral shape. , And a voice generating means for outputting.

According to a fourth aspect of the present invention, in the configuration of the first or second aspect, the reference gain level setting means has two resonances adjacent to each other on a frequency-gain axis.
It is characterized in that a straight line connecting points is defined and a curve constituted by a plurality of continuous straight lines is set as the reference gain level.

According to a fifth aspect of the present invention, in the configuration according to the first or second aspect, the reference gain level setting means has two preset values on a frequency-gain axis.
A feature is that a straight line connecting the resonance points is set as the reference gain level.

According to a sixth aspect of the present invention, in the configuration of the first or second aspect, the reference gain level setting means passes through a peak portion corresponding to a first formant frequency on a frequency-gain axis, It is characterized in that a straight line having an average slope of the spectral shape is set as the reference gain level.

According to a seventh aspect of the present invention, in the third aspect, the reference gain level setting means defines a straight line that connects two adjacent dip portions on a frequency-gain axis, and a plurality of consecutive straight lines are defined. The curve constituted by the straight line is set as the reference gain level.

According to an eighth aspect of the present invention, in the third aspect, the reference gain level setting means uses a straight line connecting the two dip portions preset on the frequency-gain axis as the reference gain level. It is characterized by setting.

According to a ninth aspect of the present invention, there is provided a spectral shape extraction step of extracting a spectral shape on the frequency axis from an input audio signal, and the spectral shape extraction step.
A resonance point detecting step of detecting the resonance point of the shape, the reference gain level setting step of setting a reference gain level based on the gain of the resonance point, the reference gain level before
Note the difference between the spectral level and the gain level.
In response to changing the bandwidth of the resonance point of the spectral shape, a modified spectral shape generation step of generating a modified spectral shape, and a voice generation step of generating a converted audio signal based on the modified spectral shape. It is characterized by having.

According to a tenth aspect of the present invention, a spectral shape extracting step of extracting a spectral shape on a frequency axis from an input audio signal, a resonance point detecting step of detecting a resonance point of the spectral shape, and the resonance point. and reference gain level setting step of setting a reference gain level based on the gain, and the reference gain level
In the difference with the gain level of the spectral shape
Correspondingly, a modified spectral shape generation step of generating a modified spectral shape by changing the gain of the dip portion of the spectral shape, and a voice generation step of generating a converted audio signal based on the modified spectral shape, It is characterized by having.

According to the eleventh aspect of the present invention, a spectral shape extracting step of extracting a spectral shape on a frequency axis from an input audio signal, a dip portion detecting step of detecting a dip portion of the spectral shape, and the dipping portion. A reference gain level setting step of setting a reference gain level based on the gain of the reference gain level, and a gain level of the reference gain level and the spectral shape.
Change spectral shape is generated by changing the gain of the peak portion of the spectral shape corresponding to the difference between the spectral shape and the spectral shape, and a converted audio signal is generated based on the modified spectral shape. And a voice generation step.

[0015]

BEST MODE FOR CARRYING OUT THE INVENTION Next, preferred embodiments of the present invention will be described with reference to the drawings. [1] Outline Process of Embodiment First, the outline process of the embodiment will be described. [1.1] Step S1 First, an SMS (Spectr) including an FFT (Fast Fourie Transform) of a singer's voice (input voice signal) in real time.
al modeling synthesis) to extract the sine wave component (Sine component) in frame units, and to extract the residual component (Resi
dual component) is generated. That the input audio signal in parallel with this, it is determined whether or not unvoiced (silence including), if it is unvoiced, without performing the processing of step S2~ Step S 5 below, which directly outputs the input audio signal Becomes In this case, pitch synchronization analysis in which the analysis window width is changed according to the pitch in the previous frame is adopted as the SMS analysis.

[1.2] Step S2 Next, when the input voice signal is a voiced sound, a spectral shape (Spectral Shape) is extracted from the extracted sine wave component.
e) is extracted. [1.3] Step S3 Next, the resonance point (formant) of the extracted spectral shape is detected, and the gain of the detected formant (further, if necessary, the average slope of the spectral shape)
Set the reference gain level based on.

[1.4] Step S4 Next, the changed spectral shape is generated by changing the bandwidth of the resonance point of the spectral shape based on the reference gain level. [1.5] Inverse SFT of the local peak information corresponding to the modified spectral shape obtained in step S5 is performed to obtain a converted speech signal. [1.6] Summary According to the converted voice signal obtained as a result of these processes, the reproduced voice is a singing voice in which the input voice is nasalized or non-nasalized.

[2] Detailed Configuration of the Embodiment FIGS. 1 and 2 show detailed configuration diagrams of the embodiment. In addition,
The present embodiment is an example of a case where the voice conversion device (voice conversion method) according to the present invention is applied to a karaoke device and configured as a karaoke device capable of more natural voice conversion. In FIG. 1, the microphone 1 collects the voice of the former singer (me) and outputs it as the input voice signal Sv to the input voice signal cutout unit 3. In parallel with this, the analysis window generation unit 2 generates an analysis window (for example, Hamming window) AW having a cycle of a fixed multiple (for example, 3.5 times) of the pitch cycle detected in the previous frame. , To the input voice signal cutout unit 3. When the initial state or the previous frame is unvoiced (including silent), the analysis window of the preset fixed cycle is output to the input audio signal cutout unit 3 as the analysis window AW.

With this, the input voice signal cutout unit 3 multiplies the input analysis window AW and the input voice signal Sv, cuts out the input voice signal Sv in frame units, and the fast Fourier transform unit 4 as a frame voice signal FSv. Is output to. More specifically, the relationship between the input audio signal Sv and the frame is as shown in FIG. 3, and each frame FL is set so as to partially overlap the previous frame FL.

The frame voice signal FSv is analyzed in the fast Fourier transform unit 4 and, as shown in FIG. 4, a local peak is detected by the peak detection unit 5 from the frequency spectrum output from the fast Fourier transform unit 4. To be detected. More specifically, a local peak marked with X is detected for the frequency spectrum as shown in FIG. This local peak is represented as a combination of a frequency value and an amplifier (amplitude) value. That is, as shown in FIG. 4, (F0, A0), (F1, A1),
A local peak is detected and represented for each frame as (F2, A2), ..., (FN, AN).

Then, as schematically shown in FIG. 3, one set (hereinafter referred to as a local peak set) for each frame is output to the unvoiced / voiced detection section 6 and the peak cooperation section 8. The unvoiced / voiced detection unit 6 detects unvoiced ('t', 'k', etc.) according to the magnitude of the high frequency component based on the input local peak for each frame, and
The voiced detection signal U / Vme is supplied to the pitch detection section 7, the easy synchronization processing section 22 and the crossfader section 3.
Output to 0. Alternatively, it is detected as unvoiced according to the number of zero crossings per unit time on the time axis ('s' etc.)
Then, the original unvoiced / voiced detection signal U / Vme is supplied to the pitch detection unit 7,
It is output to the easy synchronization processing unit 22 and the crossfader unit 30.

Furthermore, if the unvoiced / voiced voice detection unit 6 does not detect that the input frame is unvoiced, the pitch detection unit 7 outputs the input local peak set as it is.
Output to. The pitch detection unit 7 detects the pitch Pme of the frame to which the local peak set corresponds, based on the input local peak set. As a more specific method of detecting the pitch Pme of the frame, for example, Maher, RC
and J.W.Beauchamp: "Fundamental Frequency Estimation
of Musical Signal using a two-way Mismatch Proced
ure "(Journal of Acounstical Society of America95
(4): 2254-2263).
Next, the local peak set output from the peak detection unit 5 is determined by the peak cooperation unit 8 to be related to the preceding and following frames, and the local peaks that are recognized to be related are converted into a series of data peaks. Coordination processing that connects the two is performed.

Here, this cooperation process will be described with reference to FIG. Now, a local peak as shown in FIG. 5A is detected in the previous frame, and FIG.
It is assumed that a local peak as shown in (1) is detected in this frame. In this case, the peak coordinating unit 8 uses the local peaks (F0, A) detected in the previous frame.
0), (F1, A1), (F2, A2), ..., (F
It is checked whether or not a local peak corresponding to (N, AN) has been detected in this frame. Whether or not there is a corresponding local peak is determined by whether or not the local peak of this frame is detected within a predetermined range centered on the frequency of the local peak detected in the previous frame.

More specifically, in the example of FIG. 5, local peaks (F0, A0), (F1, A1), (F2, A
For 2) ..., the corresponding local peaks are detected, but for the local peaks (FK, AK) (see FIG. 5A), the corresponding local peaks (see FIG. 5).
(See (B)) has not been detected. The peak cooperation unit 8 is
When the corresponding local peaks are detected, they are connected in chronological order and output as a set of data strings. In addition,
When the corresponding local peak is not detected, it is replaced with data indicating that there is no corresponding local peak for the frame.

Here, FIG. 6 shows an example of changes in the frequency F0 and the frequency F1 of the local peak over a plurality of frames. Such a change is caused by the amplifier (amplitude) A
The same applies to 0, A1, A2, .... In this case, the data string output from the peak cooperation unit 8 is a discrete value output at every frame interval. The peak value output from the peak cooperation unit 8 will be referred to as a deterministic component hereinafter. This means a component that is definitely replaced as an element of a sine wave in the original signal (that is, the audio signal Sv). Further, each of the replaced sine waves (strictly speaking, the frequency and the amplifier (amplitude) that are parameters of the sine wave) will be referred to as a sine wave component.

Next, the interpolating / synthesizing unit 9 performs an interpolating process on the deterministic component output from the peak coordinating unit 8 and synthesizes a waveform by a so-called oscillator method based on the deterministic component after the interpolation. The interpolation interval in this case is determined by the output unit 3 described later.
4 is performed at intervals corresponding to the sampling rate of the final output signal (for example, 44.1 KHz). The solid line shown in FIG. 6 described above indicates the frequencies F0 and F1 of the sine wave components.
The image when the interpolation process is performed is shown.

[2.1] Configuration of Interpolation Synthesizing Unit Here, the configuration of the interpolation synthesizing unit 9 is shown in FIG. The interpolating / synthesizing unit 9 is configured to include a plurality of partial waveform generating units 9a, and each partial waveform generating unit 9a outputs a frequency (F0, F1, ...) And an amplifier (amplitude) of a designated sine wave component. Generates a corresponding sine wave. However, the sine wave components (F0, A0), (F1, A1), (F2,
.. A2), ..., change from time to time in accordance with the interpolation intervals, so that the waveform output from each partial waveform generator 9a becomes a waveform according to the change. That is, the sine wave component (F0, A0) from the peak cooperation unit 8
Since (F1, A1), (F2, A2), ... Are sequentially output and interpolation processing is performed for each sine wave component, each partial waveform generation unit 9a has a frequency and an amplitude within a predetermined frequency range. Outputs a waveform that fluctuates. Then, the waveforms output from the respective partial waveform generating sections 9a are added and synthesized in the adding section 9b. Therefore, the output signal of the interpolation synthesizing unit 9 becomes the sine wave component synthetic signal SSS obtained by extracting the deterministic component from the input audio signal Sv.

[2.2] Operation of Residual Component Detecting Unit Next, the residual component detecting unit 10 is a deviation between the sine wave component synthetic signal SSS output from the interpolation synthesizing unit 9 and the input voice signal Sv. A residual component signal SRD (time waveform) is generated. The residual component signal SRD contains many unvoiced components included in the voice. On the other hand, the above-mentioned sine wave component composite signal SSS corresponds to a voiced component. By the way, in order to resemble the voice of a singer as a target, only voiced sound needs to be processed, and unvoiced sound need not be processed. Therefore, in the present embodiment, the voice conversion process is performed on the deterministic component corresponding to the voiced vowel component. More specifically, for the residual component signal SRD,
The fast Fourier transform unit 11 converts the signal into a frequency waveform, and holds the obtained residual component signal (frequency waveform) in the residual component holding unit 12 as Rme (f).

[2.3] Operation of Average Amplifier Operation Unit On the other hand, as shown in FIG. 8 (A), the sine wave component (F0, A) output from the peak detection unit 5 through the peak cooperation unit 8 is output.
0), (F1, A1), (F2, A2), ..., (F (N
-1), A (N-1)) N sine wave components (hereinafter, these are collectively referred to as Fn and An. N = 0 to (N-1).)
Is held in the sine wave component holding unit 13, and the amplifier An is input to the average amplifier calculating unit 14, and the average amplifier Ame is calculated for each frame by the following equation. Ame = Σ (An) / N

[2.4] Operation of Amplifier Normalization Unit Next, in the amplifier normalization unit 15, each amplifier An is normalized by the average amplifier Ame by the following equation, and the normalized amplifier A '
Find n. A'n = An / Ame [2.5] Operation of Spectral Shape Operation Unit And in the spectral shape operation unit 16,
As shown in FIG. 8B, a spectral shape Sme (f) is an envelope having an sine wave component (Fn, A'n) obtained by the frequency Fn and the normalization amplifier A'n as a breakpoint. Generate as. In this case, the value of the amplifier at the frequency between the two breakpoints is
It is calculated by linear interpolation. The interpolation method is not limited to linear interpolation.

[2.6] Operation of Pitch Normalization Unit Subsequently, the pitch normalization unit 17 normalizes each frequency Fn by the pitch Pme detected by the pitch detection unit 7 to obtain a normalized frequency F'n. F'n = Fn / Pme As a result, the original frame information holding unit 18 causes the average amplifier Ame, the pitch Pme, and the spectral shape Sme (f) which are the original attribute data corresponding to the sine wave component included in the input audio signal Sv. ), The normalized frequency F'n is held. In this case, the normalized frequency F'n
Represents the relative value of the frequency of the overtone string, and if the overtone structure of the frame is treated as the complete overtone structure, it need not be held.

In this case, when the male / female voice conversion is going to be performed, at this stage, when the male voice → female voice conversion is performed, the pitch is raised by an octave and the female voice →
When performing male voice conversion, it is preferable to perform male voice / female voice pitch control processing for lowering the pitch by an octave.
Next, of the original attribute data held in the original frame information holding unit 18, the average amplifier Ame and the pitch Pme are further changed into the static change / vibrato change separation unit 1.
9, a filtering process or the like is performed to separate and hold the static change component and the vibrato change component. It is also possible to further separate the jitter change component, which is a higher frequency change component, from the vibrato change component.

More specifically, the average amplifier Ame is separated into an average amplifier static component Ame-sta and an average amplifier vibrato component Ame-vib and held. Further, the pitch Pme is changed to the pitch static component Pme-sta and the pitch vibrato component Pme.
-Separate and keep vib. As a result, the original frame information data INFme of the corresponding frame is shown in FIG.
As shown in FIG. 5, the average amplifier static component Ame-sta, which is the original attribute data corresponding to the sine wave component of the input audio signal Sv, the average amplifier vibrato component Ame-vib, and the pitch static component P.
Me-sta, a pitch vibrato component Pme-vib, a spectral shape Sme (f), a normalized frequency F'n, and a residual component Rme (f) are held.

On the other hand, the target frame information data INFtar composed of target attribute data corresponding to a singer, which is a target of imitation, is previously analyzed and held in advance in a hard disk or the like which constitutes the target frame information holding unit 20. Has been done. In this case, of the target frame information data INFtar,
As the target attribute data corresponding to the sine wave component,
Average amp static component Atar-sta, average amp vibrato component Atar-vib, pitch static component Ptar-sta, pitch vibrato component Ptar-vib, spectral shape St
There is ar (f). Also, the target frame information data I
Among the NFtar, the target component attribute data corresponding to the residual component is the residual component Rtar (f).

[2.7] Operation of Key Control / Tempo Change Unit Next, the key control / tempo change unit 21 responds to the sync signal SSYNC from the target frame information holding unit 20 based on the sync signal SSYNC from the sequencer 31. Target frame information INFtar of the frame
Read-out processing and correction processing of the target attribute data constituting the read-out target frame information data INFtar, and the read target frame information INFtar and a target unvoiced / voiced detection signal indicating whether the frame is unvoiced or voiced. Output U / V tar. More specifically, a key control unit (not shown) of the key control / tempo change unit 21 is provided when the keys of the karaoke device are raised or lowered from the reference,
Pitch static component Ptar-sta which is target attribute data
Also, for the pitch vibrato-like component Ptar-vib, the correction processing for raising and lowering the same amount is performed. For example, 50 [ce
nt], the pitch static component Ptar-
5 for sta and pitch vibrato-like component Ptar-vib
You have to raise it by 0 [cent].

When the tempo of the karaoke apparatus is raised or lowered, the tempo change unit (not shown) of the key control / tempo change unit 21 reads the target frame information data INFtar at the timing corresponding to the changed tempo. There is a need to do. In this case, if the target frame information data INFtar corresponding to the timing corresponding to the required frame does not exist, the target frame information data INFtar of two frames existing before and after the required frame timing is read. Interpolation processing is performed using these two pieces of target frame information data INFtar to generate target frame information data INFtar of the frame at the necessary timing, and thus target attribute data.

In this case, the vibrato-like components (average amp vibrato-like component Atar-vib and pitch vibrato-like component Ptar-vib) are unsuitable because the vibrato period itself changes as they are. It is necessary to perform interpolation processing so that the cycle does not change. Alternatively, as the target attribute data, not the data representing the locus of the vibrato itself but the parameters of the vibrato period and the vibrato depth are held and the actual locus is obtained by calculation, so that this problem can be avoided.

[2.8] Operation of Easy Synchronization Processing Unit Next, the easy synchronization processing unit 22 adds the original frame information data INFme to the frame (hereinafter referred to as the original frame) of the singer who imitates. The frame of the singer that is the target of the corresponding imitation (hereinafter referred to as the target frame) despite the existence of the.
If the target frame information data INFtar does not exist in the target frame, the target frame information data INFta of the frame existing in the front-back direction of the target frame.
An easy synchronization process is performed in which r is the target frame information data INFtar of the target frame.

Then, the easy synchronization processing unit 22 selects target attribute data (average amplifier static component Atar-sync-Amp-sync-Atar-sync-Amp) from the target attribute data included in the replaced target frame information data INF tar-sync, which will be described later. sta, average amp vibrato component Atar-sync-vib, pitch static component Ptar-sync-st
a, the pitch vibrato component Ptar-sync-vib and the spectral shape Star-sync (f)) are output to the sine wave component attribute data selection unit 23.

Further, the easy synchronization processing unit 22 selects the target attribute data (residual component Rtar-sync (f)) related to the residual component of the target attribute data included in the replaced target frame information data INF tar-sync which will be described later. ) Is output to the residual component selection unit 25. Also in the processing in the easy synchronization processing section 22, vibrato-like components (average amp vibrato-like component Atar-vib and pitch vibrato-like component Ptar are included.
With regard to -vib), the vibrato cycle itself changes as it is, which is inappropriate, so it is necessary to perform interpolation processing so that the cycle does not change. Alternatively, as the target attribute data, not the data representing the locus of the vibrato itself but the parameters of the vibrato period and the vibrato depth are held and the actual locus is obtained by calculation, so that this problem can be avoided.

[2.8.1] Details of Easy Synchronization Process Here, the easy synchronization process will be described in detail with reference to FIGS. 9 and 10. FIG. 9 is a timing chart of the easy synchronization processing, and FIG. 10 is a flow chart of the easy synchronization processing. First, the easy synchronization processing unit 22 sets synchronization mode = “0” indicating the method of synchronization processing (step S11). This synchronization mode = “0” corresponds to the case of normal processing in which the target frame information data INFtar is present in the target frame corresponding to the original frame.

Then, at a certain timing t, the former unvoiced /
Voiced detection signal U / Vme (t) changes from unvoiced (U) to voiced (V)
It is determined whether or not it has changed to (step S12). For example, as shown in FIG. 9, at the timing t = t1, the original unvoiced / voiced detection signal U / Vme (t) changes from unvoiced (U) to voiced (V). In the determination of step S12, when the original unvoiced / voiced detection signal U / Vme (t) changes from unvoiced (U) to voiced (V) (step S12; Yes), the previous timing t of the timing t Original unvoiced / voiced detection signal U / Vme (t-1) at -1 is unvoiced (U) and target unvoiced / voiced detection signal U / Vtar
It is determined whether (t-1) is unvoiced (U) (step S18). For example, as shown in FIG. 9, timing t =
At t0 (= t1-1), the original unvoiced / voiced detection signal U /
Vme (t-1) is unvoiced (U) and the target unvoiced / voiced detection signal U / Vtar (t-1) is unvoiced (U). In the determination of step S18, the original unvoiced / voiced detection signal U / V
If me (t-1) is unvoiced (U) and the target unvoiced / voiced detection signal U / Vtar (t-1) is unvoiced (U) (step S18; Yes), the target frame is , Since the target frame information data INFtar does not exist, the synchronization mode is set to “1” and the target frame information data INFhold for replacement is set.
Is the target frame information of the frame existing in the backward direction (Backward) of the target frame.

For example, as shown in FIG. 9, timing t
Since the target frame information data INFtar does not exist in the target frames of t1 to t2, the synchronization mode is set to “1”, and the replacement target frame information data INFhold is present in the rearward direction of the target frame (that is, , Target frame information data backward at the timing t = t2 to t3). Then, the process proceeds to step S15, and the synchronization mode =
It is determined whether it is "0" (step S15). When it is determined in step S15 that the synchronization mode = “0”, the target frame information data INFtar (t) exists in the target frame corresponding to the original frame at the timing t, that is, the normal process. Therefore, the replaced target frame information data INF tar-sync is set as the target frame information data INF tar (t). INFtar-sync = INFtar (t)

For example, as shown in FIG. 9, timing t =
Since the target frame information data INFtar exists in the target frame from t2 to t3, INFtar-sync = INFtar (t). In this case, the target attribute data (average amplifier static component Ata) included in the replaced target frame information data INF tar-sync used for the subsequent processing.
r-sync-sta, average amp vibrato component Atar-sync-v
ib, Pitch static component Ptar-sync-sta, Pitch vibrato component Ptar-sync-vib, Spectral shape Star-
The sync (f) and the residual component Rtar-sync (f)) have substantially the following contents (step S16). Atar-sync-sta = Atar-sta Atar-sync-vib = Atar-vib Ptar-sync-sta = Ptar-sta Ptar-sync-vib = Ptar-vib Star-sync (f) = Star (f) Rtar-sync (f) = Rtar (f)

When it is determined in step S15 that the synchronization mode = "1" or the synchronization mode = "2", the target frame information data INFtar (t) is added to the target frame corresponding to the original frame at the timing t. ) Does not exist, the replaced target frame information data I
Target frame information data I for replacing NF tar-sync
NFhold. INFtar-sync = INFhold For example, as shown in FIG. 9, target frame information data INFtar does not exist in the target frame at timing t = t1 to t2, and the synchronization mode = “1”, but the timing t = Target frame information data IN in the target frame from t2 to t3
Since Ftar exists, processing P1 is performed in which the replaced target frame information data INFtar-sync is used as the replacement target frame information data INFhold which is the target frame information data of the target frame at the timing t = t2 to t3. The target attribute data included in the replaced target frame information data INF tar-sync used is the average amplifier static component Atar-sync-s.
ta, average amp vibrato component A tar-sync-vib, pitch static component P tar-sync-sta, pitch vibrato component P
tar-sync-vib, Spectral Shape Star-sync (f)
And the residual component R tar-sync (f) (step S1)
6).

Further, as shown in FIG. 9, timing t =
The target frame information data INFtar does not exist in the target frame from t3 to t4, and the synchronization mode is "2", but the timing t = t2 to
Since the target frame information data INFtar exists in the target frame at t3, the replaced target frame information data INFtar-sync is timing t = t.
Replacement target frame information data IN which is the target frame information data of the target frame of 2 to t3
The target attribute data included in the replaced target frame information data INF tar-sync used for the subsequent processing after the processing P2 for Fhold is the average amplifier static component Atar-sync-sta and the average amplifier vibrato component Atar-sy.
nc-vib, Pitch static component Ptar-sync-sta, Pitch vibrato component Ptar-sync-vib, Spectral shape S
The tar-sync (f) and the residual component R tar-sync (f) are obtained (step S16).

In the determination of step S12, the original unvoiced /
Voiced detection signal U / Vme (t) changes from unvoiced (U) to voiced (V)
If it has not changed to (step S12; No),
It is determined whether or not the target unvoiced / voiced detection signal U / Vtar (t) changes from voiced (V) to unvoiced (U) (step S13). In the determination of step S13,
When the target unvoiced / voiced detection signal U / Vtar (t) changes from voiced (V) to unvoiced (U) (step S13; Yes), the original unvoiced at the timing t-1 at the previous timing t-1 Voiced detection signal U / Vme (t-1) is voiced (V) and target unvoiced / voiced detection signal U / Vtar
It is determined whether or not (t-1) is voiced (V) (step S19).

For example, as shown in FIG. 9, timing t
Target unvoiced / voiced detection signal U / V tar (t) in 3
Changes from voiced (V) to unvoiced (U), timing t-1
= T2 to t3, the original unvoiced / voiced detection signal U / Vme
(t-1) is voiced (V) and the target unvoiced / voiced detection signal U / Vtar (t-1) is voiced (U). Step S
The original unvoiced / voiced detection signal U / Vme (t
-1) is voiced (V) and target unvoiced / voiced detection signal U
When / Vtar (t-1) is voiced (V) (step S19; Yes), the target frame is
Since the target frame information data INFtar does not exist, the synchronization mode = “2” is set, and the replacement target frame information data INFhold is set as the target frame information of the frame existing in the forward direction of the target frame.

For example, as shown in FIG. 9, timing t
Since the target frame information data INFtar does not exist in the target frame of = t3 to t4, the synchronization mode is set to "2", and the replacement target frame information data INFhold is present in the front direction of the target frame (that is, , Target frame information data forward at the timing t = t2 to t3). Then, the process proceeds to step S15, and the synchronization mode =
It is determined whether or not it is "0" (step S15), and thereafter, the same processing is performed. In the determination in step S13, if the target unvoiced / voiced detection signal U / Vtar (t) has not changed from voiced (V) to unvoiced (U) (step S13; No), the original unvoiced / voiced at timing t The detection signal U / Vme (t) changes from voiced (V) to unvoiced (U), or the target unvoiced / voiced detection signal U /
It is determined whether or not Vtar (t) has changed from unvoiced (U) to voiced (V) (step S14).

In the determination in step S14, the original unvoiced / voiced detection signal U / Vme (t) at the timing t changes from voiced (V) to unvoiced (U), or the target unvoiced / voiced detection signal U / Vtar ( When t) changes from unvoiced (U) to voiced (V) (step S14; Ye)
s), the synchronization mode = “0”, the replacement target frame information data INFhold is initialized (cleared), the process proceeds to step S15, and the same process is performed thereafter. In the determination of step S14, the original unvoiced / voiced detection signal U / V at the timing t
me (t) does not change from voiced (V) to unvoiced (U), and
If the target unvoiced / voiced detection signal U / Vtar (t) has not changed from unvoiced (U) to voiced (V) (step S14; No), the process directly shifts to step S15, and so on. I do.

[2.9] Operation of Modified Spectral Shape Generation Unit Subsequently, the modified spectral shape generation unit 23 corresponds to the sine wave component of the input audio signal Sv input from the static change / vibrato change change separation unit 19. The original attribute data is the average amp static component Ame-sta, the average amp vibrato component Ame-vib, the pitch static component Pme-sta, the pitch vibrato component Pme-vib, and the spectral shape Sme.
(f), normalized frequency F'n, target attribute data regarding the sine wave component of the target attribute data included in the replaced target frame information data INF tar-sync input from the easy synchronization unit 22 (average amplifier static Component Atar-sync-sta, average amp vibrato component Atar-sync-vib, pitch static component Ptar-sy
nc-sta, a pitch vibrato component Ptar-sync-vib and a spectral shape Star-sync (f)), and a modified spectral shape which is a new spectral shape based on the modified spectral shape generation information input from the controller 29. A shape Snew (f) is generated. The modified spectral shape is generated by changing the gain of the dip part of the original spectral shape or by changing the gain of the resonance point (formant) which is the peak part of the original spectral shape.

[2.9.1] Resonance Point Detection Method Here, prior to the description of the method of generating the modified spectral shape Snew (f), the resonance point (peak value; formant frequency) is extracted from the original spectral shape. The detection method will be described with reference to FIG. Various methods have been proposed for detecting resonance points, including methods that use linear prediction methods, and it is possible to use these methods together, but in the following explanation, the results of the original spectral shape extracted An example of the method for detecting the resonance point from the above will be described.
In this case, from the result of the original spectral shape, the data of the following conditions are excluded from the calculation target from the viewpoint of reducing the calculation amount and speeding up the calculation time. In addition, the number of resonance points to be detected is three, and the maximum value of the detection frequency (search end frequency) is 3500 [Hz] (= the third frequency).
Set as a band sufficient to detect formants). 1) Threshold level set in advance (for example, data of the input voice signal having a signal level of -60 [dB] or less 2) Frequency data other than frequency data having an integral multiple of the fundamental frequency (F0) 3) Basic Frequency data with frequencies below the frequency (F0)

Next, prepare the following working parameters (working parameters), to detect the resonance point along the flowchart of FIG. 11. -Max.frq: Frequency of resonance point having maximum gain-Max Gain: Gain corresponding to resonance point having maximum gain-Min.frq: Frequency corresponding to resonance point having minimum gain-Min Gain: Having minimum gain Gain corresponding to resonance point ・ Peak [N] .frq: Frequency storage area of detected resonance point (N = 0 to 2) ・ Peak [N] .gain: Gain storage area of detected resonance point (N = 0 to 0) 2) ・ DiffGain: Reference gain difference between peak (peak) and dip (valley) ・ EndFrq: Search end frequency ・ PeakCnt: Counter of detected resonance points

First, the following processing is performed as the overall initialization (step S1 ). DiffGain = 15 [dB] EndFrq = 3500 [Hz] Then, the following processing is performed as the search initial setting (step S2 ). PeakCnt = 0 Max Gain = -999 [dB] Min Gain = 0 [dB]

Subsequently, the data corresponding to the original spectral shape is searched in order from the fundamental frequency F0 toward the high frequency side, and the work parameters Max.frq, Max G.
ain, Min.frq, and Min Gain are updated based on the search result (step S3 ). Further, in parallel with this updating process, it is determined whether or not the search target frequency exceeds the search end frequency corresponding to the work parameter EndFrq (step S4 ).
In the determination of step S4 , if the search target frequency exceeds the search end frequency corresponding to the work parameter EndFrq and does not exceed the search end frequency corresponding to the work parameter EndFrq (step S4 ; No), the search is continued. In the determination of step S4, if the search target frequency exceeds the search end frequencies corresponding to the work parameters EndFrq (Step S4; Yes), it determines whether or not the value of the work parameter PeakCnt is less than 3 ( Step S5 ).

If it is determined in step S5 that the value of the work parameter PeakCnt is less than 3, (step
S5 : Yes), the reference gain difference between the peak portion (mountain portion) and the dip portion (valley portion) is reduced in order to broaden the search target (step S6 ), and the process proceeds to step S2 again.
Perform search initialization again. More specifically, DiffGain = DiffGain−2, the reference gain difference is reduced by 2 [dB], the process proceeds to step S2 again, and the search initialization is performed again.

In the determination of step S5 , if the value of the work parameter PeakCnt is 3 or more (step
S5; No), the work parameters Max Gain of comparing the values of the work parameters Min Gain, reference gain difference Not that the gain difference is set to work parameter DiffGain
It is determined whether or not it is full (step S7 ), and if the gain difference is less than the reference gain difference (step S7 ; Ye).
s), the process proceeds to step S3 , and the search process is continued.

If it is determined in step S7 that the gain difference is equal to or larger than the reference gain difference (step S7 ; N
o), the value of the work parameter Min.frq is compared with the value of the work parameter Max.frq to determine whether the frequency corresponding to the work parameter Min.frq is higher than the frequency of the work parameter Max.frq ( Step S8 ). Step
When the frequency corresponding to the work parameter Min.frq is equal to or lower than the frequency of the work parameter Max.frq in the determination of S8 (step S8 ; No), Min Gain = 0 (step S9 ), and the process proceeds to step S3 . Migrate again.

In the determination in step S8 , the frequency corresponding to the work parameter Min.frq is the work parameter Ma.
If it is higher than the frequency of x.frq (step S8 ; Y
es), the following processing is performed in order (step S10 ),
The processing shifts to step S3 again. Peak [PeakCnt] .frq = Max.frq Peak [PeakCnt] .gain = Max Gain PeakCnt = PeakCnt + 1 Max Gain = −999 Min Gain = 0 [dB]

In this way, the frequencies and gains of the peak parts corresponding to the first resonance point (first formant) to the third resonance point (third formant) are obtained. In the above description, the search end frequency is set to 3500 [Hz], and detection is performed up to the third resonance point. However, if a frequency band in which at least two resonance points including the fundamental frequency can be detected is set, the search ends. The frequency can be set arbitrarily.

[2.9.2] Method of Generating Modified Spectral Shape Next, referring to FIG. 12, modified spectral shape S
Generation of new (f) is explained. Two adjacent resonance points P (n) and P detected in the above resonance point detection processing
A straight line connecting (n + 1) on the frequency logarithmic axis-gain logarithmic axis is defined as a reference gain level straight line LREF, and the gain value at the frequency f of the reference gain level straight line LREF is defined as a reference gain level PK (f). Further, the gain value at the frequency f of the original spectral shape is PS (f).

Here, the resonance points P (n) and P (n + 1)
Let fPn and fPn1 be the frequencies corresponding to, respectively, within the range of frequency f of fPn ≦ f <fPn1, the modified spectral shape Snew
The gain PD (f) of (f) is determined by the following equation. PD (f) = PK (f) + K · (PS (f) −PK
(F) Here, K is a nasal sound ratio parameter given by the controller. When K <1, the input voice is made nasal, and when K> 1, the input voice is made non-nasal.

More specifically, as shown by the broken line a in FIG. 12 , the depth of the dip portion is compared with the waveform (represented by the solid line) of the spectral shape (original spectral shape) of the input voice. By making it shallow, the bandwidth of the resonance point is effectively widened and nasalization is performed. That is, it is possible to convert the voice into a voice with a stuffy nose.
Further, as shown by a broken line b in FIG. 12 ,
By increasing the depth, the bandwidth of the resonance point is effectively narrowed and denasification is performed. That is, it is possible to convert to a clearer voice.

[2.10] Operation of Deformed Spectral Shape Processing Section Generally, when the amplifier component is large, a bright sound that extends to the high frequency range is produced, and when the amplifier component is small, the sound is muffled. Makes a sound. Therefore, regarding the modified spectral shape Snew (f), in order to simulate such a state, as shown in FIG. 13 , the high-frequency component of the spectral shape, that is, the slope of the spectral shape of the high-frequency component part A new amplifier component Anew
Spectral tilt compensation (sp
By performing ectral tilt correction) and controlling it, more realistic sound can be reproduced.

Subsequently, the generated modified spectral shape Snew (f) is controller 2 if necessary.
Based on the modified spectral / shape processing information input from 9, the modified spectral / shape processing unit 24 further processes the waveform. For example, waveform processing such as extending the modified spectral shape Snew (f) as a whole is performed. [2.11] Operation of Residual Component Selection Unit On the other hand, the residual component selection unit 25 selects the target attribute data included in the replaced target frame information data INF tar-sync input from the easy synchronization processing unit 22. Target attribute data regarding the residual component (residual component Rtar-sync (f)), residual component signal (frequency waveform) Rme (f) held in the residual component holding unit 12, and residual input from the controller 29. A new residual component Rnew (f) which is new residual component attribute data is generated based on the difference component attribute data selection information.

That is, the new residual component Rnew (f) is generated by the following equation. Rnew (f) = R * (f) (where * is me or tar-sync) In this case, whether to select me or tar-sync is the same as the new spectral shape Snew (f) More preferably, one selected from Further, as for the new residual component Rnew (f), in order to simulate a state similar to that of the new spectral shape, as shown in FIG. 13, the high frequency component of the residual component, that is, the high frequency component part, Spectral tilt compensation that compensates the slope of the residual component according to the magnitude of the new amplifier component Anew (spectral tilt corr
You can play more realistic sound by controlling and controlling the section.

[2.12] Operation of Sine Wave Component Generation Unit Subsequently, the sine wave component generation unit 26 changes the shape of the modified spectral
A new sine wave component (F ″ 0, A ″ 0) in the frame is output based on the modified spectral shape Snew (f) that is output from the shape processing unit 24 without waveform processing or with waveform processing. (F "1, A" 1),
(F "2, A" 2), ..., (F "(N-1), A" (N-1))
N sine wave components of (hereinafter, these are collectively referred to as F ″ n,
Notated as A ″ n, n = 0 to (N−1). [2.13] Operation of inverse fast Fourier transform unit

Next, the inverse fast Fourier transform unit 28 stores the obtained new frequency F ″ ′ n, new amplifier A ″ ′ n (= new sine wave component) and new residual component Rnew (f) in the FFT buffer. , The inverse FFT is sequentially performed, and the obtained time axis signals are overlapped so as to partially overlap,
By performing addition processing for adding them, a converted voice signal which is a time axis signal of a new voiced sound is generated. At this time, the mixing ratio of the sine wave component and the residual component is controlled based on the sine wave component / residual component balance control signal input from the controller 29 to obtain a more realistic voiced signal. In this case, generally, a rough voice can be obtained by increasing the mixing ratio of the residual components. In this case, when the new frequency F ″ n and the new amplifier A ″ n (= new sine wave component) and the new residual component Rnew (f) are stored in the FFT buffer, they are converted with different pitches and appropriate pitches. A harmony can be obtained as a converted audio signal by further adding a sine wave component. Furthermore, by giving a harmony pitch suitable for the accompaniment sound by the sequencer 31, a musical harmony suitable for the accompaniment can be obtained.

[2.14] Operation of Crossfader Next, the crossfader 30 detects the original unvoiced / voiced detection signal U /
If the input voice signal Sv is unvoiced (U) based on Vme (t), the input voice signal Sv is output to the mixer 30 as it is. When the input audio signal Sv is voiced (V), the converted audio signal output by the inverse FFT converter 28 is output to the mixer 30. In this case, the reason why the crossfader 30 is used as the changeover switch is to prevent the generation of a click sound when the switch is changed by performing the crossfade operation.

[2.15] Operation of Sequencer, Sound Source Section, Mixer, and Output Section On the other hand, the sequencer 31 outputs sound source control information for generating an accompaniment sound of karaoke, for example, MIDI (Musical Inst).
rument Digital Interface) Data source etc. 3
Output to 2. As a result, the sound source section 32 generates an accompaniment signal based on the sound source control information and outputs it to the mixer 33.
The mixer 33 mixes either the input audio signal Sv or the converted audio signal and the accompaniment signal, and outputs the mixed signal to the output unit 34. The output unit 34 has an amplifier (not shown) and amplifies the mixed signal and outputs it as an acoustic signal.

[3] Modification of Embodiment [3.1] First Modification In the description of the above embodiment, all resonance points are detected in the search target frequency range.
Even if all resonance points are not detected as shown in FIG. 14 (resonance point P1.5 is not detected in FIG. 14),
By performing the same processing as that of the embodiment, it is possible to obtain substantially the same effect.

[3.2] Second Modification In the description of the above embodiment, the reference gain level was set based on the straight line connecting two adjacent resonance points, but as shown in FIG. , The resonance point having the maximum gain corresponding to the fundamental frequency and the average slope of the spectral shape are calculated, and a straight line passing the maximum gain value and having the average slope of the spectral shape is set as the reference gain level line LREF, and the same processing is performed. It is also possible to do so.

[3.3] Third Modification In the above description of the embodiment, the straight line connecting the resonance points (peak portions) was used as the reference gain level straight line, but similarly, as shown in FIG. It is also possible to use a straight line connecting the portions (valley portions) as the reference gain level straight line LREF and change the frequency band of the resonance point by raising or lowering the height of the peak portion (for example, P1).

[4] As a result of the above effects, when the song of the original singer is output along with the accompaniment of karaoke, the singing voice of the original singer is made nasalized or nonnasalized in terms of hearing. It will be.

[0075]

As described above, according to the present invention, it is possible to easily obtain a nasalized converted voice or a non-nasalized converted voice that is natural to the sense of hearing.

[Brief description of drawings]

FIG. 1 is a block diagram (No. 1) showing a configuration of an embodiment of the present invention.

FIG. 2 is a block diagram (part 2) showing the configuration of an embodiment of the present invention.

FIG. 3 is a diagram showing a state of a frame in the embodiment.

FIG. 4 is an explanatory diagram for explaining peak detection of a frequency spectrum in the embodiment.

FIG. 5 is a diagram showing cooperation of peak values for each frame in the embodiment.

FIG. 6 is a diagram showing how the frequency value changes in the embodiment.

FIG. 7 is an explanatory diagram of signal processing according to the embodiment.

FIG. 8 is an explanatory diagram of signal processing according to the embodiment.

FIG. 9 is a timing chart of an easy synchronization process.

FIG. 10 is a flowchart of an easy synchronization process.

FIG. 11 is a resonance point detection processing flowchart.

FIG. 12 is a diagram illustrating the operation of the embodiment.

FIG. 13 is a diagram illustrating spectral tilt compensation of a spectral shape.

FIG. 14 is an explanatory diagram of a first modified example.

FIG. 15 is an explanatory diagram of a second modified example.

FIG. 16 is an explanatory diagram of a third modified example.

[Explanation of symbols]

DESCRIPTION OF SYMBOLS 1 ... Microphone, 2 ... Analysis window generation part, 3 ... Input audio signal cutout part, 4 ... Fast Fourier transform part, 5 ... Peak detection part, 6 ...
Unvoiced / voiced detection unit, 7 ... Pitch extraction unit, 8 ... Peak cooperation unit, 9 ... Interpolation synthesis unit, 10 ... Residual component detection unit, 11 ... Fast Fourier transform unit, 12 ... Residual component holding unit, 13 ... Sine Wave component holding unit, 14 ... Average amplifier calculation unit, 15 ... Amplifier normalization unit, 16 ... Spectral shape calculation unit, 17
... pitch normalization section, 18 ... original frame information holding section, 19
… Static change / vibrato-like change separation unit, 23… Modified spectral shape generation unit, 24… Modified spectral
Shape processing unit, 26 ... Sine wave component generation unit, 27 ... Sine wave component transformation unit, 28 ... Inverse fast Fourier transform unit, 29 ... Controller, 30 ... Crossfader unit, 31 ... Sequencer, 32 ... Sound source unit, 33 ... Mixer , 34 ... Output unit

─────────────────────────────────────────────────── ─── Continuation of the front page (72) Inventor Xavier Serra Spain Barcelona Cardedue 08440 2-2 Vizcaia 19 (56) References JP-A-1-93796 (JP, A) JP-A-6-208385 (JP, A) JP-A-6-110499 (JP, A) JP-A-4-9900 (JP, A) JP-A-9-330099 (JP, A) JP-A-6-175691 (JP, A) JP-A-10-62460 (JP, A) (58) Fields surveyed (Int.Cl. ⁷ , DB name) G10L 21/04 G10K 15/04 302

Claims (11)

(57) [Claims] 1. A spectral shape extraction means for extracting a spectral shape on a frequency axis from an input audio signal, a resonance point detection means for detecting a resonance point of the spectral shape, and a reference based on a gain of the resonance point. Reference gain level setting means for setting a gain level, and the reference gain level and the spectral shape
A modified spectral shape generating means for generating a modified spectral shape by changing the bandwidth of the resonance point of the spectral shape corresponding to the difference with the gain level, and a converted audio signal based on the modified spectral shape. A voice conversion device comprising: a voice generation unit that generates and outputs. 2. A spectral shape extracting means for extracting a spectral shape on a frequency axis from an input audio signal, a resonance point detecting means for detecting a resonance point of the spectral shape, and a reference based on a gain of the resonance point. Reference gain level setting means for setting a gain level, and the reference gain level and the spectral shape
A modified spectral shape generating unit that generates a modified spectral shape by changing the gain of the dip portion of the spectral shape in accordance with the difference from the gain level, and a converted audio signal based on the modified spectral shape A voice conversion device comprising: 3. A spectral shape extracting means for extracting a spectral shape on a frequency axis from an input audio signal, a dip portion detecting means for detecting a dip portion of the spectral shape, and a reference based on a gain of the dip portion. Reference gain level setting means for setting a gain level, and the reference gain level and the spectral shape
A modified spectral shape generating means for generating a modified spectral shape by changing the gain of the peak portion of the spectral shape in accordance with the difference from the gain level, and a converted audio signal based on the modified spectral shape. A voice conversion device comprising: 4. The voice conversion device according to claim 1, wherein the reference gain level setting means defines a straight line connecting two adjacent resonance points on a frequency-gain axis, and a plurality of consecutive straight lines are defined. A voice conversion device, wherein a curve constituted by the straight line is set as the reference gain level. 5. The voice conversion device according to claim 1, wherein the reference gain level setting means sets a straight line connecting two resonance points set in advance on a frequency-gain axis to the reference line. > A voice conversion device characterized by being set as a gain level. 6. The voice conversion device according to claim 1, wherein the reference gain level setting means passes through a peak portion corresponding to a first formant frequency on a frequency-gain axis,
A voice conversion device, wherein a straight line having an average slope of the spectral shape is set as the reference gain level. 7. The voice conversion device according to claim 3, wherein the reference gain level setting means defines a straight line connecting two adjacent dip portions on a frequency-gain axis,
A voice conversion device, wherein a curve constituted by a plurality of continuous straight lines is set as the reference gain level. 8. The voice conversion device according to claim 3, wherein the reference gain level setting means forms a straight line connecting the two dip portions set in advance on a frequency-gain axis.
A voice conversion device characterized by being set as a reference gain level. 9. A spectral shape extracting step of extracting a spectral shape on a frequency axis from an input audio signal, a resonance point detecting step of detecting a resonance point of the spectral shape, and a reference based on a gain of the resonance point. A reference gain level setting step of setting a gain level, and the reference gain level and the spectral shape
A modified spectral shape generation step of generating a modified spectral shape by changing the bandwidth of the resonance point of the spectral shape corresponding to the difference with the gain level, and a converted voice signal based on the modified spectral shape.
Speech conversion method of the speech production process, comprising the generating the issue. 10. A spectral shape extracting step of extracting a spectral shape on a frequency axis from an input audio signal, a resonance point detecting step of detecting a resonance point of the spectral shape, and a reference based on a gain of the resonance point. A reference gain level setting step of setting a gain level, and the reference gain level and the spectral shape
A modified spectral shape generation step of generating a modified spectral shape by changing the gain of the dip portion of the spectral shape corresponding to the difference with the gain level, and a converted audio signal generated based on the modified spectral shape. A voice conversion method, comprising: 11. A spectral shape extracting step of extracting a spectral shape on a frequency axis from an input audio signal, a dip portion detecting step of detecting a dip portion of the spectral shape, and a reference based on a gain of the dip portion. A reference gain level setting step of setting a gain level, and the reference gain level and the spectral shape
A modified spectral shape generation step of generating a modified spectral shape by changing the gain of the peak portion of the spectral shape corresponding to the difference with the gain level, and a converted audio signal generated based on the modified spectral shape. A voice conversion method, comprising: