JP2005107315A - Apparatus, method, and program for musical sound processing - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To provide an apparatus, a method, and a program for musical sound processing that imparts natural unison singing effect. <P>SOLUTION: A pseudo-random signal generation unit 500 comprises a plurality of pseudo-random signal generation parts 510. The respective pseudo-random signal generation parts 510 generate pseudo-random signals differing in way of varying etc., and supply those pseudo-random signals to corresponding clone signal generation parts 410. A pseudo-random signal generated by each pseudo-random signal generation part 510 has its white noise processed by an LPF. The cutoff frequency of the LPF is set to a value matching natural ways of variation and fluctuation when a person sings most (in concrete, about 2 Hz). Each clone signal generation part 410 uses the pseudo-random signal as a modulation signal to perform voice conversion. <P>COPYRIGHT: (C)2005,JPO&NCIPI

Description

  The present invention relates to a musical sound processing apparatus, and more particularly, to a musical sound processing apparatus, a musical sound processing method, and a musical sound processing program suitable for obtaining an effect (singing effect) in which a plurality of people sing the same melody.

  Devices that give a chorus effect (the effect of changing the sound of a single sound source so that multiple sound sources are playing simultaneously) to the input musical sound are widely known. Has been used to get. As an apparatus for giving such a chorus effect, Patent Document 1 below discloses that an input musical sound signal is divided into three parts, a low-frequency component, a mid-frequency component, and a high-frequency component, and each band-divided signal component is divided. An apparatus is disclosed that performs different modulation processing (processing for giving periodic pitch changes, delays, etc.) and mixes these with the input musical sound signal to give a chorus effect. Patent Document 2 below discloses a singing voice synthesizing apparatus that extracts pitch, volume, sound output timing, and the like from musical score information stored in advance in a memory or the like, and performs a modulation process on these to synthesize a chorus voice. Has been. Further, in Patent Document 3 below, when an input audio signal is converted through a key changing circuit, a filter, and a reverberation adding circuit, these parameters are changed by a fluctuation controller, which is different from the input audio signal. A chorus effect device that generates a voice effect and obtains a chorus effect by synthesizing them is disclosed.

JP 2003-122361 A JP-A-7-146695 Japanese Patent Laid-Open No. 9-281966

  However, in the modulation processing disclosed in each of the above patent documents, a modulation signal such as a triangular wave generated by an LFO (Low Frequency Oscillator) is used, so the change is monotonous and unnatural. The effect that multiple people are singing was not obtained. This is because when multiple people actually sing, the voice quality, singing method, pitch shift, etc. are subtly different for each person, and this subtle shift creates a lustrous sound gloss and fluctuation. Because.

  The present invention has been made in view of the above-described circumstances, and an object thereof is to provide a musical sound processing device, a musical sound processing method, and a musical sound processing control program that can impart a natural sung effect or the like.

  In order to solve the above-described problem, the musical sound processing apparatus according to the present invention removes a signal of a specific frequency component from the noise signal according to a noise generation unit that generates a noise signal and a set cutoff frequency, and generates a pseudo-random signal. Filter means for outputting, and modulation means for modulating the input musical sound based on the pseudo-random signal.

  According to such a configuration, a pseudo-random signal from which a signal of a specific frequency component is removed is used as a modulation signal for modulating an input musical sound. By using such a pseudo-random signal as a modulation signal, a more natural chorus effect or the like can be imparted as compared with the case where a triangular wave or the like is used as a modulation signal.

  Here, in the above configuration, it is desirable that the cutoff frequency setting unit is a unit that sets the cutoff frequency and changes the cutoff frequency within a certain frequency range. In this way, the cutoff frequency is not fixed, but is oscillated within a certain frequency range (for example, around 2 Hz), so that it is possible to match the natural change when the person sings, and how to sway. .

Further, in the above-described configuration, by further analyzing / extracting means for extracting a parameter representing characteristics of the musical sound by analyzing the musical sound, the modulating means comprises:
The musical sound may be modulated by changing the extracted parameter based on the pseudo-random signal. Furthermore, it is desirable that the apparatus further includes a synthesizing unit that synthesizes the musical sound after being modulated by the modulating unit and the musical tone before being modulated.

  In addition, the musical sound processing method according to the present invention is a cut-off frequency setting for setting a cut-off frequency for a filter means that removes a signal of a specific frequency component from a noise signal generated by a noise generator and outputs it as a pseudo-random signal. And a modulation process for modulating a musical sound input based on the pseudo-random signal output from the filter means in which the cutoff frequency is set.

  Further, the musical sound processing program according to the present invention includes a noise generating means for generating a noise signal and a filter for removing a signal of a specific frequency component from the noise signal according to a set cutoff frequency and outputting the signal as a pseudo random signal. And a means for modulating the input musical sound based on the pseudo-random signal.

  As described above, according to the present invention, it is possible to give a natural chorus effect or the like.

Embodiments according to the present invention will be described below with reference to the drawings.
A. Embodiment A-1. Overall Configuration FIG. 1 is a diagram showing a configuration of a sound processing apparatus (musical sound processing apparatus) 100 according to the present embodiment.
The audio signal input unit 200 is configured by a microphone or the like, and inputs the audio uttered by the user into the audio processing apparatus 100 as an input audio signal.
The audio signal analysis unit (analysis / extraction means) 300 performs an FFT (Fast Fourier Transform) analysis on the input audio signal supplied from the audio signal input unit 200 in units of frames (about 5 to 10 ms), and the like. , Silent determination, pitch, volume, spectrum extraction. Then, the audio signal analysis unit 300 uses each clone signal generation unit 410-k (1) as a frame information with parameters indicating the characteristics of the audio obtained by such analysis, that is, voiced / unvoiced determination results, pitch, volume, and spectrum. ≦ k ≦ n) etc. Whether the input voice signal is voiced or unvoiced may be determined by analyzing the energy and frequency components of the voice signal.

  The clone signal generation unit 400 is a means for generating a converted audio signal (hereinafter referred to as a clone signal) whose tone color, pitch, volume, output timing, etc. are slightly different from the input audio signal, and a plurality of clone signal generation units 410-k. It is constituted by. Each clone signal generation unit (modulation unit) 410-k converts the frame information supplied from the audio signal analysis unit 300 into a pseudo random signal (1 ≦ k ≦ n) supplied from the pseudo random signal generation unit 510-k (1 ≦ k ≦ n). The clone signal is generated by changing based on (described later).

  The pseudo-random signal generation unit 500 is a means for generating a pseudo-random signal used when generating a clone signal, and includes a plurality of pseudo-random signal generation units 510-k (1 ≦ k ≦ n). Each pseudo-random signal generator 510-k generates pseudo-random signals having different amplitudes and the like, and supplies these pseudo-random signals to the corresponding clone signal generators 410-k. FIG. 2 is a diagram illustrating a configuration of the pseudo random signal generation unit 510-k, and FIG. 3 is a diagram illustrating a waveform of the pseudo random signal generated in the pseudo random signal generation unit 510-k. In the following description, when there is no need to particularly distinguish each pseudo-random signal generator 510-k and each clone signal generator 410-k, the pseudo-random signal generator 510 and clone signal generator 410 Abbreviated. A function representing a pseudo random signal as shown in FIG. 3 is abbreviated as a pseudo random function rand (t).

  A white noise generator (noise generating means) 511 shown in FIG. 2 randomly generates a noise signal within a certain level under the control of a control unit (not shown), and supplies the noise signal to an LPF (filter means) 512. The LPF 512 removes a signal having a frequency component higher than the cutoff frequency Fc set by the cutoff frequency setting unit 513 from the supplied noise signal, and outputs this to the normalizing unit 514. The cut-off frequency setting means 513 generates a cut-off frequency Fc that fluctuates within a certain frequency range around 2 Hz, and sets this in the LPF 512. Thus, the reason why the cutoff frequency Fc is set to swing around 2 Hz is because it most closely matches the natural change and way of shaking when a person sings (because it swings around 2 Hz). Note that the timing at which the cutoff frequency Fc is fluctuated around 2 Hz can be arbitrarily set. Of course, the cutoff frequency Fc may be fixed without being fluctuated around 2 Hz.

  When the normalizing means 514 receives the noise signal from which the high frequency component has been removed from the LPF 512, the normalizing means 514 normalizes the noise signal to fall within the range of −1 to 0 to 1 as shown in FIG. Output to. As a result, each pseudo-random signal generator 510 outputs a pseudo-random signal that best matches the natural change and shaking method when a person sings (however, the method of amplitude differs for each pseudo-random signal). Will be.

  Returning to FIG. 1, when each clone signal generation unit 410 receives a pseudo-random signal from each corresponding pseudo-random signal generation unit 510, the clone information generation unit 410 changes the frame information based on the received pseudo-random signal, thereby different clone signals. (That is, converted audio signals having different shift amounts such as timbre and pitch) are generated.

The control information input unit (input unit) 600 includes operation buttons, operation switches, and the like, and receives control instructions regarding various effects (vibrato effect and the like; details will be described later) input from the outside via the operation buttons.
The signal synthesizer (synthesizing unit) 700 is a unit that synthesizes each clone signal generated in the clone signal generation unit 400 and the input audio signal, and includes a first adder 710, a second adder 720, and a converter. 730.

  The first adder 710 adds the spectrum of each clone signal supplied from each clone signal generation unit 410 and outputs the addition result to the second adder 720. The second adder 720 adds the spectrum addition result output from the first adder 710 and the spectrum of the input audio signal supplied from the audio signal analysis unit 300, and outputs the addition result to the converter 730. . The converter 730 performs inverse FFT or the like on the addition result (that is, the addition result of all the spectra) output from the second adder 720 to obtain a synthesized voice signal obtained by synthesizing the input voice signal and each clone signal. Then, the signal synthesis unit 700 supplies the synthesized voice signal obtained by the inverse FFT or the like (that is, a synthesized voice signal having a slightly shifted tone color, pitch, etc.) to the voice output unit 800.

The audio output unit 800 is configured by a speaker or the like, and outputs the synthesized audio signal supplied from the signal synthesis unit 700 to the outside. By using the speech processing apparatus 100 having such a configuration, it is possible to obtain an effect as if a plurality of people are actually singing.
Hereinafter, various functions realized by each clone signal generation unit 410 will be described in detail.

A-2. Output Timing Change Function The output timing change function is a function that changes the output timing of the clone signal with respect to the input audio signal, and is realized by the timing change means 411 shown in FIG.
FIG. 4 is a diagram illustrating a state when the output timing of the frame information is changed by the timing changing unit 411. In FIG. 4, the pitch included in the frame information is exemplified, the pitch before the output timing is changed is indicated by a solid line, and the pitch after the output timing is changed is indicated by a broken line. As shown in FIG. 4, the pitch output timing (that is, the pitch time delay amount) is changed in the phrase switching portion. FIG. 5 is a flowchart showing a process (timing changing process) for changing the output timing, and FIG. 6 is a diagram for explaining the timing changing process. In the following description, it is assumed that state value = 2 and delay value = 0 are preset in a memory (not shown) as initial conditions.

  Upon receiving the frame information from the audio signal analysis unit 300, the timing changing unit (detection unit) 411 acquires the volume value of the input audio signal from the frame information, and stores this as a volume value AMP in a register (not shown). (Step S1). Then, the timing changing unit 411 refers to the memory and determines the state value at that time (step S2). When the timing changing unit 411 determines that the state value is “2”, the process proceeds to step S3, and whether or not the current volume value AMP is greater than a preset second volume threshold G2 (> G1). Judging. When the timing changing unit 411 determines that the current volume value AMP is equal to or lower than the second volume threshold value G2 (step S3; NO), the timing changing unit 411 ends the process as it is, while the current volume value AMP is set to the second volume threshold value G2. If it is determined that the value is larger (step S3; YES), the process proceeds to step S4, the state value is switched from “2” to “1” (see P1 shown in FIG. 6), and the process ends.

If the timing change unit 411 determines that the state value is “1” in step S2, the process proceeds to step S5, where the current volume value AMP is smaller than a preset first volume threshold G1. Determine whether. When the timing changing unit 411 determines that the current volume value AMP is equal to or greater than the first volume threshold G1 (step S5; NO), the timing changing unit 411 ends the process as it is, while the current volume value AMP is equal to the first volume threshold G1. If it is determined that the delay time is smaller (step S5; YES), the process proceeds to step S6, a new delay value is generated using the following equation (1), the delay value is rewritten (delay ← new delay), and the state value Is switched from “1” to “2” (see P2 shown in FIG. 6), and the process ends. The change amount z1 shown in the following formula (1) is control information input from the outside via the control information input unit 600, and the output timing can be adjusted by changing the change amount z1. (This also applies to the amount of change z2 and the like described below).
New Delay [s] = {1 + rand (t)} * k1 * z1 (1)
k1; constant
z1: Change amount (0 to 1)

  As described above, the timing changing unit 411 obtains a new delay value when the volume of the input audio signal decreases and the volume falls below the first threshold G1 (<G2). When the timing changing unit 411 obtains a new delay value, the timing changing unit 411 appropriately changes the output timing of the frame information according to the new delay value. Here, changing the Delay value causes the problem that the audio waveform becomes discontinuous and abnormal noise occurs, but the above condition (that is, a decrease in the volume of the input audio signal is detected, and the According to the condition that the sound volume is lower than the first threshold value G1, the abnormal sound is masked, so that it is possible to prevent a problem that an unnatural noise that is unnatural for hearing is heard.

  The frame information output from the timing changing unit 411 in this way is supplied to the supply control unit 412 shown in FIG. When the supply control means 412 receives the frame information, the supply control means 412 refers to the frame information to determine whether the input voice is voiced or unvoiced. If it is determined that the input voice is voiced, the supply control unit 412 outputs the pitch and volume to the trend changing unit 413, outputs the spectrum to the first spectrum changing unit 418, and further outputs the spectrum to the output switching unit 420. A determination result to the effect is output (see the voiced system shown in FIG. 1). On the other hand, if it is determined that the input voice is silent, the supply control unit 412 outputs the spectrum to the second spectrum changing unit 419, and further outputs a determination result indicating that the input voice is silent to the output switching unit 420 (FIG. 1). (See the unvoiced system shown below).

A-3. Trend Change Function The trend change function is a function that gives a relatively large change (hereinafter referred to as trend change) to the supplied pitch and volume, and is realized by the trend change means 413 shown in FIG.
FIG. 7 is a diagram illustrating a change in the trend of the pitch. The pitch before the trend change is indicated by a solid line, and the pitch after the trend change is indicated by a broken line.
When the trend changing means (parameter control means) 413 receives the pitch and volume from the timing changing means 411, the trend changing means 413 substitutes these into formulas (2) and (3) below, respectively, to give a trend change to the pitch and volume. In addition, the input pitch (t) [Hz] and the input volume (t) [dB] shown in the following formulas (2) and (3) indicate the pitch and volume supplied from the supply control unit 412 respectively.

Output pitch (t) [Hz] = Input pitch (t) [Hz] * {1 + rand (t) * k2 * z2} (2)
Output volume (t) [dB] = Input volume (t) [dB] + rand (t) * k3 * z3} (3)
k2, k3; constants
z2, z3: Amount of change (0 to 1)
Thus, by using a pseudo random signal as a signal for giving a trend change to the pitch and volume, a more natural change can be given compared to the case of using a sine wave signal or the like.

A-4. Scribbing effect imparting function The scribbling effect imparting function is a function for changing the pitch and volume trajectory when an attack of the input voice signal is detected, and is realized by the sneezing effect imparting means 414 shown in FIG.
FIG. 8 is a diagram showing how the pitch changes when the scouring effect is applied. The pitch before the sneezing effect is given is shown by a solid line, and the pitch after the sneezing effect is given is shown by a broken line. As is well known, when a person sings, they sometimes “suck” at the beginning of the sound (attack). This “shakuri” may be different for each person or depending on the situation of singing. It is the scouring effect imparting means 414 that simulates this scouring and imparts a natural sneezing effect.

  FIG. 9 is a diagram for explaining a method for controlling the scouring effect. The sneezing effect applying means (detecting means) 414 first detects the attack time (see P1 shown in FIG. 9) by comparing the given volume with a preset threshold value. When the attack time is detected, the shackle effect imparting means (change amount calculating means) 414 obtains the pitch change amount ΔPitch using the pseudo random function rand (t). Specifically, the sneezing effect giving means 414 reaches the maximum value ΔPitch when a predetermined entry time (a certain time) has elapsed from the attack time (see P2 shown in FIG. 9), and from the attack time. The change amount ΔPitch is obtained so that the change amount ΔPitch converges to “0” when a predetermined convergence time (a certain time) has passed (see P3 shown in FIG. 9).

At this time, the sneezing effect applying means (time calculating means) 414 calculates not only the change amount ΔPitch but also the entry time and convergence time using the pseudo-random function rand (t). It should be noted that how to use the pseudo-random function rand (t) may be appropriately determined according to the size, length, etc. of the squealing effect desired by the user. Then, the scooping effect imparting means (parameter changing means) 414 substitutes the change amount ΔPitch obtained as described above into the following equation (4) to obtain the output pitch [cent]. Note that the input pitch [cent] shown in the following formula (4) indicates the pitch supplied from the trend changing means 413.
Output pitch [cent] = input pitch [cent] + ΔPitch [cent] (4)

  The scouring effect imparting means 414 obtains the change amount ΔPitch based on the pseudo-random function rand (t) as described above, and adds the obtained change amount ΔPitch to the input pitch, whereby the sneezing effect as shown by a broken line in FIG. Is obtained. It should be noted that the volume change when the squealing effect is applied can be explained by the logic that is almost the same as the pitch change described above, and is therefore omitted.

A-5. Vibrato effect imparting function The vibrato effect imparting function is a function for adding vibrato to a portion where sound is extended, and is realized by vibrato effect imparting means 415 shown in FIG.
FIG. 10 is a diagram showing how the pitch changes when the vibrato effect is applied. The pitch before the vibrato effect is given is shown by a solid line, and the pitch after the vibrato effect is given is shown by a broken line.

  As a premise for providing such a vibrato effect, the user operates control buttons or the like of the control information input unit 600 to control instructions relating to vibrato effects such as average vibrato delay, average vibrato depth, and average vibrato speed (= rate). Enter the vibrato control information). The input vibrato control information is supplied from the control information input unit 600 to the vibrato effect applying means 415. When receiving the vibrato control information, the vibrato effect applying means (modulating means) 415 determines the volume supplied from the sneezing effect applying means 414 and a preset threshold value in order to determine whether or not to apply the vibrato effect. Compare When the vibrato effect imparting means 415 determines that the volume exceeds a preset threshold, the average vibrato delay, average vibrato depth, and average vibrato speed are substituted into the following formulas (5), (6), and (7). And seek new vibrato delay, vibrato depth and vibrato speed.

Vibrato delay = average vibrato delay * {1 + rand (t) * k4} (5)
Vibrato depth = average vibrato depth * {1 + rand (t) * k5} (6)
Vibrato speed = average vibrato speed * {1 + rand (t) * k6} (7)
k4, k5, k6; constants

  Thus, the vibrato effect imparting means 415 changes the vibrato control information such as the average vibrato delay, the average vibrato depth, and the average vibrato delay based on the pseudo random function rand (t), and obtains new vibrato control information. Then, after a new vibrato delay time elapses, the vibrato effect imparting means 415 applies vibrato at the vibrato depth and vibrato speed obtained by this calculation. As a result, a vibrato having different phases, different start times, different depths, and different speeds is applied to each clone signal (see FIG. 10), and it is possible to give a more dispersed feeling. Note that the volume change when the blur effect is applied can be described in substantially the same manner as the pitch change described above, and is therefore omitted.

A-6. Transition Section Changing Function The transition section changing means 416 is a function that changes the way of change when the pitch or volume changes greatly (transition section), and is realized by the transition section changing means 416 shown in FIG.
FIG. 11 is a diagram showing the state of the pitch change before and after the transition part. The pitch before the change is indicated by a solid line (where the transition part is a thick solid line), and the pitch after the change is indicated by a broken line. As is well known, even if the singing melody and the like are the same, if the person who sings is different, the way the pitch and volume change at a place where the pitch and volume change greatly (that is, the transition part) is different. The transition part changing means 416 realizes a natural change in the transition part by simulating such changes in pitch and volume.

  Here, the detection of the transition part of the pitch will be described. First, the transition part changing means (transition part detecting means) 416 is a short-term average value (for example, 50 [ms] interval) of the pitch given from the vibrato effect applying means 415. Find the average value of the pitch. Next, the transition part change means 416 takes the difference (that is, differentiation) between the short-time average value obtained last time and the short-time average value obtained this time for the short-time average value thus obtained. Then, the transition portion changing means 416 sets the change amount of the pitch in advance after the absolute value of the differentiation (that is, the absolute change amount of the pitch) exceeds a preset first threshold value. Until it falls below the second threshold value (<first threshold value), it is detected as a transition part (see the transition part indicated by a thick solid line in FIG. 11). More specifically, the transition part changing means (transition part time detecting means) 416 detects the time when the pitch change amount exceeds a preset first threshold as the start time of the transition part (see FIG. 12). The time after the start time and when the amount of change in the pitch falls below the second threshold is detected as the end time of the transition unit (see P2 shown in FIG. 12).

FIG. 12 is a diagram for explaining a pitch control method in the transition section detected in this way.
First, the transition portion changing means (calculating means) 416 uses the following equation (8) to calculate the amount of pitch change per unit time (for example, one frame time) based on the pseudo-random function, that is, how much pitch per unit time. Ask whether to change.
Pitch change amount [cent] = rand (t) * k7 (8)
k7; constant

Next, the transition portion changing means 416 obtains the pitch displacement amount ΔPitch by substituting the pitch change amount into the following equation (9). However, if the pitch is changed too much, it will sound audible. Therefore, the pitch change function f (t) is defined not to change more than a certain amount (limit value) as shown in FIG. Further, after the end of the transition, the pitch displacement amount ΔPitch is converged to “0” over a certain period of time from the transition end time (P2 shown in FIG. 12). However, how the pitch displacement amount ΔPitch after the transition end time converges can be arbitrarily set.
ΔPitch [cent] = Pitch change [cent] * f (t) (9)
f (t); pitch change function (see Fig. 12)

Then, the transition portion changing means (parameter changing means) 416 obtains the output pitch [cent] by substituting the pitch displacement amount ΔPitch obtained in this way into the following equation (10). Note that the input pitch [cent] shown in the following formula (10) indicates a pitch supplied from the vibrato effect applying means 415.
Output pitch [cent] = input pitch [cent] + ΔPitch [cent] (10)
The transition part changing means 416 obtains the pitch change amount and the pitch displacement amount ΔPitch based on the pseudo-random function rand (t) in this way, and adds the obtained pitch displacement amount ΔPitch to the input pitch, so that the broken line in FIG. An output pitch having a different manner of change is obtained at the transition part as shown in FIG. Note that the volume change before and after the transition portion can be described in substantially the same manner as the pitch change, and is omitted.

A-7. Small Change Function The small change function is a function for adding a fine change (hereinafter referred to as small change) to the supplied pitch and volume, and is realized by the small change means 417 shown in FIG.
FIG. 13 is a diagram illustrating a state of small pitch change, in which the pitch before the small change is indicated by a solid line, and the pitch after the small change is indicated by a broken line. In the above-described trend change, a relatively large change is given to the pitch and volume, but here, a fine change is given to the pitch and volume at even shorter time intervals. However, if speech synthesis or the like is performed without giving such a small change, since the speech is synthesized at a constant pitch and volume, it sounds like a mechanical sound (for example, a buzzer-like sound). .

  On the other hand, when a fine change is given to a pitch and a sound volume at a short time interval, it is possible to enjoy an effect that the sound is naturally heard. As described above, in order to hear the sound as a sound, a slight change in pitch and volume is necessary. However, the way of the change naturally varies from person to person. In order to simulate this, the small change means (parameter control means) 417 gives a slight random change by changing the supplied pitch and volume using a pure random signal. Note that a pseudo-random function rand (t) may be used instead of the pure random signal.

A-8. Tone change function A-8-1. First timbre change function The first timbre change function is a function that gives a different timbre change for each clone signal, and is realized by the first spectrum changing means 418 shown in FIG.
FIG. 14 is a diagram illustrating a spectrum of a certain frame of the input audio signal. In FIG. 14, the frequency f [Hz] is on the horizontal axis and the amplitude value magnitude [dB] is on the vertical axis. In FIG. 14, the spectrum envelope is indicated by a solid line, and a curve indicating a slope (hereinafter referred to as ECurve) is indicated by a broken line. First, ECurveMag (f), which is the amplitude value of ECurve, can be expressed by the following equation (1 ′).
ECurveMag (f) = Gain + 100 * (e ^{−slope * f} −1) (1 ′)
Gain: Gain of the frame
slope; slope of the frame

  When receiving the spectrum of the input audio signal from the supply control means 412, the first spectrum changing means 418 changes the ECurve slope expressed as described above for each clone signal. 15 and 16 are diagrams showing changes in the spectral envelope when the slope of ECurve is changed. FIG. 15 shows changes in the spectral envelope when the slope is increased. FIG. The change of the spectrum envelope when making it small is shown. Note that the spectrum envelope and ECurve shown in FIGS. 15 and 16 are all based on the spectrum envelope and ECurve shown in FIG.

  As is obvious from comparison between FIGS. 15 and 14 and FIGS. 16 and 14, when the slope is changed, the gain (the ECurve intercept in each figure) indicating the overall volume does not change, but the ECurve slope is changed. With the change, the shape of the spectrum envelope changes, thereby changing the timbre. More specifically, as shown in FIG. 15, when the slope is increased, the high-frequency spectrum does not appear, and the tone becomes muffled. On the other hand, when the slope is reduced as shown in FIG. 16, the spectrum appears uniformly from the low range to the high range, so that a bright tone is obtained. The first spectrum changing means 418 can give a different timbre change for each clone signal by changing the slope of the spectrum envelope for each clone signal in this way. Note that the method of changing the slope of the spectrum envelope for each clone signal can be set as appropriate.

A-8-2. Second timbre change function The second timbre change function is a function that changes the timbre with time (that is, temporally continuously). Like the first timbre change function, the first spectrum change means shown in FIG. 418. For example, if the method of changing the spectral envelope is changed with time in the attack portion of the input audio signal, etc., there is an effect that a gospel chorus effect or the like can be obtained. The gospel-like singing effect means an effect that gives a situation as if the singing expression by each singer is various, for example, because the time change of the timbre is different for each singer. Here, as a method of changing the timbre with time as described above, for example, there is a method of changing the change value of the first formant frequency (frequency at which the spectrum peak first appears) with time. FIG. 17 is a diagram illustrating a change in the first formant frequency, in which the first formant frequency before the timbre change is indicated by a solid line, and the first formant frequency after the timbre change is indicated by a broken line. FIG. 18 is a diagram for explaining a control method of the first formant frequency for realizing such a timbre change.

  First, upon receiving the spectrum of the input audio signal from the timing changing unit 411, the first spectrum changing unit (modulating unit) 418 extracts a spectrum envelope from the spectrum and analyzes the spectrum to detect an attack start time. Do. When the first spectrum changing means 418 detects the attack time (see P1 shown in FIG. 18), the first spectrum changing means 418 obtains a change target value (referred to as a first formant change value) of the first formant frequency using the pseudo-random function rand (t). . More specifically, the first spectrum changing means 418 reaches the change target value set in advance when the predetermined entry time has elapsed from the attack time (see P2 shown in FIG. 18). ), The first formant change value is obtained so as to converge to “0” when a predetermined convergence time has elapsed from the attack time (see P3 shown in FIG. 18).

At this time, the first spectrum changing means 418 calculates not only the first formant change value but also the entry time and convergence time using the pseudo-random function rand (t). Note that how to use the pseudo-random function rand (t) may be appropriately determined according to the magnitude, length, etc. of the gospel chorus effect desired by the user. Then, the first spectrum changing means 418 substitutes the first formant change value obtained as described above into the following equation (11) to obtain the output first formant frequency [Hz]. Note that the input first formant frequency [Hz] shown in the following formula (11) indicates the first formant frequency of the input audio signal supplied from the supply control means 412.
Output first formant frequency [Hz] = input first formant frequency [Hz] + first formant change value [Hz] (11)

  In this way, by changing the first formant frequency with time, the above-described spectrum envelope (see FIG. 14 and the like) changes with time. Along with this, the slope of the spectrum envelope changes, so that the timbre eventually changes with time. In the above description, the first formant frequency is changed with time. However, for example, the mth formant frequency (2 ≦ m) is changed with time, or the amplitude value of the mth formant is changed with time. Depending on which mode the timbre is changed with time, it can be appropriately selected. The first spectrum changing means 418 also realizes the function of changing the spectrum based on the pitch and volume supplied from the small changing means 417 in addition to realizing the first timbre changing function and the second timbre changing function described above. To do. Therefore, when the input voice is a voiced sound, the spectrum of the clone signal to which various effects are given by each means of the voiced system is output from the first spectrum changing means 418.

A-9. Unvoiced sound quality changing function The unvoiced sound quality changing function is a function of changing the sound quality of the unvoiced sound when the input voice is unvoiced, and is realized by the second spectrum changing means 419 shown in FIG.
When receiving the spectrum of the input voice signal (unvoiced sound) from the supply control means (determination means) 412, the second spectrum change means (change means) 419 receives a random signal (in this case, a random signal generator). The random signal is a pure random signal), and the amplitude value magnitude [dB] (see FIG. 14) and the phase value at each frequency f [Hz] of the spectrum are randomly changed. As a result, an unvoiced sound such as “s” having a different nuance sound quality can be output for an unvoiced sound having no pitch such as “s”. As described above, when the input voice is an unvoiced sound, the spectrum of the clone signal in which the amplitude value magnitude [dB] and the phase value are changed by the second spectrum changing unit 419 of the unvoiced system is output. . Of course, a pseudo-random signal may be used instead of the pure random signal.

A-10. Output switching function The output switching function is a function for switching the output spectrum between a voiced system and an unvoiced system, and is realized by the output switching means 420 shown in FIG. When the determination result supplied from the supply control means 412 is “voiced”, the output switching means 420 supplies the spectrum (voiced spectrum) output from the voiced system to the signal synthesis unit 700 by switching to the voiced system side. On the other hand, when the determination result is “unvoiced”, the spectrum output from the unvoiced system (unvoiced spectrum) is output to the signal synthesis unit 700 by switching to the unvoiced system side.

  The signal synthesis unit 700 adds all voiced or unvoiced spectra supplied from each clone signal generation unit 410 and the spectrum of the input voice signal, and performs a reverse FFT or the like to obtain a synthesized voice signal. This synthesized speech signal is a synthesized speech signal to which various effects are given based on the pseudo-random function rand (t) as described above. Therefore, the user or the like can enjoy the effect as if a plurality of people are actually singing by listening to the synthesized voice generated from the synthesized voice signal via the voice output unit 800. It becomes.

  As described above, according to the audio processing device according to the present embodiment, since a pseudo random signal obtained by multiplying white noise by LPF is used as a modulation signal used at the time of audio conversion, a natural change (ie, actually, Change as if multiple people are singing). In the LPF, the cut-off frequency Fc is set to swing around 2 Hz in consideration of human voice characteristics such as shaking around 2 Hz. As a result, compared with the case where the cut-off frequency Fc is fixed, it becomes possible to match the human natural change and the way of shaking. Of course, the pseudo-random function rand (t) used in each process may be different for each process.

B. Modification <Modification 1>
FIG. 19 is a diagram for explaining the transition part changing function according to the first modification, and corresponds to FIG. 11 described above. In FIG. 19, as in FIG. 11, the pitch before the change is shown by a solid line (however, the pitch transition part is a thick solid line), and the pitch after the change is shown by a broken line.
The transition part changing function according to the modified example 1 is configured to change the detuning amount (a slight shift amount of the pitch) of the pitch stable part every time the pitch transition part changes from a pitch changing part to a pitch stable part where the pitch is stable. This is a function of determining and changing the detune amount for each pitch stable part, and is realized by the transition part changing means 416 ′ shown in FIG. The transition part changing function according to the first modification does not simulate a natural change of a human voice like the transition part changing function according to the present embodiment, but changes a musical sound emitted from an instrument such as a violin. Is to simulate. As is well known, a stringed instrument such as a violin is different from a keyboard instrument such as a piano, and a musical tone having a just pitch (for example, “A” = 440 [Hz]) corresponding to the key is always emitted when a certain key is pressed. Rather, the sound changes slightly depending on where the string is pressed. In other words, every time a musical instrument such as a violin is changed, the pitch slightly deviates from the just pitch (for example, “A” = 440 [Hz]). It has the characteristics of being stable. It is the transition part changing means 416 ′ that simulates such changes (see the respective detune amounts dt1 to dt3 shown in FIG. 19).

  FIG. 20 is a diagram for explaining a pitch control method by the transition portion changing unit 416 ′ according to the first modification, and is a diagram schematically showing an α portion shown in FIG. 19. The transition part change means (detection means) 416 'detects the pitch transition part in the same manner as the transition part change means 416 described above, and detects the other part as a pitch stabilization part. When the transition section changing means 416 ′ detects the transition from the pitch stable section to the pitch transition section (see P1 in FIG. 20), the detune amount dt2 (> 0) in the pitch stable section determined as described later is given for a certain time. Multiply t1 to converge to “0”. Thereafter, when the transition portion changing means (parameter changing means) 416 ′ detects a transition from the pitch transition portion to the pitch stable portion (see P2 shown in FIG. 20), a new detune is generated based on the pseudo-random function rand (t). The quantity dt3 (<0) is determined. Then, the transition part changing means 416 ′ controls to obtain a new detune amount dt3 over a period of time t2, and thereafter, until this transition is detected from the pitch stable part to the pitch transition part. Maintain the quantity dt3.

The transition section changing means 416 ′ obtains the output pitch [cent] by substituting each detune amount dt [cent] obtained in this way into the following equation (12). Note that the input pitch shown in the following formula (12) indicates the pitch supplied from the vibrato effect applying means 415.
Output pitch [cent] = input pitch [cent] + dt [cent] (12)
The transition part changing means 416 ′ obtains the detune amount dt based on the pseudo-random function rand (t) in this way, and adds the obtained detune amount dt to the input pitch. An output pitch having a different detune amount dt is obtained for each pitch stable part.

  If the method according to this modification described above is applied to an input musical tone such as a violin, a user or the like can listen to a realistic musical tone like a performance by a strings section. In the above example, the case where the detune amount dt is determined based on the pseudo random signal has been described. However, the time for the detune amount to converge to “0” (a certain time t1) and the time for increasing the detune amount to a certain value. (A certain time t2) may be determined based on the pseudo-random signal. Moreover, both the transition part changing means 416 ′ according to the first modification and the transition part changing means 416 according to the present embodiment may be mounted on the speech processing apparatus 100, and any one of the transition part changing means is provided. You may mount in this apparatus 100.

<Modification 2>
Further, in the present embodiment described above, the case where the chorus effect (basic pitch is the same for all clone signals) has been described. For example, the basic pitch is different for each clone signal, and these are synthesized. The present invention can also be applied to the case of obtaining a harmony effect that composes a musical harmony. In order to obtain the harmonizer effect, for example, the pitch output from each timing changing means 411 shown in FIG. . For example, when the sound of C4 is input to the inside of the apparatus, the pitch changed from C4 to C3 is output from the timing changing unit 411-1 and the pitch changed from C4 to A4 is output from the timing changing unit 411-2. Is output from the timing changing means 411-n. The pitch changed from C4 to G4 is output. In this way, a harmonicizer effect may be obtained.

<Modification 3>
In the above-described embodiment, the voice is exemplified as the input musical sound. However, the present invention can also be applied to the case where an instrument sound (for example, strings) is input. In addition, it is of course possible to appropriately change the pan parameters for controlling the sound localization for the clone signals output from the clone signal generation units 410 shown in FIG. According to this aspect, it is possible to obtain a chorus effect or the like that appropriately spreads in the stereo space. In the present embodiment described above, an LPF that fluctuates the cutoff frequency Fc around 2 Hz is exemplified, but it is needless to say that not only the LPF but also a BPF (Band Pass Filter) or the like may be used. Further, the present invention is not limited to a mode in which the cut-off frequency Fc is fluctuated around 2 Hz (that is, the cut-off frequency Fc is changed within a certain frequency range), and can be appropriately changed according to each change item.

<Modification 4>
Further, since the functions of the respective units of the voice processing apparatus 100 described above are realized by a program stored in a ROM or the like, the program is recorded on a recording medium such as a CD-ROM and distributed, or the Internet or the like. You may distribute via a communication network. Of course, the function of each unit of the speech processing apparatus 100 may be realized by hardware.

It is a figure which shows the structure of the audio processing apparatus which concerns on this embodiment. It is a figure which shows the structure of the pseudo random signal generation part which concerns on the same embodiment. It is the figure which illustrated the waveform of the pseudorandom signal concerning the embodiment. It is a figure which shows a mode when the output timing of the frame information which concerns on the embodiment is changed. It is a flowchart which shows the timing change process which concerns on the same embodiment. It is a figure for demonstrating the timing change process which concerns on the same embodiment. It is a figure which shows the mode of the trend change of the pitch which concerns on the same embodiment. It is a figure which shows the mode of a pitch change when the scooping effect which concerns on the embodiment is provided. It is a figure for demonstrating the control method of the scooping effect which concerns on the same embodiment. It is a figure which shows the mode of a pitch change when the vibrato effect which concerns on the embodiment is provided. It is a figure which shows the mode of the pitch change before and behind the transition part which concerns on the same embodiment. It is a figure for demonstrating the control method of the pitch in the transition part which concerns on the embodiment. It is a figure which shows the mode of the small change of the pitch which concerns on the same embodiment. It is the figure which illustrated the spectrum of a certain frame with the input voice signal concerning the embodiment. It is a figure which shows the change of the spectrum envelope when the slope of ECurve which concerns on the same embodiment is enlarged. It is a figure which shows the change of the spectrum envelope when the slope of ECurve which concerns on the embodiment is made small. It is a figure which shows the mode of a change of the 1st formant frequency concerning the embodiment. It is a figure for demonstrating the control method of the 1st formant frequency which concerns on the embodiment. It is a figure for demonstrating the transition part change function which concerns on the modification 1. FIG. It is a figure for demonstrating the control method of the pitch which concerns on the modification.

Explanation of symbols

DESCRIPTION OF SYMBOLS 100 ... Audio processing apparatus, 200 ... Audio signal input part, 300 ... Audio signal analysis part, 400 ... Clone signal generation unit, 410 ... Clone signal generation part, 411 ... Timing change Means 412 ... Supply control means 413 ... Trend changing means 414 ... Scribbing effect applying means 415 ... Vibrato effect applying means 416, 416 '... Transition part changing means 417 ..Small changing means, 418 ... first spectrum changing means, 419 ... second spectrum changing means, 420 ... output switching means, 500 ... pseudorandom signal generating unit, 510 ... pseudorandom Signal generator, 511... White noise generator, 512... LPF, 513... Cutoff frequency setting means, 514. 515 ... Output means, 600 ... Control information input unit, 700 ... Signal synthesis unit, 710 ... First adder, 720 ... Second adder, 730 ... Conversion unit, 800: Audio output unit.

Claims (18)

Noise generating means for generating a noise signal;
Filter means for removing a signal of a specific frequency component from the noise signal according to a set cutoff frequency and outputting as a pseudo-random signal;
A musical sound processing apparatus comprising: modulation means for modulating an input musical sound based on the pseudo-random signal.   2. The musical tone processing apparatus according to claim 1, further comprising cut-off frequency setting means for setting the cut-off frequency and changing the cut-off frequency within a certain frequency range.   The filter means is a low-pass filter that removes a signal having a frequency component higher than the cut-off frequency, and the cut-off frequency setting means changes the cut-off frequency within a certain frequency range around 2 Hz. The musical tone processing apparatus according to claim 2. An analyzing / extracting means for extracting a parameter representing the characteristic of the musical sound by analyzing the musical sound;
The modulating means includes
The musical tone processing apparatus according to claim 1, wherein the musical tone is modulated by changing an extracted parameter based on the pseudo-random signal.   5. The musical tone processing apparatus according to claim 1, further comprising a synthesizing unit that synthesizes the musical sound after being modulated by the modulating unit and the musical tone before being modulated. .   The musical sound processing apparatus according to claim 4, wherein the parameter is a pitch or a volume of the musical sound. The parameter control means includes
Detecting means for detecting an attack time of the input musical sound;
A change amount calculating means for calculating a change amount per unit time of the pitch or volume of the musical sound until a certain time has elapsed since the attack time was detected;
7. The musical tone processing apparatus according to claim 6, further comprising parameter changing means for changing the pitch or volume of the musical tone by adding the calculated change amount to the pitch or volume of the musical tone. The parameter control means includes
8. The musical sound processing apparatus according to claim 7, further comprising time calculating means for calculating the certain time using the pseudo random signal.   Detecting means for detecting the volume of the sound; and when the volume of the sound detected by the detecting means decreases and the volume falls below a threshold value, the output timing of the parameter is based on the pseudo-random signal. 5. The musical tone processing apparatus according to claim 4, further comprising timing changing means for changing. It further comprises input means for inputting vibrato control information for controlling the addition of vibrato,
The modulating means includes
4. The musical tone is modulated by changing the vibrato control information based on the random signal and adding vibrato to the musical tone according to the changed vibrato control information. The musical sound processing apparatus according to claim 1. The parameter control means includes
When a condition of how to change the pitch or volume of the musical sound satisfies a certain condition, a transition detection unit that detects the part as a transition part of the pitch or volume of the musical sound,
Transition part time detecting means for detecting a start time and an end time of the transition part;
Calculating means for calculating at least a change amount per unit time of pitch or volume of the musical sound from when a start time of the transition unit is detected until an end time is detected;
7. The musical tone according to claim 6, further comprising parameter changing means for changing the calculated pitch or volume by adding the calculated change amount to the corresponding pitch or volume of the musical tone. Processing equipment.   The detection means obtains an average value of the pitch or volume of the musical sound at regular intervals, and when the difference between the average value obtained last time and the average value obtained this time exceeds a threshold value, the portion is detected as the pitch of the musical sound or The musical tone processing apparatus according to claim 11, wherein the musical tone processing apparatus is detected as a volume transition unit.   The calculation means calculates the amount of change from when the start time of the transition unit is detected until the end time is detected, and also the pitch of the musical sound until a certain time has elapsed since the end time was detected. Alternatively, the musical sound processing apparatus according to claim 12, wherein the amount of change per unit time of the volume is calculated using the pseudo-random signal. The parameter is the amount of music detuning,
The parameter control means includes
A detecting means for detecting a portion satisfying a certain condition of a change in the pitch of the musical sound as a pitch transition portion and the other portion as a pitch stable portion;
The parameter changing means for changing the detune amount based on the pseudo-random signal every time a transition from the pitch transition portion to the pitch stabilization portion is detected. Music processing device. The musical sound is a voice,
A determination means for determining whether the voice is voiced or unvoiced;
When it is determined that the voice is unvoiced, it further comprises changing means for randomly changing the amplitude value or phase value of each frequency component constituting the voice modulated by the modulation means. The musical sound processing apparatus according to any one of claims 1 to 3. The modulating means includes
The musical sound processing according to any one of claims 1 to 3, wherein a spectral envelope of the musical sound is extracted, and the spectral envelope is continuously changed in time based on the pseudo-random signal. apparatus. A cut-off frequency setting process for setting a cut-off frequency for the filter means that removes the signal of the specific frequency component from the noise signal generated by the noise generator and outputs it as a pseudo-random signal,
A musical sound processing method comprising: modulating a musical sound that is input based on the pseudo-random signal output from the filter means in which the cutoff frequency is set. Computer
Noise generating means for generating a noise signal;
Filter means for removing a signal of a specific frequency component from the noise signal according to a set cutoff frequency and outputting as a pseudo-random signal;
A musical sound processing program for causing an input musical sound to function as modulation means for modulating the musical sound based on the pseudo-random signal.

Images