JP4444751B2 - Musical sound generating apparatus and method - Google Patents

Description

  The present invention relates to a musical sound generation apparatus and method, and more particularly to a musical sound generation apparatus and method for modulating a musical sound signal.

  The electronic musical instrument reads musical tone information recorded in a storage device such as a memory in accordance with the performance information, and determines a pitch, volume, envelope, and filter characteristics based on the performance information, thereby generating a musical tone.

  The problem with this electronic musical instrument is that the tone signal produced is monotonous with respect to changes in performance information, which limits the music expression of the performer. Further, this electronic musical instrument has only time waveform information and has no concept of spatial coordinates. This is an unrealistic sound considering that real instruments have physical dimensions. A musical process that expresses a skillful performance cannot be achieved with only linear processing.

  Patent Document 1 below discloses an electronic musical instrument that repeatedly calculates a solution of an equation of motion and generates musical sound waveform data that changes with time.

Japanese Patent No. 2722727

  An object of the present invention is to generate a musical sound signal rich in performance expression by imitating a natural musical instrument.

Tone generation apparatus of the present invention uses the performance information corresponding to the performance operation as an initial value of nonlinear partial differential equations to calculate the solution of the time function of the nonlinear partial differential equations, the modulation factor of the time function based on the solution A modulation coefficient generation means for generating a musical tone signal based on the performance information, and a modulation means for modulating the musical tone signal generated by the sound source in accordance with the modulation coefficient of the time function .

Further, the tone generating method of the present invention, the performance information corresponding to the performance operation calculates the solution of the time function of the nonlinear partial differential equations using as an initial value of nonlinear partial differential equations, the time function of the basis of the solution modulation and the modulation coefficient generation step of generating a modulation coefficient, and tone signal generation step of generating a sound source tone signals based on the performance information, in accordance with the modulation factor of the time function, a tone signal generated by the sound source Modulation step.

By generating a modulation coefficient of the time function solutions based on the time function of nonlinear partial differential equations as input performance information, it can be taken into account information of spatial coordinates as well the time waveform, the sound of the musical instrument Can be close. Therefore, it is possible to flexibly respond to changes in performance information and generate musical sound signals rich in performance expression.

  FIG. 1 is a block diagram illustrating a configuration example of an electronic keyboard instrument (musical sound generating device) according to an embodiment of the present invention. A live piano, which is a natural instrument, has a soundboard, and the sound has a spatial spread. In this embodiment, a musical tone signal such as a soundboard is imitated and a musical tone signal is modulated in consideration of spatial coordinate information.

  The keyboard 101 is a performance operator that outputs performance information (performance event) in accordance with the performance operation of the performer. Instead of the keyboard 101, other performance operators may be used. The tone generator system 102 generates and outputs a musical sound (PCM) signal based on the performance information output from the keyboard 101.

  The first processing unit 111 uses the performance information output from the keyboard 101 as an initial value of the nonlinear partial differential equation, and calculates a solution of the nonlinear partial differential equation by a difference method. This performance information is, for example, velocity (key pressing speed). Details of the processing of the first processing unit 111 will be described later with reference to Equation (4). The second processing unit 112 discretizes the solution of the equation calculated by the first calculation unit 111 in a space (for example, a two-dimensional space), takes a norm, and generates a multiplication coefficient for each of the discretized coordinates.

  The third processing unit 113 is a signal processing unit such as a DSP (digital signal processor), and calculates a head-related transfer function for the musical sound signal output from the sound source system 102 and multiplies the multiplication coefficient of each coordinate. Specifically, the musical tone signal output from the sound source system 102 is convoluted with a head-related transfer function that matches each coordinate, and is further delayed in consideration of the arrival delay time corresponding to the distance, so that each spatial coordinate is Sound signal can be obtained. The acoustic signals corresponding to those spatial coordinates are added to become an output musical tone signal. Details of the processing of the third processing unit 113 will be described later with reference to equations (5) and (6).

  FIG. 2 is a diagram for theoretically explaining the processing of the first to third processing units 111 to 113 in FIG. 1. The first stage 201 corresponds to the first processing unit 111 in FIG. 1, the second stage 202 corresponds to the second processing unit 112 in FIG. 1, and the third stage 203 and the computing unit 204 are in FIG. This corresponds to the third processing unit 113.

  The first stage 201 solves a nonlinear partial differential equation having performance information (for example, velocity) as an initial value by a difference method to derive a solution u (t). The second stage 202 generates a modulation coefficient M (t) by discretizing the space with respect to the solution u (t) in order to reduce the amount of calculation by reducing the number of modulation coefficients. In the third stage 203, amplitude modulation (acoustic conversion) is performed on the head-related transfer function corresponding to each coordinate using the generated modulation coefficient M (t) to generate the modulation coefficient G (t). . The computing unit 204 performs computation based on the monaural one-channel music signal f (n) and the modulation coefficient G (t) output from the sound source system 102, and outputs a stereo two-channel music signal g (t). . Hereinafter, the first to third stages 201 to 203 will be described in detail.

(1) First stage 201
In the first stage 201, a nonlinear partial differential equation is solved. A linear one-dimensional wave equation having a viscosity term is expressed as shown in Equation (1).

The result of the solution u (t) at an arbitrary time with respect to the initial value u (0) is similar to the magnitude of the initial value u (0) when the magnitude is changed. That is, it is linear.
Here, VanDerPol's ordinary differential equation used as a chaotic oscillator is shown in Equation (2).

This equation gives the following solution:
When η = 0 Linear oscillation When η> 0 Non-linear oscillation When η <0 Non-linear attenuation

  Therefore, a second partial term on the left side of equation (1) is expanded to the second term on the left side of equation (2) and further expanded to two dimensions, so that a nonlinear partial differential equation is proposed as equation (3). However, since the nonlinear attenuation state is mainly used, the sign of the second term on the left side is “positive” and the wave equation is expanded.

  As a method for sequentially acquiring the solution of Equation (3) according to performance information, Equation (3) must be discretized in both time and space and solved as a differential scheme. What converted Formula (3) into the difference scheme is shown in Formula (4).

  n is a discretized time axis, i and j are two-dimensional coordinate systems, Δt is a discrete interval in time, and Δx is a discrete interval in space.

(2) Second stage 202
In the second stage 202, spatial discretization is performed. Consider a case where a spatial norm is calculated for each time for the two-dimensional nonlinear wave equation (3). Numerical calculation is performed under the following conditions. Numerical calculation was performed on the following condition for the expression (4).

Time discrete Δt = 0.1 [sec]
Spatial discrete Δx = 0.1 [m]
Group velocity c = 0.5 [m / sec]
Decay rate η = 0.6
Boundary condition 0 (at t ≧ 0)
Initial value coordinates (i−10) ² + (j−5) ² <10 (at t = 0)

  FIG. 3 is a graph showing each norm when the initial value u (0) of n = 0 is a δ function. The characteristic I1 has an initial value of 1, the characteristic I2 has an initial value of 2, the characteristic I3 has an initial value of 3, and the characteristic I3.5 has an initial value of 3.5. The horizontal axis indicates time, and the vertical axis indicates the norm of the solution u. The norm indicates the length of the vector. It can be observed that the larger the amplitude of the initial value, the smaller the irregular signal appears. It can be seen that even if the spatial norm is taken, the property of nonlinearity is not lost. From these, the coordinates can be separated by the spatial norm.

Since the size of u _{i, j} ^{n + 1} is currently a 100 × 100 two-dimensional matrix, there are a total of 10,000 discrete coordinates. This requires a large amount of computation, and cannot be calculated practically at the next acoustic conversion. Therefore, by taking the norm of a certain coordinate section, the coordinate area can be discretized without impairing the non-linear behavior.

  FIG. 4 is a diagram illustrating an example of discretizing 100 × 100 discrete coordinates into 4 × 4. A two-dimensional space 401 corresponds to a soundboard, and shows a state in which the two-dimensional space of the X axis and the Y axis is divided into 4 × 4.

(3) Third stage 203
In the third stage 203, acoustic conversion is performed. As shown in FIG. 4, the solution u _{i, j} ^{n + 1} is converted so as to be a 1 m square matrix M with 0.25 m as one cell. The listener 402 receives a musical sound signal from the soundboard in the two-dimensional space 401.

  As shown in Expression (5), the musical sound signal is convolved with the input signal f (n) with respect to the impulse response in which the direction and distance between the head-related transfer function HRTF and each element of the matrix M are considered. On the other hand, each element of the matrix M is multiplied and amplitude-modulated on the time axis. The delay function delay is delayed in consideration of the arrival time λ corresponding to the distance.

  The signals amplitude-modulated by each element of the matrix M are added as shown in Equation (6), and an output signal g (n) is obtained by adding directly reaching sounds. The direct arrival sound is a sound that is generated at 0 seconds and serves as a reference for the time axis, and the relationship between the direct arrival sound and other sounds is indispensable for generating an acoustic space.

  In this way, the three kinds of sounds A2, A4 and A6 were processed. The original waveform to be processed is a primitive piano simulation sound in which a simple envelope is added to a zero-phase fundamental harmonic generated by sine wave synthesis.

FIG. 5 is a diagram showing an original waveform of the pitch A6 before processing. 6A is a diagram showing the waveform of the left channel of the processed pitch A6, and FIG. 6B is a diagram showing the waveform of the right channel of the processed pitch A6. The horizontal axis indicates time, and the vertical axis indicates amplitude. As a condition, a waveform example is shown as a result of calculation assuming that λ _0,0 = 0.45, coordinates of u _j ¹ are x = 0.25 m, y = 0.25 m, and the initial value of the δ function is 3.5.

  By modulating the original waveform f (n) in FIG. 5, the two-channel waveforms in FIGS. 6A and 6B can be obtained. The listener 402 can obtain a left channel tone signal that can be heard by the left ear 403L and a right channel tone signal that can be heard by the right ear 403R.

  In the waveforms of FIGS. 6A and 6B, it is observed that irregular amplitude changes appear compared to the original waveform of FIG. It is also slightly different between the left and right signals. Such an irregular amplitude change is a phenomenon also seen in a signal when a grand piano is actually picked up. This is a behavior often seen in actual piano waveforms. That is, this embodiment can generate a sound imitating a natural musical instrument live piano.

  The present embodiment can dramatically improve the player's expression compared to the normal method. In the normal method, the tone color is changed by changing the overtone structure by filter processing or the like according to the strength (for example, velocity) of performance information. However, the magnitude of the overtone simply changes, and the timbre changes are small, resulting in a narrowing of the musical expression.

  These processes are performed for each keyboard, and the sounds of each keyboard do not interfere with each other. In this case, when a plurality of sounds are pronounced at the same time as a chord, the sound is uncoordinated and a beautiful sound cannot be obtained.

  In the present embodiment, the performance information is used as an initial value to obtain a solution of the nonlinear partial differential equation. Therefore, the modulation coefficient is sequentially changed according to the performance information. Further, since the coordinates are modulated by spatial acoustics such as a head-related transfer function based on the coefficient, it can be said that, as a result of signal processing, the result is a state in which cross modulation of amplitude, frequency, and phase is applied. This intermodulation creates sidebands in the harmonic structure and creates a complex harmonic structure. An example is shown in FIGS.

  FIG. 7 is a frequency spectrum of a waveform showing a modulated overtone structure, and FIG. 8 is an expanded frequency spectrum of the fundamental part of FIG. The signal 700L indicates a left channel signal, and the signal 700R indicates a right channel signal. These produce sidebands in the harmonic structure and have a complex harmonic structure, both of which are slightly different.

  Also, when multiple sounds are pronounced simultaneously like chords, the performance information is the initial value of the nonlinear partial differential equation, so each sound that is pronounced has causality to each other and a beautiful sound is obtained. .

  As described above, according to the present embodiment, the performance information is used as the initial value of the nonlinear partial differential equation, the solution of the nonlinear partial differential equation is calculated, and the modulation coefficient is generated based on the solution. Then, the tone signal is modulated according to the modulation coefficient.

  By generating a modulation coefficient based on the partial differential equation with performance information as input, it is possible to take into account not only the temporal waveform but also spatial coordinate information, making it closer to the sound of a natural instrument. . Therefore, it is possible to flexibly respond to changes in performance information and generate musical sound signals rich in performance expression.

  The above-described embodiments are merely examples of implementation in carrying out the present invention, and the technical scope of the present invention should not be construed in a limited manner. That is, the present invention can be implemented in various forms without departing from the technical idea or the main features thereof.

It is a block diagram which shows the structural example of the electronic keyboard instrument (musical sound production | generation apparatus) by embodiment of this invention. It is a figure for demonstrating theoretically the process of the 1st-3rd process part of FIG. It is a graph which shows each norm in case initial value u (0) is (delta) function. It is a figure which shows the example which discretizes 100x100 discrete coordinates into 4x4. It is a figure which shows the original waveform of the pitch A6 before a process. FIG. 6A is a diagram showing the waveform of the left channel of the processed pitch A6, and FIG. 6B is a diagram showing the waveform of the right channel of the processed pitch A6. It is a figure which shows the frequency spectrum of the waveform which shows the modulated overtone structure. It is a figure which shows the frequency spectrum which expanded the fundamental part of FIG.

Explanation of symbols

101 keyboard 102 sound source system 111 first processing unit 112 second processing unit 113 third processing unit 201 first stage 202 second stage 203 third stage 204

Claims (9)

Using the performance information corresponding to the performance operation as an initial value of nonlinear partial differential equations to calculate the solution of the time function of the nonlinear partial differential equations, a modulation coefficient generating means for generating a modulation coefficient of the time function based on the solution ,
A sound source that generates a musical sound signal based on the performance information;
A musical sound generating apparatus comprising: modulation means for modulating a musical sound signal generated by the sound source according to a modulation coefficient of the time function . The modulation coefficient generating means, the musical tone generating apparatus according to claim 1 for generating a modulation coefficient by taking the norm by discretizing a solution of the partial differential equations in a space. 3. A musical sound generating apparatus according to claim 2 , wherein the modulation coefficient generating means discretizes in a two-dimensional space. The modulation coefficient generating means, the musical tone generating apparatus according to claim 2 or 3, wherein generating a modulated coefficients by performing acoustic conversion using the head-related transfer function to the value obtained by taking the norm. It said modulating means, the musical tone generating apparatus according to any one of claims 1-4 for generating a musical tone signals of two channels by modulating the tone signal of one channel. The musical tone generation apparatus according to any one of claims 1 to 5, further comprising a performance operator that outputs performance information in accordance with a performance operation. The musical tone generation apparatus according to claim 6 , wherein the performance operator is a keyboard. The performance information is musical sound generating apparatus according to any one of claims 1 to 7, which is a velocity. Using the performance information corresponding to the performance operation as an initial value of nonlinear partial differential equations to calculate the solution of the time function of the nonlinear partial differential equations, a modulation coefficient generation step of generating a modulation coefficient of the time function based on the solution ,
A musical sound signal generating step for generating a musical sound signal by a sound source based on the performance information;
A musical sound generation method comprising: a modulation step for modulating a musical sound signal generated by the sound source according to a modulation coefficient of the time function .

Images