JP2024046785A - Effect imparting device, method, and program - Google Patents

Abstract

[Problem] This relates to an effect imparting technology that imparts sound effects to musical sound data generated by a sound source in units based on block units consisting of a predetermined number of samples, and improves memory utilization efficiency in the computational process that imparts sound effects to musical sound data in units of blocks. [Solution] In a block judgment process (block judgment processing unit) executed within a block interrupt process that is started by interrupt on a block basis, the computation process in the computation processing unit that imparts effects in units of blocks based on block management information that indicates whether musical sound data matches computational processing conditions in units of blocks, for example whether the data is all 0, is adjusted, for example, between normal computation and simple output processing of all 0 values. [Selected Figure] Figure 10

Description

The present invention relates to an effect imparting device, method, and program that imparts sound effects to musical sound data generated by a sound source in units based on block units consisting of a predetermined number of samples.

In an effect imparting device that imparts sound effects such as reverberation and resonance effects by convolving an impulse response with a musical sound signal, a technique is known that uses FFT/iFFT (Fast Fourier Transform/inverse Fast Fourier Transform) that performs convolution in the frequency domain (e.g., Patent Document 1).
In this case, each FFT frame is used multiple times for both the coefficient data of the impulse response and the tone signal.

JP 2005-215058 A

Electronic musical instruments often have a configuration in which the CPU (Central Processing Unit), sound source LSI (Large Scale Integrated Circuit), and DSP (Digital Signal Processor) share memory, so if one process occupies a large amount of memory, there is a problem that the performance of the entire electronic musical instrument decreases.

The present invention aims to improve memory usage efficiency in the computational process of adding sound effects to musical sound data on a block-by-block basis.

An example of an effect imparting device includes a calculation processing unit that executes calculation processing to impart sound effects on a block-by-block basis to musical sound data generated by a sound source, and a calculation control processing unit that adjusts the calculation processing in the calculation processing unit based on block management information indicating whether the musical sound data satisfies calculation processing conditions on a block-by-block basis.
Equipped

The present invention makes it possible to improve memory utilization efficiency in computational processing that applies sound effects to musical sound data on a block-by-block basis.

1 is a block diagram illustrating an example embodiment of an electronic musical instrument. FIG. 2 is a diagram showing an example of the flow of musical tone data in an electronic musical instrument. FIG. 2 is a block diagram of an effects device. FIG. 11 is an explanatory diagram (part 1) of the operation of the CONV calculation processing unit. FIG. 13 is an explanatory diagram (part 2) of the operation of the CONV calculation processing unit. FIG. 11 is an explanatory diagram (part 3) of the operation of the CONV calculation processing unit. 1 is an explanatory diagram of a processing delay in frequency domain convolution processing using FFT calculation. 4 is an explanatory diagram showing the processing relationship between an FIR filter calculation processor and a CONV calculation processor; FIG. 11 is a main flowchart showing an example of control processing of a main processing operation. 13 is a flowchart showing an example of detailed processing of an initialization process and a block interrupt process. 13 is a flowchart illustrating a detailed example of a block determination process. 13 is a flowchart showing an example of detailed processing of frame calculation processing and calculation processing. FIG. 11 is an explanatory diagram (part 4) of the operation of the CONV calculation processing unit. 13 is a flowchart showing another detailed example of the block determination process.

The embodiment of the present invention will be described in detail below with reference to the drawings. The filter calculation processing device used in the effect imparting section of the electronic musical instrument of this embodiment can execute FIR filter calculation processing that allows the filter order to be flexibly changed, and can simultaneously execute multiple FIR filter calculation processing with different filter orders and impulse response characteristics while flexibly changing the combinations of these. The effect imparting section in this embodiment is implemented by combining time domain convolution processing using such FIR filter calculation processing and frequency domain convolution processing using FFT calculation. In this case, it is possible to flexibly determine the FIR filter order along with the number of FFT points in the frequency domain convolution processing, so that it is possible to impart highly reproducible reverberation and resonance effects to the musical sounds of the electronic musical instrument without sacrificing responsiveness.

Figure 1 is a block diagram showing an example of an embodiment of an electronic musical instrument 100. The electronic musical instrument 100 has at least one CPU (Central Processing Unit) 101, a ROM (Read Only Memory) 102, a RAM (Random Access Memory) 103, a sound source (TG: Tone Generator) 104, an effect imparting section 105 which is a DSP (Digital Signal Processor), a keyboard 106, pedals 107, and an operator 108, all connected to a system bus 109. The output of the sound source (TG) 104 is connected to a sound system 110. The effect imparting section 105 is equipped with an effect device 211, which will be described later.

The CPU 101 executes a control program loaded from the ROM 102 to the RAM 103, and issues sound generation instructions to the sound source 104 based on performance operation information from the keyboard 106 and the controls 108.

The tone generator (TG) 104 generates musical tone data by reading waveform data from the ROM 102 or RAM 103 in accordance with the sound generation instructions. This musical tone data is output to the sound system 110 via the effect imparting unit 105. Here, for example, when the operator depresses the pedal 107, the effect imparting unit 105 imparts effects to the musical tone data, such as imparting reverberation such as reverberation or the resonance of piano strings. As a result, the musical tone data output from the effect imparting unit 105 is converted into an analog musical tone signal by a digital-to-analog converter in the sound system 110, amplified by an analog amplifier, and emitted from a speaker.

Figure 2 is a diagram showing an example of the flow of musical tone data in an electronic musical instrument having the configuration of Figure 1. The tone generator (TG) 104 includes musical tone generators 201 (CH1) to 201 (CHn) that generate musical tone data for n tone generation channels CH1 to CHn, and generate independent musical tone data for each key pressed according to tone generation instructions from the CPU 101 of Figure 1 that are generated based on the key pressed by the operator on the keyboard 106. The musical tone generator 201 (CHi) (1 ≤ i ≤ n) corresponding to tone generation channel CHi includes a waveform generator WG.CHi that generates waveform data, a filter processor TVF.CHi that processes the tone of the generated waveform data, and an amplifier envelope processor TVA.CHi that processes the amplitude envelope of the generated waveform data.

The four mixers 203, 204, 205, and 206 in the mixing unit 202 output Lch (left channel) direct sound output data 207, Rch (right channel) direct sound output data 208, Lch sound effect input data 209, and Rch sound effect input data 210, which are obtained by multiplying each musical sound data output by each musical sound generating unit 201 (CHi) (1≦i≦n) by a predetermined level and then accumulating the data, to the effect imparting unit 105. Note that in FIG. 2, the symbols "*, Σ" in each mixer 203, 204, 205, and 206 indicate that the input data is multiplied by a predetermined level, then accumulated and output.

The Lch sound effect input data 209 and the Rch sound effect input data 210 are output as Lch sound effect output data 212 and Rch sound effect output data 213, respectively, to which effects such as reverberation and resonance have been applied by the effect device 211 in the effect application unit 105. In the effect application unit 105, the Lch sound effect output data 212 is added to the Lch direct sound output data 207, and output as Lch musical sound output data 214 to the sound system 110 in FIG. 1. Similarly, the Rch sound effect output data 213 is added to the Rch direct sound output data 208, and output as Rch musical sound output data 215 to the sound system 110. In the sound system 110, the Lch musical sound output data 214 and the Rch musical sound output data 215, which are digital data, are converted into Lch analog musical sound signals and Rch analog musical sound signals, respectively, and are amplified by analog amplifiers and emitted from the Lch and Rch speakers.

Figure 3 is a block diagram of the effect device 211 in the effect imparting unit 105 of Figure 2. The effect device 211 has an effect device 211 (Lch) and an effect device 211 (Rch). The effect device 211 (Lch) outputs Lch sound effect output data 212 obtained by imparting Lch reverberation and resonance effects to the input Lch sound effect input data 209. The effect device 211 (Rch) outputs Rch sound effect output data 213 obtained by imparting Rch reverberation and resonance effects to the input Rch sound effect input data 210. Since both have the same configuration, the following description will be made without distinction between Lch and Rch unless otherwise specified. The effect device 211 (Lch) (effect device 211 (Rch)) has an FIR filter calculation processing unit 303 and a CONV calculation processing unit 304.

The Lch sound effect input data 209 or Rch sound effect input data 210 input to the effect device 211 is input in parallel to the FIR filter calculation processing unit 303 and the CONV calculation processing unit 304. The FIR filter calculation processing unit 303 is a time domain convolution unit that convolves the first half of the data portion of the impulse response data of the reverberation and resonance sound into the Lch sound effect input data 209 or the Rch sound effect input data 210 (original sound) by time domain processing in sampling period units. The CONV calculation processing unit 304, which is a calculation processing unit, is a frequency domain convolution unit that convolves the second half of the data portion of the impulse response data into the Lch sound effect input data 209 or the Rch sound effect input data 210 (original sound) by frequency domain processing using FFT calculation in block units consisting of a predetermined number of samples. The FIR filter calculation processing unit 303 and the CONV calculation processing unit 304 perform calculation processing while using the RAM 306 implemented in the DSP, which is the effect imparting unit 105 in FIG. 2, as a common area.

The outputs of the FIR filter calculation processing unit 303 and the CONV calculation processing unit 304 are added in the addition unit 305 and output as the Lch sound effect output data 212 or the Rch sound effect output data 213.

Figures 4 to 7 are diagrams explaining the operation of the CONV calculation processing unit 304 in Figure 3. First, Figure 4 shows an example of convolution with a block size of N points. Convolution using FFT calculations will result in circular convolution if left as is, so in this embodiment, when performing FFT calculations at 2N points on the impulse response data (coef) and the Lch sound effect input data 209 or the Rch sound effect input data 210 (hereinafter, these will be collectively referred to as sound effect input data (sig) without distinguishing between Lch and Rch), a linear convolution of each block is obtained.

Here, prior to each of the 2N-point FFT calculations 403 and 404, N-point zero data (thin frame part) is added to N-point impulse response data (coef) (bold frame part) to obtain 2N-point data, as shown as 401. Also, as shown as 402, the sound effect input data (sig) is overlapped while being shifted by the block size of N points (bold frame part → dashed thick frame part) to obtain 2N-point data. Then, FFT calculations are performed on the 2N-point data 401 generated from the impulse response data (coef) and the 2N-point data 402 generated from the sound effect input data (sig) as shown as 403 and 404, respectively, to obtain 2N-point data 405 and 406 in the frequency domain.

Next, as shown by 407, the 2N points of frequency domain data 405 and 406 are complex multiplied for each frequency point to obtain 2N points of complex multiplication result data 408.

Furthermore, as shown at 409, an iFFT (inverse fast Fourier transform) operation is performed on the 2N points of complex multiplication result data 408, resulting in 2N points of convolved time domain data 410.

Then, the first N points of data (thick framed portion) of the 2N points of time domain data 410 become the linear convolution result using the overlap reservation method with a block size of N points, and the data generated in this way for each block size of N points is output as the Lch sound effect output data 212 or the Rch sound effect output data 213 in Figure 3.

Figure 5 shows an example of convolution with a block size of N points when the impulse response data (coef) is also divided into blocks of N points. In this example, the convolution results for each N point are added together, so that even long-time impulse responses can be divided into small blocks of N points to perform convolution operations.

In FIG. 5, as shown by 501, impulse response data (coef) is divided into, for example, divided data N1, N2, N3, and N4, each having a block size of N. Also, as shown by 502, sound effect input data (sig) is also divided into, for example, divided data N1, N2, N3, and N4, each having a block size of N.

After that, as in the case of FIG. 4, prior to each of the 2N-point FFT calculations 505 and 506, as shown in 503, N-point zero data is added to the N-point divided data (thick frame part) in the impulse response data (coef) divided as described above to obtain 2N-point data. Also, as shown in 504, the sound effect input data (sig) is shifted by the block size N points obtained by dividing as described above and overlapped (thick frame part → dashed thick frame part) to obtain 2N-point data. Then, FFT calculations are performed on the 2N-point data 503 generated from the N-point data divided from the impulse response data (coef) and the 2N-point data 504 generated from the sound effect input data (sig) as shown in 505 and 506, respectively, and 2N-point data 507 and 508 in the frequency domain are obtained in sequence, 2N points at a time, as shown in 509 and 510. Here, the frequency data group 509 (N1, N2, N3, N4) of 2N points each generated from the divided data N1, N2, N3, N4 (501 in FIG. 5) of the impulse response data (coef) can be calculated by FFT and preset in memory as long as there is no change in the impulse response data (coef). Also, the frequency data group 510 of 2N points each generated from the sound effect input data (sig) may be stored in memory in a ring buffer format, such as t(N), t(N-1), t(N-2), t(N-3), for the same number of divided data as the frequency data group 509 (N1, N2, N3, N4) corresponding to the impulse response data (coef).

Next, as shown by 511, frequency data group 509 (N1, N2, N3, N4) and frequency data group 510 (t(N), t(N-1), t(N-2), t(N-3)) are complex multiplied for each 2N frequency point to obtain 2N complex multiplication result data N1*t(N), N2*t(N-1), N3*t(N-2), N4*t(N-3), which are then subjected to an iFFT operation to obtain convolved time domain data 513 for each 2N points.

Then, the first N points of data (thick framed parts) of each of the 2N points of time domain data 513 are output as the Lch sound effect output data 212 or the Rch sound effect output data 213 in FIG. 3.

Figure 6 shows a simple calculation example to explain the operation of the CONV calculation processing unit 304 in Figure 3.

for example,
conv(coef = [2 3 4 5 6 7 8 9]
sig = [1 2 3 4 5 6 7 8 9 10]
When calculating with block size N=2, the 9th and 10th convolution results are 200 and 244.
ifft(fft([0 0 2 3])*fft([7 8 9 10]))
ifft(fft([0 0 4 5]) * fft([5 6 7 8]))
ifft(fft([0 0 6 7])*fft([3 4 5 6]))
ifft(fft([0 0 8 9])*fft([1 2 3 4]))
Simply add up the first half of each calculation result.
Note that "fft()" and "ifft()" respectively indicate an FFT operation and an iFFT (inverse FFT) operation on the data group within the parentheses.

The above calculation example allows linear convolution using FFT calculations to be realized, but because FFT calculations are performed on a block-by-block basis, delays occur at the beginning of the blocks. Figure 7 is an explanatory diagram of processing delays in frequency domain convolution processing using FFT calculations.

For example, in block section T1, N1 block data is input, and a condition determination is made to branch to specific processing described later. When a block-based interrupt occurs at the beginning of the next block section T2, N2 block data is input, and a condition determination is made to branch to specific processing described later, and convolution processing (FFT operation, complex multiplication, and iFFT operation) is performed on the N1 block data. When a block-based interrupt occurs at the beginning of the next block section T3, N3 block data is input, and a condition determination is made to branch to specific processing described later, and convolution processing of N2 block data and output processing of N1 block data are performed. When a block-based interrupt occurs at the beginning of block section T4, N4 block data is input, and a condition determination is made to branch to specific processing described later, and convolution processing of N3 block data and output processing of N2 block data are performed. Next, when a block-based interrupt occurs at the beginning of block section T5, the N5 block is input and a condition determination is made to branch to the specific processing described below, and convolution processing of the data in the N4 block and output processing of the data in the N3 block are performed.

As can be seen from the timing relationship shown in Figure 7, in linear convolution processing using FFT calculations, a processing delay of 2N occurs for block size N, as shown as 701. Therefore, in the effect device 211 of Figure 3, an FIR filter calculation processing unit 303 with a filter order of 2N is implemented to cover the above-mentioned 2N processing delay that occurs in the CONV calculation processing unit 304.

The FIR filter calculation processing unit 303 has a configuration in which filter calculation units, each of which is composed of a multiplication processing unit, an accumulation processing unit, and a delay processing unit, are connected in tandem in a number corresponding to the filter order.

In addition, the signal input of the sound effect input data is performed in sampling cycle units from the mixer 205 or 206 in the mixing section 202 in the sound source (TG) 104 in FIG. 2.

Each order of filter calculation unit is executed within the sampling period in synchronization with a clock having a period that is a subdivision of the sampling period, and the FIR calculation processing at all orders and the output of the convolution result data from the accumulation processing unit at the final stage are completed within that sampling period. As a result, no delay occurs in the convolution processing in the FIR filter calculation processing unit 303 in Figure 3.

8 is an explanatory diagram showing the processing relationship between the FIR filter calculation processing unit 303 and the CONV calculation processing unit 304. As can be seen from this diagram, if the filter order of the FIR filter calculation processing unit 303 is equal to the FFT size (twice the block size N) of the CONV calculation processing unit 304, the effect of processing delay in the CONV calculation processing unit 304 can be ignored.
That is, when the first 2N samples of the sound effect input data are input to the FIR filter calculation processing unit 303, the sound effect input data is shifted by N samples and input to the CONV calculation processing unit 304 by 2N samples. Then, as shown in Fig. 8, the FIR filter calculation processing unit 303 first performs a convolution operation on the first 2N samples of the sound effect input data with the first 2N samples of the impulse response data of the sound effect (e.g., reverberation or resonance) for each sampling period, and outputs 2N samples of convolution result data without delay for each sampling period. Also, the CONV calculation processing unit 304 performs a convolution operation on the 2N samples of the sound effect input data inputted while overlapping by N samples with the N samples of the impulse response data of the sound effect after 2N+1 samples + N samples of value 0 = 2N samples in the frequency domain, and outputs N samples of convolution result data sequentially with a delay of 2N samples. Therefore, following output of the first 2N samples of convolution result data from the FIR filter calculation processing unit 303 from the adder 305 in FIG. 3, the convolution result data for every N samples sequentially output from the CONV calculation processing unit 304 is sequentially output for each sampling period, whereby it is possible to output sound effect output data (Lch sound effect output data 212 or Rch sound effect output data 213) obtained by performing a convolution calculation on the entire sound effect input data with the impulse response data, from the effect device 211 in FIG. 3 without delay.

Figure 9 is a flowchart showing an example of a control process for a main processing operation executed by the CPU 101 in Figure 1 to realize the effect device 211 in Figure 2. This control process is an operation in which the CPU 101 executes a control processing program that has been loaded from the ROM 102 to the RAM 103.

After the power switch of the control 108 is turned on in the electronic musical instrument 100 of FIG. 1, the CPU 101 starts executing the control process shown in the main flowchart of FIG. 9. The CPU 101 first executes an initialization process (step S901). FIG. 10(a) is a flowchart showing a detailed example of the initialization process of step S901 of FIG. 9.

In the initialization process, the CPU 101 initializes the contents stored in the RAM 103 in FIG. 1, the state of the tone generator (TG) 104, the state of the electronic musical instrument 100, etc. (step S1001 in FIG. 10(a)).

Next, the CPU 101 initializes flags F2 and F1, which indicate the state of the data (step S1002 in FIG. 10(a)). F2 is a flag that indicates the state of the data at the interrupt timing immediately preceding F1. When these flags F1 and F2 have a value of 1, this indicates that the value of the input data string (Lch sound effect input data 209 or Rch sound effect input data 210 in FIG. 2) at the corresponding interrupt timing is 0, and when the flags F1 and F2 have a value of 0, this indicates that the input data string contains a value other than 0.

Next, the CPU 101 enables block-based interrupts (see FIG. 7) (step S1003 in FIG. 10(a)).

Returning to the explanation of the flowchart in FIG. 9, after the initialization process described above in step S901, the CPU 101 repeatedly executes the series of processes from steps S902 to S907 until the power switch is turned off or until it is determined that the operator has finished playing.

In the above-mentioned repeated process, the CPU 101 first executes a switch process (step S902). Here, the CPU 101 detects the operation state of the operator 108 in FIG. 1.

Next, the CPU 101 executes a key press detection process to detect a key pressed by the operator (step S903). Here, the CPU 101 detects the key press state of the keyboard 106 in FIG. 1.

Next, the CPU 101 executes a pedal detection process to detect pedal operation by the operator (step S904). Here, the CPU 101 detects the operation state of the pedal 107 in FIG. 1.

Next, the CPU 101 executes effect processing (step S905). Here, based on the detection result of the operation state of the operator 108 for applying, for example, reverberation or resonance effects in step S902 and the detection result of the operation state of the pedal 107 in step S904, the CPU 101 causes the effect applying unit 105 to apply sound effects by the effect device 211 (Lch) and effect device 211 (Rch) in FIG. 3 to the Lch sound effect input data 209 and Rch sound effect input data 210 in FIG. 2 generated by the sound source (TG) 104.

Next, the CPU 101 executes other processes (step S906). Here, the CPU 101 executes, for example, a process for controlling the musical tone envelope.

Then, the CPU 101 executes sound generation processing (step S907). Here, the CPU 101 instructs the sound source (TG) 104 to generate or mute a sound based on the state of a key being pressed or released on the keyboard 106 in the key press detection processing in step S903.

In this embodiment, the calculation process in the CONV calculation processing unit 304 in the effect device 211 in the effect applying unit 105 in FIG. 1 is executed based on a block-based interrupt (see FIG. 7) generated by the CPU 101 in FIG. 1 every sampling time of N samples. In this way, the CPU 101 generates a block interrupt every time N samples of Lch direct sound output data 207 and Rch direct sound output data 208 are output from the sound source (TG) 104 in FIG. 1 and FIG. 2, based on the internal clock and the tempo of the given musical tone. When this block interrupt occurs, the CPU 101 suspends the operation of the main process in FIG. 9 and executes the block interrupt process.

Figure 10(b) is a flowchart showing a detailed example of block interrupt processing executed by the CPU 101 when a block interrupt occurs. In this block interrupt processing, the CPU 101 first executes block determination processing (step S1010 in Figure 10(b)).

4 to 7, in the processing in the CONV calculation processing unit 304 in FIG. 3, the Lch sound effect input data 209 and the Rch sound effect input data 210, which are musical sound data input from the sound source (TG) 104 to the effect imparting unit 105, are updated in block units of N samples, and sound effects are imparted by performing calculation processing including FFT with a calculation frame sample length of 2N, which is twice the block unit. In this case, in the block determination processing of step S1010 in FIG. 10, which operates as a block determination processing unit, the CPU 101 determines whether the data 209 or the data 210 satisfies the calculation processing condition in block = N sample units, more specifically, whether it meets the condition that all data sample values in the block are 0, and holds the result of the determination as block management information. The block management information is, for example, information that stores the results of each judgment of the previous and current blocks as a value of 1 (if the calculation processing conditions are met) or a value of 0 (if the calculation processing conditions are not met), for example, in flags F2 and F1, which are variable data in the RAM in FIG. 1.

Figure 11 is a flowchart showing a detailed example of the block determination process in step S1010 in Figure 10(b). In the process of the flowchart in Figure 11, the CPU 101 first resets the value of the register RJ1 of the CPU 101 and the value of the variable i, which is variable data in the RAM 103 in Figure 1, to 0 (step S1101 in Figure 11).

Next, the CPU 101 performs an OR (logical sum) operation on the register RJ1 and data [i] (0≦i≦N-1) of each sample in blocks of N samples temporarily stored in the RAM 103, which is the Lch sound effect input data 209 or Rch sound effect input data 210 to be input to the CONV calculation processing unit 304 in FIG. 3, as the operation "(RJ1 | data [i])", and re-stores the result of the OR operation on each bit in the register RJ1 (step S1102 in FIG. 11).

After that, the CPU 101 increments the value of the variable i in the RAM 103 by +1 (step S1103 of step S11).

Then, the CPU 101 determines whether the value of the variable i is smaller than N (step S1104 in FIG. 11).

If the determination in step S1104 is YES, the CPU 101 returns to the processing in step S1102 and continues the determination process.

When the number of processed samples exceeds the number of samples in block size N and the determination in step S1104 becomes NO, CPU 101 determines whether the value of register RJ1 is 0 (all bits are 0) (step S1105 in FIG. 11).

If the determination in step S1105 is NO, the CPU 101 stores the value 0 in flag F1 on the RAM 103 and turns off the flag (step S1106 in FIG. 11).

If the determination in step S1105 is YES, the CPU 101 stores the value 1 in flag F1 on the RAM 103 and turns the flag on (step S1107 in FIG. 11).

Returning to the explanation of the block interrupt process in FIG. 10(b), following the block determination process in step S1010 described in FIG. 11, the CPU 101 adjusts and executes the frame calculation process (step S1011 in FIG. 10(b)).

Fig. 12(a) is a flowchart showing a detailed example of the frame calculation process of step S1011 in Fig. 10 which operates as a calculation control processing unit. In the process of the flowchart in Fig. 12(a), the CPU 101 first determines whether or not the value of flag F1 on RAM 103 is 1 in the block determination process of step S1010 in Fig. 10 shown in the flowchart in Fig. 11, i.e., the data of the current block matches the calculation processing condition (its value is all 0), and the value of flag F2 on RAM 103 is also 1, i.e., the data of the previous block matches the calculation processing condition (its value is all 0) (step S1201 in Fig. 12(a)).

If the determination in step S1201 is NO, the CPU 101 stores non-specific frame information indicating that the 2N sample FFT frame consisting of the current block and the previous block does not have all zero values in the variable FL on the RAM 103 (step S1202 in FIG. 12(a)).

On the other hand, if the determination in step S1201 is YES, the CPU 101 stores in the variable FL information about a specific frame indicating that the value of an FFT frame of 2N samples consisting of the current block and the previous block is all 0 (step S1203 in FIG. 12(a)).

Then, the CPU 101 causes the CONV calculation processing unit 304 in FIG. 3 within the effect device 211 in FIG. 2 within the effect imparting unit 105 in FIG. 1 to execute calculation processing adjusted according to the contents of the variable FL (step S1204 in FIG. 12(a)).

Figure 12(b) is a flowchart showing an example of detailed processing of the calculation process of step S1204 in Figure 12(a). First, the CPU 101 determines whether or not information indicating a specific frame is stored in the variable FL in the RAM 103 (step S1210 in Figure 12(b)).

If the determination in step S1210 is NO, the CPU 101 instructs the CONV calculation processing unit 304 in FIG. 3 in the effect device 211 in FIG. 2 in the effect imparting unit 105 in FIG. 1 to execute a normal calculation to execute the normal convolution calculation process described above in the explanation of FIGS. 4 to 6 (step S1211 in FIG. 12(b)).

On the other hand, if the determination in step S1210 is YES, the CPU 101 instructs the CONV calculation processing unit 304 to execute a calculation process different from the above-mentioned normal convolution calculation process, for example, a specific frame process that outputs FFT frame data for 2N samples having predetermined sample values, more specifically, all-zero values (step S1212 in FIG. 12(b)). In this way, the CPU 101 adjusts whether to execute a normal calculation or a calculation process different from the normal convolution calculation process.

13 is an explanatory diagram of the operation of the CONV calculation processing unit 304 including the above-mentioned specific frame processing. The explanatory diagram of the operation in FIG. 6 described above was an example of a case where all normal calculations are performed without including such specific frame processing. In contrast, in the explanatory diagram of the operation in FIG. 13, a case where calculation is performed with block size N=2 as in the case of FIG. 6, for example, the following data is used:
conv(coef = [2 3 4 5 6 7 8 9]
sig = [1 2 0 0 0 0 0 0 9 10]
In this way, the value 0 is input from the third sample to the eighth sample in the musical tone data sig as a state corresponding to a specific process.

In this case, for example, at the timing of the block interrupt in FIG. 13(a), if an FFT frame [0 0 1 2] consisting of 2N samples is to be calculated as input data (data) corresponding to the Lch sound effect input data 209 or the Rch sound effect input data 210 in FIG. 3, the current block data [1 2] is at the beginning of the input data (data), and since there is no previous block data at the beginning, all 0's = [0 0] are inserted for the sake of convenience, and therefore flag F2 remains initialized as 1 by the initialization process in step S1002 in FIG. 10(a).
On the other hand, since the input data data=[1 2] of the current block is not all 0, the flag F1 is set to 0 by step S1101 of the block determination process in Fig. 11, N=2 repetitions of steps S1102 to S1104, step S1105, and step S1106. Therefore, the first FFT frame represented by the input data data=[0 0 1 2] of 2N=4 samples at the block interrupt timing in Fig. 13(a) has flag (F2, F1)=(0, 1), and becomes a non-specific frame by step S1201→S1202 in Fig. 12(a). As a result, between the coefficient coef=[0 0 2 3] and the input data data=[0 0 1 2], normal calculation processing is instructed by step S1210→S1211 in Fig. 12(b).
In this normal calculation process, the CPU 101 instructs the CONV calculation processing unit 304 to execute the normal convolution calculation process described above with reference to FIGS.

Next, for example, at the block interrupt timing of FIG. 13(b), when the input data DATA at the block interrupt timing of FIG. 13(a) is shifted by N=2 samples and an FFT frame [1 2 0 0] consisting of 2N samples is used as the input data DATA for calculation, the value of the previous block data [1 2] is not all 0, and its flag F2 is set to the value 0 copied from flag F1 by step S1205 of the frame calculation process of FIG. 12(a) at the block interrupt timing of FIG. 13(a).
Also, since the input data data=[0 0] of the current block has a value of all 0, the flag F1 is set to 1 by step S1101 of the block determination process in Fig. 11, N=2 repetitions of steps S1102 to S1104, step S1105, and step S1107. Therefore, the second FFT frame represented by the input data data=[1 2 0 0] of 2N=4 samples at the block interrupt timing in Fig. 13(b) has flag (F2, F1)=(1, 0), and becomes a non-specific frame by step S1201→S1202 in Fig. 12(a). As a result, between the coefficient coef=[0 0 2 3] and the input data data=[1 2 0 0], the normal calculation described above is instructed by step S1210→S1211 in Fig. 12(b).
In this normal calculation process, similarly to the case of the block interrupt timing of FIG. 13A, the CPU 101 instructs the CONV calculation processing unit 304 to execute the normal convolution calculation process described above with reference to FIGS.

Next, for example, at the block interrupt timing of FIG. 13(c), if the input data 'data' at the block interrupt timing of FIG. 13(b) is shifted by N=2 samples and an FFT frame [0 0 0 0] consisting of 2N samples is used as the input data 'data' for calculation, the previous block data [0 0] has a value of all 0 and its flag F2 is set to the value 1 copied from flag F1 by step S1205 of the frame calculation process of FIG. 12(a) at the block interrupt timing of FIG. 13(b).
Also, since the input data data=[0 0] of the current block has a value of all 0, the flag F1 is set to 1 by step S1101 of the block determination process in Fig. 11, N=2 repetitions of steps S1102 to S1104, step S1105, and step S1107. Therefore, the third FFT frame represented by the input data data=[0 0 0 0] of 2N=4 samples at the block interrupt timing in Fig. 13(b) has flag (F2, F1)=(0, 0), and becomes a specific frame by step S1201→S1203 in Fig. 12(a). As a result, between the coefficient coef=[0 0 2 3] and the input data data=[0 0 0 0], specific frame processing is executed by step S1210→S1212 in Fig. 12(b).
In processing this particular frame, the CPU 101 causes the CONV calculation processing unit 304 to output FFT frame data for 2N samples having a value of all 0. Therefore, in processing this third FFT frame, the memory required for multiplication is released, and other processing can use the free slot, making it possible to improve the efficiency of memory usage in the effect imparting unit 105.

12A in step S1205 of the frame calculation process in FIG. 12A. Furthermore, for example, at the block interrupt timing of FIG. 13D, if the input data 'data' at the block interrupt timing of FIG. 13C is shifted by N=2 samples and an FFT frame [0 0 0 0] consisting of 2N samples is subjected to calculation as the input data 'data', the previous block data [0 0] has a value of all 0 and its flag F2 is set to the value 1 copied from flag F1 by step S1205 of the frame calculation process in FIG. 12A in step S1205 of the block interrupt timing of FIG. 13C.
Also, since the input data data=[0 0] of the current block has a value of all 0, the flag F1 is set to 1 by step S1101 of the block determination process in Fig. 11, N=2 repetitions of steps S1102 to S1104, step S1105, and step S1107. Therefore, the fourth FFT frame represented by the input data data=[0 0 0 0] of 2N=4 samples at the block interrupt timing in Fig. 13(b) has flag (F2, F1)=(0, 0), and becomes a specific frame by step S1201→S1203 in Fig. 12(a). As a result, between the coefficient coef=[0 0 2 3] and the input data data=[0 0 0 0], specific frame processing is instructed by step S1210→S1212 in Fig. 12(b).
In the specific frame processing executed at the block interrupt timing of Fig. 13(d), similarly to the specific frame processing executed at the block interrupt timing of Fig. 13(c), the CPU 101 causes the CONV calculation processing unit 304 to output 2N samples of FFT frame data having a value of all 0. Therefore, in the processing of this fourth FFT frame, similarly to the processing of the third FFT frame, the memory required for multiplication is released, and other processing can use the free slot, thereby making it possible to improve the efficiency of memory utilization in the effect imparting unit 105.

Similarly, in the operation explanatory diagram of FIG. 13, at the block interrupt timings of (e), (f), (g), (h), (k), (l), (m), (n), (q), (r), (s), (t), (w), and (x), the CPU 101 instructs the CONV calculation processing unit 304 to execute the normal convolution calculation processing described above with reference to FIGS. 4 to 6.
13, at the block interrupt timings of (i), (j), (o), (p), (u), and (v), the CPU 101 instructs the CONV calculation processing unit 304 to process a specific frame and causes it to output FFT frame data for 2N samples having a value of all 0. Therefore, in the processing of these FFT frames, the memory that was required for multiplication is released, and other processing can use the free slots, making it possible to improve the efficiency of memory usage in the effect imparting unit 105.

Figure 14 is a flowchart showing another detailed example of the block determination process of step S1010 in the block interrupt process of Figure 10. In Figure 14, the same step numbers are assigned to processes that execute the same processes as in Figure 11.

In the detailed processing example of the block determination process in FIG. 11 described above, in step S1102 in FIG. 11, the CPU 101 performs an OR (logical sum) operation on the register RJ1 and the data [i] (0≦i≦N-1) of each sample in a block unit of N samples as the operation "(Rj1|data [i])" for each corresponding bit of each data, and by repeatedly storing the results of the OR operation of each bit back in the register RJ1, it is detected that the data [i] (0≦i≦N-1) of all samples in a block unit of N samples is all 0, and a specific frame is detected.

In contrast to this, in another detailed processing example of the block determination process in FIG. 14, the CPU 101 first performs an OR (logical sum) operation on the corresponding bits of each data in step S1401 in FIG. 14 using register RJ1 and the result of calculating the absolute value of the schematic amplitude value of each sample data [i] (0≦i≦N-1) in block units of N samples as the operation "(Rj1|abs(data [i]))," and repeats the process of re-storing the results of the OR operation of each bit in register RJ1. Here, "abs()" represents the process of calculating the absolute value of the number in parentheses.

Next, CPU 101 performs a bitwise AND (logical product) operation indicated by "(RJ1 & Th)" on the value obtained in register RJ1 by repeating the process of steps S1401 → S1103 → S1104 → S1401 for N samples of the entire block and masking data Th in which bits corresponding to values less than a specific threshold are 0 and bits corresponding to values greater than the threshold are 1, and then re-stores the results of the AND operation for each bit in register RJ1 (step S1402 in FIG. 14).

Then, similarly to steps S1105 to S1107 in FIG. 11, if the value of register RJ1 obtained in step S1402 is 0 (all bits are 0), CPU 101 stores a value of 1 in flag F1 on RAM 103 and turns the flag on (YES in step S1105 in FIG. 14 → S1107), and if not, stores a value of 0 in flag F1 and turns the flag off (NO in step S1105 in FIG. 14 → S1106).

In another detailed processing example of the block determination process of FIG. 14 described above, if the values of block-based data [i] (0≦i≦N-1) for N samples are not all 0 but are small values, then the accumulated value of small values obtained in register RJ1 is masked by the lower bits of the masking data Th, which has upper bits = 1 and lower bits = 0, by a bit-wise AND (logical product) operation shown as "(RJ1 & Th)", and the result becomes 0, which is reset in register RJ1, the determination in step S1105 becomes YES, and flag F1 = 1, i.e., it is determined to be a specific frame.

On the other hand, if any of the values of the block-unit data [i] (0≦i≦N-1) for N samples has a large value, the bitwise AND (logical product) operation indicated by "(RJ1 & Th)" causes the accumulated value of the large value obtained in register RJ1 to be unable to be completely masked by the AND operation with the upper bit of the masking data Th = 1 and remain as a value, and this value is reset in register RJ1. As a result, the determination in step S1105 becomes NO, and flag F1 = 0, i.e., it is determined to be a non-specific frame.

As described above, in another detailed processing example of the block determination process in FIG. 14, if the block-unit data [i] (0≦i≦N-1) has a very small value, it is determined to be a specific frame, making it possible to further reduce the calculation processing in the CONV calculation processing unit 304 and improve the utilization efficiency of the effect imparting unit 105 as a whole.

Although not specifically shown, as yet another modified example, it is possible to detect when a block crosses a threshold defined by the masking data Th, and if the block crosses the threshold, perform a level operation equivalent to an envelope on the input block-based data before performing specific frame processing.

With a similar intention, it is also possible to add noise gate processing before the input.

Furthermore, in the embodiment described so far, block determination is performed when the Lch sound effect input data 209 or the Rch sound effect input data 210 in FIG. 2 is transferred from the sound source (TG) 104 to the effect imparting unit 105, but flag information F1 may be generated and transferred when the sound source (TG) 104 generates musical tones.

In addition, in the above embodiment, convolution calculation processing using FFT was exemplified as block-based calculation processing, but even when performing effect processing, it is possible to determine a predetermined condition on a block-by-block basis and adjust whether to perform normal calculation or specific frame processing. This is also effective when processing the same block in parallel.

As described above, according to this embodiment, when memory is shared by the sound source, CPU, effect imparting section, etc., it is judged whether the musical sound data matches the calculation processing conditions in units of blocks or in units of frames based thereon, and the result of the judgment is stored as block management information, and memory access is reduced by thinning out the calculation processing for imparting sound effects based on the block management information. As a result, memory slots become available, and other processing can use the vacant slots, making it possible to improve the processing efficiency of the entire effect imparting device.
Since the number of memory accesses is reduced when performing the same processing content, this is also advantageous electrically when the memory is an external high-speed memory.
In the above description, the program relating to the audio output sound output control of the present invention is stored in ROM 102, but the present invention is not limited to this. A non-volatile memory such as a flash memory may be used as the computer-readable medium, and other non-volatile memories such as MRAM, HDDs (Hard Disk Drives), CD-ROMs, DVDs, and other portable recording media may be used as the other computer-readable media. In addition, a carrier wave is also applied to the present invention as a medium for providing data of the program according to the present invention via a communication line.
In addition, the specific configurations, contents and procedures of the processing operations, etc. shown in the above embodiments can be modified as appropriate without departing from the spirit of the present invention. The scope of the present invention includes the scope of the invention described in the claims and its equivalents.

100 Electronic musical instrument 101 CPU
102 ROM
103, 306 RAM
104 Sound source (TG)
REFERENCE SIGNS LIST 105 Effect applying section 106 Keyboard 107 Pedal 108 Operator 109 System bus 110 Sound system 201 Musical sound generating section 202 Mixing section 203, 204, 205, 206 Mixer 207 Lch direct sound output data 208 Rch direct sound output data 209 Lch sound effect input data 210 Rch sound effect input data 212 Lch sound effect output data 213 Rch sound effect output data 214 Lch musical sound output data 215 Rch musical sound output data 303 FIR filter calculation processing section 304 CONV calculation processing section 305 Addition section

Claims (9)

A calculation processing unit that executes calculation processing for adding sound effects to musical sound data generated by a sound source in units of blocks;
an arithmetic control processing unit that adjusts the arithmetic processing in the arithmetic processing unit based on block management information that indicates whether or not the musical sound data satisfies an arithmetic processing condition in block units;
An effect imparting device comprising: The effect imparting device according to claim 1, wherein, when the block management information indicates that the calculation processing conditions are met in the block unit, the calculation control processing unit instructs the calculation processing unit to perform a process of outputting data having a predetermined number of sample values instead of a calculation process for imparting the sound effect. The arithmetic processing includes a fast Fourier transform calculation of a predetermined calculation frame sample length,
3. The effect imparting device of claim 2, wherein, in each calculation frame, when the block management information contained in the calculation frame indicates that the calculation processing conditions are met, the calculation control processing unit instructs the calculation processing unit to perform a process of outputting data having a specific sample value instead of a calculation process that imparts the sound effect. The effect imparting device according to claim 3, wherein the calculation processing condition is that all sample values of the musical sound data in the block are the specific sample value. The effect imparting device according to claim 3, wherein the calculation processing condition is that the absolute values of the amplitude values of all samples of the musical sound data in the block are equal to or less than a threshold value. The effect imparting device according to claim 1, further comprising a block determination processing unit that determines whether the musical sound data satisfies a calculation processing condition on a block basis and stores the result of the determination as block management information. A computation process for adding sound effects to the musical sound data generated by the sound source in units of blocks;
a calculation control process for adjusting the calculation process based on block management information indicating whether the musical sound data matches a calculation processing condition on a block basis;
A method of applying the effect. The effect imparting method according to claim 7, wherein the calculation processing condition is that the absolute value of the amplitude of all of the musical sound data in the block is equal to or less than a threshold value. A computation process for adding sound effects to the musical sound data generated by the sound source in units of blocks;
a calculation control process for adjusting the calculation process based on block management information indicating whether the musical sound data satisfies a calculation processing condition on a block-by-block basis;
A program for causing a computer to execute the following.

Images