AU5012499A - Sound effect adding apparatus - Google Patents

Description

S F Ref: 479903

AUSTRALIA

PATENTS ACT 1990 COMPLETE SPECIFICATION FOR A STANDARD PATENT

ORIGINAL

a a a &w4 4** Name and Address of Applicant: Actual Inventor(s): Address for Service: Invention Title: Sony Corporation 7-35, Kitashinagawa 6-chome Shinagawa-ku Tokyo

JAPAN

Shigetaka Nagatani Spruson Ferguson, Patent Attorneys Level 33 St Martins Tower, 31 Market Street Sydney, New South Wales, 2000, Australia Sound Effect Adding Apparatus 1 The following statement is a full description best method of performing it known to me/us:of this invention, including the 5845 19994 9A20B 13B09 SONY ATSUGI ff 4J' No, 0923 P. 4/63 SOUND EFFECT ADDING APPARATUS BACKGROUND OF THE INVENTION Field of the Invention The present invention relates to a sound effect adding apparatus for use with a reverberation adding apparatus that adds a reverberation to an original audio signal corresponding to a real apparatus or a real space.

Description of the Related Art 10 As an apparatus that adds a sound effect to an audio signal, a reverberator is known. The reverberator is used to add reverberation to an audio signal in for example a recording studio so that listeners can have a spatial impression and a deep 15 impression. When reverberation is added to an audio signal that has been recorded in a studio or the like, a sound effect performed in a hall and a special effect can be added to the audio signal.

:"Formerly, to add reverberation to an audio signal, sound was recorded in for example a hall where reverberation was obtained. Alternatively, such as a steel-plate echo apparatus was used to obtain a reverberative effect. 'In a recent reverberator, such an effect is electrically accomplished. More recently, as digital technologies have advanced, an apparatus that digitally synthesizes reverberation is becoming common.

1999~ 920B 1A 110 SONY ATSUGI 'No. 0923 P. 5/63 When reverberation is added to an audio signal by a digital process. for example a recursive digital filter Is used. With the recursive digital filter, an input digital audio signal is attenuated and recurred. Thus, reverberation is generated. The generated reverberation is mixed with the original digital audio signal. In reality, initial reflection sound is added at a position delayed by a predetermined time period against direct sound. After a predetermined time period, reverberation is added. The delay time period of the reverberation against the direct sound Is referred to as pre-delay. By adjusting :the reverberation time, adding sub-reverberation, and finely adjusting the level of reverberation, a variety 15 of types of sound can be generated.

Reverberation in a real hall has a complicated waveform because of various reflections and interferences of sound due to the shape of the hall and the position of, a sound source. However, as described above. In the method of which an original digital audio signal is processed with a filter, since the original signal is simply attenuated, there is the problem of the listeners have an artificial impression about the generated sound with the resultant signal.

Also, the method of which an original digital audio signal is recurred by a filter process, after an input signal ceases, since the final pitch of 1999 9A20B 138,10 SONY A'fSIGI 4$11' No. 0923 P. 6/63 reverberation is equal to the pitch of the inner feedback loop of d recursive filter. Thus, the problem with this method is expressed as natural and high quality reverberation cannot be obtained.

When sound Is recorded In a real hall or the like, reverberation can be obtained more naturally.

However, in a real hail, parameters with respect to reverberation (such as reverberative time) cannot be varied. In addition, the positions and types (characteristics) of microphones cannot be quickly changed. Moreover, many apparatuses are required. In addition, due to noise of air conditioners, SIN ratio :of sound is low. Therefore, there are many problems to be solved in the related art.

3. 5 Likewise, a mechanical reverberator such as a steel-plate echo apparatus or a spring echo apparatus may be used. However, such apparatuses have problems of aged tolerance and necessity of maintenance. These problems become critical for an apparatus that cannot be obtained due to out-of-fabrlcation. In addition, such apparatuses are adversely affected by vibration and external noise. The reverberation time cannot be freely adjusted. Moreover, such apparatuses do not have good reproducibility. Furthermore, the weight and size of these apparatuses are large and S/N ratio of obtained sound is not high.

On thts other hand, a method for generating 1999 9K00~ 1301 1 SONY ATSUIJG id No. 0923 P. 7/63 reverberation in a real hall or with a steel-plate echo apparatus, collecting an impulse response corresponding to the generated reverberation, and performing a convolution calculation for the collected impulse response and the input data by a filter process has been proposed. Thus, more natural reverberation corresponding to an impulse response of a real space or an apparatus can be obtained.

When an Impulse response collected in the above-described method is used as it is. since there Is only one type of sound effect, the following problems will take place. Firstly, the reverberation time :cannot be varied. Secondly, when a reverberation contains a direct sound or an Initially reflected is 1 sound, the reverberation is similar to a simulated sound of a hall. Thirdly, since the level of a direct sound is too high, when it is mixed with an input sound, the reverberation becomes an unnatural sound.

Fourthly, when there are a plurality of impulse responses corresponding to different reverberation times, the data amount of the impul~se responses becomes large. Fifthly, the delay after an audio signal is Input until a direct sound is generated cannot be decreased to a predetermined value. Sixthly, unlike with a digital reverberation adding apparatus using an IIR filter, an initially reflected sound cannot be ddjusted.

.19994 9,20B 13B11S SONY AT'SUGI i44: No. 0923 P. 8/63 10 10 OBJECTS AND SUMMARY OF THE INVENTION Therefore, an object of the present invention is to provide a reverberation adding apparatus that solves the problems that take place in the case that collected impulse responses are used as they are.

A sound effect adding apparatus for performing a convolution calculation process for an input digital audio signal and impulse response data and thereby adding a sound effect to the input digital signal, comprising: impulse response editing means for editing an original impulse response; and convoluting means for performing a convolution calculation process for the edited impulse response and a digital audio signal.

Since an original impulse response is edited, a direct sound and an initially reflected sound can be removed from a reverberation. In addition, the reverberation time can be adjusted. Moreover, the delay until an initially reflected sound is generated can be adjusted. Thus, a high quality reverberation can be added.

The above, and other, objects, features and advantage of the present invention will become readily apparent from the following detailed description thereof which is to be read in connection with the accompanying drawings.

1999 9A206 1A 1 1 SONY ATSUITG i No. 0923 P. 9/63 BRIEF DE-SCRIPTION OF THE DRAWINGS Fig. 1 is a block diagram showing an example of the structure of a reverberation adding apparatus according to the present invention; Fig. 2 Is a block diagram showing another example of the structure of a reverberation adding apparatus according to the present invention; Figs., 3A and 3B are schematic diagrams showing an impulse response corresponding to a reverberation of a conventional recursive filter and a reverberation according to an embodiment of the present invention; Fig. 4 is a block diagram showing an example of the structure of an impulse response collecting 15 apparatus according to the present invention; Fig. 5 is a block diagrams showing examples of the case that an impulse response is collected in a hall: Fig. 6 Is a block diagram showing an example 20 of an editing process for an impulse response; Fig. 7A to 7C are schematic diagrams showing examples of an editing process for an impulse response; Fig. 8 is a block diagram showing an example of the structu'~e of a reverberation adding apparatus that performs a convolution calculation process using impulse response data.

Fig. 9 is a block diagram showing a real example of the structure of a reverberation adding 1999 920B 13,12 SONY ATSUGI ffqW 4Y No. 0923 P. 10/63 apparatus.

Fig. 10 is a is a block diagram showing an example of the structure of an option board for use with the reverberator; Fig. 11 is a schematic diagram showing an example of the structure of a front panel of the reverberator: Fig. 12A to 12H are schematic diagrams showing examples of ripples displayed in a display 10 area; Fig. 13A to 13H are schematic diagrams showing examples of ripples displayed in a display area; Fig. 14A to 14C are schematic diagrams 15 showing other examples of ripples displayed in the display area; Fig. 15 is a schematic diagram showing a process performed in each DSP that performs a convolution calculation process; Fig. 16 is a schematic diagram showing the details of the process performed in each DSP; Fig. 17 is a schematic diagram for explaining a convolution calculation process of which one impulse response is divided into a plurality of blocks; Fig. 18 is a enlarged schematic diagram showing the convolution calculation process of which one impulse response is divided into a plurality of 1999 9K06 IA12 1999 9~2O l3U25~ SONY ATSUGI fYNo093 P 1/3 No. 0923 P. 11/63 blocks; Fig. 19 is a is a block diagram showing an example of the structure of a convolution calculation filter of each DSP; Fig. 20 a schematic diagram showing a process of a convolution calculation filter on time axis; Fig. 21 is a block diagram showing a function of each DSP that performs a parallel process; Fig. 22 is a schematic diagram showing an original impulse response of a hall; Fig. 23 Is a enlarged schematic diagram showing a part of the impulse response shown in Fig.

22; Fig. 24 is a schematic diagram showing an impulse response of which impulse response components corresponding to a direct sound and an initially reflected sound are removed fromi an original impulse response; Fig. 25 is a enlarged schematic diagram showing a part of the impulse response shown in Fig.

24; Fig. 26A to 26D are schematic diagrams for explaining a process for adjusting the balance of impulse response components corresponding to a direct sound and an initially reflected sound and an Impulse response component corresponding to a reverberation; Fig. 27 Is a schematic diagram showing an 1999~ 9A200 1A 12 SONY ATSUGI No. 0923 P. 12/63 example of a variation of a frequency characteristic corresponding to a variation of a reverberation time; Fig. 28A to 28C are schematic diagrams for explaining a process for adjusting a reverberation time of an original impulse response; Fig. 29A to 29E are schematic diagrams for explaining a process for combining two different impulse responses; Fig. 30 is a block diagram showing an example :e 10 of the structure for composing an initially reflected *6 sound; and Fig. 31A and 31B are schematic diagrams for ~:explaining a process for composing an initially reflected sound.

DETAILED DESCRIPTION OF THE PREFERRED EMBODIMEITS Fig. 1 shows an example of a structure for performing a convolution calculation for an impulse response in time axis direction using an FIR (Finite 20 Impulse Response) filter. Coefficients of an impulse response are required corresponding to samnples of an input digital audio signal. Thus, when the impulse response data of 219 points (524,288 points i- 512 k points) is obtained with a digital audio signal at a sampling frequency of 48 k~z, the reverberation time becomes around 10 seconds.

In Fig. 1, a digital audio signal is supplied 1999 9K20H 13B 13 SONY ATSUGI 5 No. 0923 P. 13/63 from a termina 310. The number of quantizing bits of the digital audio signal is for example 24. The sampling frequency of the digital audio signal Is 48 kHz. The input signal is supplied to 512 k delaying circuits 311 connected in series. Each of the 512k delaying circuits 311 has a delay of one sample.

Output signals of the individual delaying circuits 311 are suppli-ed to respective coefficient multiplying devices 312. Impulse response data of the first point 10 to 512 k-tb. point Is supplied to the delaying circuits 311 with 24 quantizing bits. The coefficient 0 multiplying devices 312 multiply respective output signals of the delaying circuits 311 by respective Impulse response data. The multiplied results are added by an adding device 313. The added result Is supplied as reverberation data against the input data to a terminal 314.

too* Fig. 2 shows another example of the structure the reverberation adding apparatus. In this 20 example, after an input digital audio signal and impulse response data are converted into frequency element data corresponding to Fourrier transform method and then a convolution calculation process is performed for the frequency element data. Referring to Fig. 2, an input digital audio signal is supplied from a terminal 320. Data for samples corresponding to a required reverberation time (namely, data for 512 k 1999~ 9A20B 1AD4135 SONY AISUGI 11j$4y No. 0923 P. 14/63 points) is stored in a memory 321. Data stored in the memory 321 is supplied to an FFT (Fast Fourrier Transform) circuit 322. The FFT circuit 322 performs fast Fourrier transform for the data received from the memory 321 and outputs frequency element data.

Likewise, impulse response data is supplied from a terminal 323. The impulse response data is stored in a memory 324. The impulse response data is supplied to an FFT circuit 325. The FFT circuit 355 performs fast 10 Fourier transform for the impulse response data received from the memory 324 and outputs frequency element data. Since the impulse response data is known, the FFT 325 and the memory 324 may be composed of a ROM 326.

Output data of the FFT circuits 322 and 325 :is supplied to a multiplying device 327. -The multiplying device 327 multiplies the output data of .*.the FFT circuit 322 by the output data of the FFT circuit 325 in such a manner that the frequency components thereof match. The multiplied result is supplied to an IFFT circuit 328. The IFFT circuit 328 performs inversely fast Fourrier transform for the data received from the multiplying device 327 and outputs the resultant data as time axis data to a terminal 329.

As shown In Fig. 1, in the convolution calculation process for an impulse response on time base, a large number of delaying circuits 311 and a 1999 9K20E 13AB1I3 SONY ATSUGI No, 0923 P. 15/63 large number of coefficient multiplying devices 312 are required. In contrast, the hardware scale of the structure shown in Fig. 2 is small. However, since input data corresponding to a desired reverberation time should be temporarily stored to the memory 321 and a convolution calculation process should be performed for the input data, a large delay takes place between the input signal and the output signal. To solve such a problem, a method for dividing an impulse response on 10 time base and performing a convolution calculation process for input data corresponding to each divided ~.impulse response has been proposed (as Japanese Patent Publicized Publication No. 8-501667).

The present invention can be applied to a rever'beration adding apparatus corresponding to any method. In a method according to an embodiment of the present invention, the hardware scale for the convolution calculation process is small. The delay *time is small. An impulse respons e is divided on time.

base. A convolution calculation process for each divided impulse response and audio data Is processed in parallel.

Figs. 3A and 3B show the relation between reverberation according to the invention and reverberation of a conventional recursive filter. Fig.

3A shows reverberation of the conventional recursive filter. The reverberation shown in Fig. 3A is 1999 920B 13140 SONY ATSUGI fi t'4 No, 0923 P. 16/63 generated in the following manner. Direct sound is delayed by a predetermined time period and thereby initial reflection sound is generated. The initial reflection sound is further delayed by a predetermined time period. Reverberation generated by the filter is added. The generated reverberation attenuates corresponding to a simple attenuation curve. In contrast, according to the embodiment of the present invention, sinci reverberation is generated with an 10 impulse response corresponding to really recorded data, the reverberation corresponds to acoustic characteristics of a real hall or the like (namely, it does not correspond to a simple attenuation curve).

Thus, according to the embodiment of the present invention, as mentioned in Fig. 3B, more natural and high quality reverberation can be obtained.

According to the present invention, an impulse response collecting method for obtaining natural reverberation is accomplished. Fig. 4 shows an example of the structure of an impulse response collecting apparatus 97 according to the present invention. In this example, the impulse response collecting apparatus 97 measures an impulse response of a steel-plate echo apparatus 92. The impulse response collecting apparatus 97 can be composed of for example a personal computer. The apparatus 97 generates a signal for measuring an impulse response and outputs 1999 9K200 1AB10 SONY AISUGI fi4 ly No. 0923 P. 17/63 the signal to a measurement object. In addition, the apparatus 97 collects measured results and converts them Into impulse response data. The impulse response data is stored as for example a file.

A measurement signal generating portion generates a TSP (Time Stretch Pulse) signal for measuring an impulse response. The TSP signal is a kind of a -sweep signal. W'hen generating impluse signal is oonv].uated by an inverse function to TSP signal, an impulse signal is obtained. To measure the impulse response, it Is preferred to directly generate the **:impulse signal. However, it is difficult to measure the impulse signal directly generated. Thus, generating impluse signal is convoluted by inverse function to TSP signal, the TSP signal generated by the measurement signal generating portion 90 is supplied to a DIA converter 91. The D/A converter 91 converts the TSP signal as a digital signal Into an analog signal.

The resultant analog signal is supplied to a steelplate echo apparatus 92.

The stLeel-plate echo apparatus 92 generates reverberation with the input TSP signal. The reverberation is output as analog audio signals on left and right channels. The analog audio signals on L and R channels are supplied to an A/D converter 93. The A/D converter 93 converts the analog audio signals on L and R channels into respective digital 19994 MO2B 13B,155 SONY ATSUGI No, 0923 P. 18/63 audio signals. The AID converter 93 samples the digital audio signals at a sampling frequency of 48 kftz or 96 kHz with 24 quantizing bits. Output signals on L and R channels of the A/D converter 93 are supplied to an impulse response collecting apparatus 97. The input signals of the impulse response collecting apparatus 97 are stored to a hard disk unit or a memory (not shown).

-The measurement signal generating portion generates the TSP signal N times. A synchronously 10 adding portion 94 synchronously adds N output signals of the measurement signal generating portion 90. The synichronousl-y adding process is performed in such a manner that the output signals of the synchronously adding portion 94 are synchronized corresponding to the generation timing of the TSP signal. By synchronously adding N signals, only reproducible signals are added.

Thus, since noise components that generate at random are not added, the SIN ration of the resultant signal can be improved. In other words, the SIN ratio of the resultant signal is improved by (10 log N) dB. For example, with N =16, the S/N ratio of the resultat signal is improved by 12 dB.

The synchronously added signals on L and R channels are supplied to an Impulse response converting portion 95. In the impulse response converting portion a convolution calculation with the TSP siginal is performed to a supplied digital audio signal. Thus, 19994 920B 13 15 S SONY ATSUGI No. 0923 P. 19/63 too.

10 *o the TSP signal is converted into an impulse signal.

The measured result is converted into an impulse response corresponding to reverberation generated with the impulse signal. The impulse response data has peak values obtained at intervals corresponding to the sampling frequency. After the A/D converter 93 samples a signal with 24 quantizing bits, the number of quantizing bits becomes 32.

Impulse response data 96L on L channel and impulse response data 96R on R channel that are supplied from the impulse response converting portion 95 are stored to a predetermined record medium such as a CD-ROM or an MD. Alternatively, the impulse response collecting apparatus 97 may be provided with an interface such as Ethernet so as to supply the impulse response data to an external apparatus.

Fig. 5 shows an example of the case that an impulse response is collected in a hall. Referring to Fig. 5, a hall 101 has a stage portion 101A and a guest seat portion 101B. A sound source 102 is disposed at a particular position of the stage portion 101A. The sound source 102 is a dodecahedron speaker of which 12 speakers are disposed in 12 directions on a sphere. A microphone 103L on L channel and a microphone 103R on R channel are disposed in the guest seat portion 101B.

A TSP signal is supplied from an impulse response collecting apparatus 97 to a D/A converter 91.

1999 9K09U 1 3B,1 5 SONY ATSUGJ f No, 0923 P. 20/63 The D/A converter 91 converts the TSP signal as d digital signal Into an analog signal. The analog signal is supplied to an amplifier 100. The amplif ier 100 amplifies the analog signal. The amplified signal I-s supplied to the sound source 102. The sound source 102 reproduces the amplified signal as sound. The reproduced sound is collected by the microphones 103L and 103R. Output signals of the microphones 103L and 103R are supplied to an A/D converter 93. The A/D 10 converter 93 samples the output signals of the microphones 103L and 103R at a predetermined sampling ~.frequency and with a predetermined number of quantizing bits. The resultant signals are supplied as digital audio signals on L and R channels to the impulse response collecting apparatus 97. The process of the d impulse response collecting apparatus 97 is the same as the process of the above-described steel-plate echo apparatus 92.

In thibLs example, the position of the sound source 102 is varied and impulse response data corresponding to varied positions is collected. In addition, the brands of the speakers used as the sound source 102 are also changed and impulse response data corresponding to the individual brands of the speakers is collected. Likewise, the positions and brands of the microphones 103L and 103R are changed and impulse response data corresponding thereto is collected. In 1999 9N2OB 1AO 16 SONY A'ISUGI ff4'No, 0923 P. 21/63 such a manner, a plurality of types of data in the hall 101 are collected. When reverberation is added, one of these types can be selected as variations of reverberation.

On thra other hand, the impulse response data 96L and 96R obtaiined in the impulse response converting portion 95 can be edited. Fig. 6 shows a flow of an editing process of impulse response data. Referring to Fig. 6, impulse response data 110 is supplied to an 10 editing process portion 111. Figs. 7A, 7B, and 7C show examples of the editing process 111. As shown in Fig.

7A, a system delay takes place in datd due to propagation of sound (a system delay portion is denoted 2..by in Fig. 7A). The editing process portion 111 sets the value of the system delay portion to 10"1 so as to remove noise therefrom.

At the last half of the data, a fade-out process is performed so as to converge the last end of .:the data at With the fade-out process, noise of a low level portion of the second half of the signal is removed. Figs. 7B and 7C show examples of the fade-out process.

Fig. 7B shows an example of which the fadeout process is performed corresponding to an attenuation exponential function. In Fig. 7B, the original impulse response is denoted by h(n) and the fade-out function is denoted by (where n 19994 9a20B 13B160 SONY ATSUGI ff4i\t Y No. 0923 P. 22/63 represents a point of impulse response data). It should be noted that a point of impulse response data corresponds to a sampling point of a digital audio signal. In the case of n s 0, the relation of Fo(n) 1 is satisfied. In contrast, in the case of n 0, represents an attenuation exponential function as shown in Fig. 7B.

Output data x(n) is represented by the following expression x(n) h(n) F 0 (1) where a is the number of samples corresponding to the position of direct sound in the original impulse response. The fade-out process is performed after the position of direct sound. If the fade-out process is 15 performed at the position of the direct sound (namely, n the level of the direct sound also decreases.

It should be noted that the fade-out function is not limited to an attenuation exponential function.

For example, as shown in Fig. 7C, the fade-out function 20 may be a function having a linear attenuation characteristic.

The number of points of the impulse response data can be adjusted corresponding to the process capability of the reverberator that adds reverberator to an audio signal with such data of the fade-out process. In the case that the number of points of impulse response data is limited to a predetermined 19994 9K20U 1316 SONY ATSUGI 0923 P. 23/63 value (for example, 256 kc points 262,144 points), as shown in Fig. 7A, at the 128 k-th point, the fade-out process is started and at the 256 k-th point, the data becomes As an example of the editing process ill, a l~evel. adjusting process may be performed. The edited impulse response data is recorded as an FIR filter coefficient 112 for a convolution calculation process of the FIR filter to for example a CD-ROM In addition to impulse response data, data that represents a feature of the impulse response data is stored as additional data to the CD-ROM 45. The *~*additional data contains for example data that :represents a reverberation time of an impulse response.

start/end point values of a direct sound and an initially reflected sound (for example, the value of an input point Is and a start point value of a *.reverberation. For example, a plurality of types of impulse responses collected by a steel-plate echo apparatus, one type of impulse response of a hall, and additional data corresponding to each impulse response are recorded on the CD-ROM 45. In the case of the steel-plate echo apparatus, since the reverberation time can be varied, a plurality of types of impulse responses corresponding to a plurality of reverberation times are collected.

Fig. 8 shows an example of the structure of a 19994 920B 13B17 SONY ATSUGI imt~ No. 0923 P. 24/63 reverberator that performs a convolution calculation process with impulse response data generated in the above-described manner. Referring to Fig. 8, a digital audio signal is input from an input terminal 120. The input signal is supplied to a multiplying device 126.

In addition, the input signal is supplied to a predelaying portion 121. The pre-delaying portion 121 delays the input data. Output data of the pre-delaying portion 121 is supplied to a convolution calculation process portion 122.

The convolution calculation process portion 122 is composed of FIR filters on L and R channels (namely, a filter 122L and a filter 122R). The impulse response data 96L and 96R generated by the impulse response collecting apparatus 97 are supplied as FIR filter coefficients on L and R channels from terminals 123L and 123R, respectively. The impulse response data 96L and 96R are read from for example a CD-ROM (not shown).

20 The filters 122L and 122R perform convolution calculation processes for the input digital audio signals with the impulse response data 96L and 96R, respectively. Thus, reverberation corresponding to the impulse response data 96L and 96R is generated. Output signals of the filters 122L and 122R are supplied to multiplying devices 124L and 124R, respectively.

The multiplying devices 124L and 124R, the 1999 9N2OB 1 A 171~ SONY ATSUGI #o No. 0923 P. 25/63 above-described multiplying devices 126, and the adding devices 128L and 128R compose a mixer of the original sound (a dry component) and reverberation (a wet component). Coxresponding to the ratio of the original sound and reverberation supplied to the terminals 127 and 125, the multiplying device 126 and the multiplying devices 124L and 124R adjust the input digital audio signal and the output signal of the convolution calculation process portion 122. The adding devices 128L and 12SR add these signals. Thus, the output signal on L channel and the output signal on R channel are supplied to output terminals 129L and 129R, respectively.

:Fig. 9 shows a detailed example of the structure of the reverberator. In the reverberator 1, digital audio signals of two channels (channel 1 and channel 2) are input from a digitztl audio input 2..terminal 10 corresponding to AES/EBU (Audio Engineering Society/European Broadcasting Union) standard. The digital audio signals received from the input terminal 10 are supplied to an input switcher 12 through a digital inputting portion 11.

In this example, the sampling frequency and the number of quantizing bits of the input digital audio signals are 48 kHz and 24 bits, respectively.

When an option board 50 (that will be described later) is connected to the reverberator 1, the sampling 19994 9120B 13B18 SONY ATSUGI fi" '4t No. 0923 P. 26/63 frequency of digital audio signals handled by the reverberator 1 can be doubled (namely, 96 kHz). In addition, digital audio signals at a sampling frequency of 44.1 kHz can be handled by the reverberator 1. In this case, when the option board 50 is connected to the reverberator 1, signals at a sampling frequency of 88.2 kHz can be handled by the reverberator 1.

-When analog audio signals are input to the reverberator 1, analog audio input terminals 13L and 13R are used. Audio signals on L and R channels are input from the input terminals 13L and 13R, respectively. An A/D converter 14 samples the audio signals at a sampling frequency of for example 48 kHz with 24 quantizing bits so as to convert these signals into respective digital audio signals. Output signals of the A/D converter 14 are supplied to an input switcher 12.

S..The input switcher 12 switches a source of input audio signals under the control of a controller 20 40 (that will be described later) or with a manual switch. Output signals of the input switcher 12 are supplied to a DSP (Digital Signal Processor) 30 through a path 31.

The input switcher 12 switches a source of input audio signals under the control of a controller (that will be described later) or with a manual switch. Output signals of the input switcher 12 are i999 9H20B 13h1SH SONY ATSUGI No, 0923 P. 27/63 supplied to a DSP (Digital Signal Processor) 30 through a path 31.

The DSP 30 has a DRAM (Dynamic Random Access Memory) and performs various control processes for input/output digital audio signals corresponding to a program received from the controller 40. The DSP supplies Input digital audio signals to DSPs 32A to 32K that perform convolution calculation processes for obtaining impulse response data corresponding to a predetermined process. In addition, the DSP generates initial reflection sound corresponding to the input signals. Moreover, the DSP 30 receives the result of the convolution calculation process for the impulse response data from a DSP 34 (that will be is:. 1 described later).

The DSPs 32A to 32K divide input digital audio signals into blocks with predetermined sizes and perform convolution calculation processes for the divided blocks with the pre-supplied impulse response data. The DSPs 32A to 32K have respective DRAMs with relevant capacities corresponding to the number of samples to be processed. In this example, each of the DSPs 32A to 32H has one DRAM. The DSP 321 has two DRAMs. Each of the DSPs 32J a.nd 32K has one DRAM with a capacity of 16 Mbits.

The results of the convolution calculation processes for the impulse response data for individual 19994 920B 13A180 SONY ATSUGI f i'4Y y No, 0923 P. 28/63 blocks performed by the DSPs 32A to 32K are added by an adding device 33. The added result is supplied from the adding device 33 to the DSP 30 through a DSP 34.

When the DSP 34 detects an overflow in the added result, the DSP 34 sets the data of the overflow to a predetermined value.

The DSP 30 combines the input digital audio signals, the initial reflection sound, and the result of the convolution calculation process for the impulse response data received from the DSP 34 so as to add reverberation to the input digital audio signals.

Output data 35 of the DSP 30 is supplied to an output switcher 18.

The generated reverberation and non-processed input digital audio signals are referred to as "wet component" an "dry component", respectively. The DSP can vary the mixing ratio of the wet component and the dry component on each of L and R channels. In addition, the DSP 30 adjusts the levels of the output 20 signals.

A clock signal FS or 2FS with a frequency corresponding to the sampling frequency of the handled digital audio signals is supplied to the DSP 30. The DSP 30 processes signals corresponding to the clock signal FS or 2FS.

The output switcher 18 selects an output destination of output signals under the control of the 1999 9A20G 13B19 SONY ATSUGI fitt'4 No. 0923 P. 29/63 controller 40 or with a manul switch. The output signals are digital audio signals or analog audio signals. The output switcher 18 supplies digital audio signals of two channels to output terminal corresponding to the AES/EBU standard through a digital outputting portion 19. The digital audio signals that are output from the output switcher 18 are supplied to a D/A converter 21. The D/A converter 21 converts the digital audio signals received from the output switcher 18 into analog audio signals. The analog audio signals on L and R channels are supplied to analog output terminals 22L and 22R, respectively.

In this example, the input terminal 10, the input terminals 13L and 13R, the output terminal 15 and the output terminals 22L and 22R are of cannon type having three signal lines of hot, cold, and ground lines.

The output switcher 18 allows the reverberation adding process in the reverberator 1 for 20 the input audio signals to be bypassed. When the reverberation adding process is bypassed, the input digital audio signals are directly supplied to the output switcher 18 through the input switcher 12 and a bypass path 17.

All portions of the reverberator 1 are controlled by the controller 40. The controller comprises for example a CPU (Central Processing Unit), 1999 9H20B 13 1 9 SONY ATSUGI No. 0923 P. 30/63 a RAM (Random Access Memory), a ROM (Redd Only Memory), and predetermined input/output interfaces. The ROM stores a boot program for starting up the system and serial number. The RAM is a work memory with which the CPU operates. An external program is loaded to the

RAM.

The control~ler 40 is connected to a bus 41 with for example eight bits parallel. The bus 41 is connected to the DSP 30, 32A to 32H, and 34. The controller 40 communicates with each of the DSPs :32A to 32H, and 34 through the bus 41. Thus, the controller 40 supplies programs to the DSPs 30, 32A to 32H, and 34. In addition, the controller 40 exchanges data and commands with the DSPs 30, 32A to 32H, and 34.

As dejoribed above, the input switcher 12 and the output switcher 18 are connected to for example the bus 41 (not shown) and controlled by the controller For example, a display unit 42 that is a full-dot LCD (Liquid Crystal Display) is connected to .5::20 the controller 40. The display unit 42 displays data generated by the controller The inputting portion 43 has a plurality of inputting means (for example, a rotary encoder for Inputting data corresponding to the rotation angle and a plurality of push switches). By operating these inputting means, relevant control signals are supplied from the inputting portion 43 to the controller 1999~ H200 A 13 95 SONY ATSUGI No. 0923 P. 31/63 Corresponding to the control signals, the controller supplies predetermined programs and parameters to the DSPs 30, 32A to 32H, and 34.

The reverberator 1 has a CD-ROM (Compact Disc-ROM) drive 44. A CD-ROM 45 is loaded to the CD- ROM drive 44. Data and programs are read from the CD- ROM 45. The data and programs that have been read from the CD-ROM 45 are supplied to the controller For example, impulse response data has been recorded on the CD-ROM 45. The impulse response data is read from the CD-RM 45 and supplied to the controller 40. The data Is supplied from the controller 40 to the DSPs 32A to 32K. The DSPs 32A to 32K perform convolution calculation processes for impulse response data corresponding to the received to impulse response data.

The reverberator 1 has an external interface *too.*MIDI (Musical. Instrument Digital Interface). An MIDI signal is supplied from an MIDI input terminal 46 to 20 the controller 40. Corresponding to the MIDI signal, the controller 40 controls a relevant function of the reverberator 1. The controller 40 generates and outputs the MIDI signal. The controller 40 can edit the MIDI signal received from the MIDI input terminal 46 and outputs the resultant signal. The MIDI signal is supplied from the controller 40 to an external apparatus through the MIDI output terminal. 47. An MIDI 1999~ 9A20B 1A M05 SONY ATSUGI No, 0923 P. 32/63 through-terminal 48 is used to directly output the MIDI signal received from the MIDI input terminal 46.

W~hen the option board 50 is connected to the reverberator 1, extended functions can be obtained. As an example of the extended functions, two more digital audio signals at a sampling frequency of 48 kHz can be handled. The digital audio signals of two channels (channels 3 and 4) are received from a terminal through the option board 50. The digital audio signals are supplied to the Input switcher 12 through the digital Inputting portion 16. In addition, digital audio signals of two channels corresponding to a process of the option board 50 are output to a terminal 24 through the digital outputting portion 23. The 15 digital audio signals are output to an external apparatus from the terminal 24 through the option board As another example of the extended functions, audio signals o2 two channels (channels 1 and 2) can be 20 handled at a sampling frequency of 96 kclz that is twice as high as the normal sampling frequency.

The option board 50 and the reverberator 1 are connected with terminals 51 to 56, 15, and 24.

Fig. 10 shows an example of the structure of the option board 50. The option board 50 performs an extended convolution calculation process for impulse response data using the DSPs 32A to 32K and the adding device 19994 9A208 13m20m SONY ATSUGI fi 4y No. 0923 P. 33/63 33. Thus, the option board 50 has DSPs 32L, 32M, and to 60L, an adding device 61, and a DSP 62. The DSPs 32L, 32M, and 60A to 60L correspond to the DSPs 32A to 32K shown in Fig. 10, respectively. In addition, the DSP 62 corresponds to the DSP 34 shown in Fig. A bus 41' of the option board 50 is connected to the bus 41 of the reverberator 1 through a terminal 56. The DSPs 32L, 32M, and 60A to 60L of the option board 50 can communicate with the controller 40 through the bus 41'.

•The DSPs 32L and 32M have eight 16-Mbit DRAMs each and perform convolutional calculation processes along with the DSPs 32A to 32K. Input digital audio S: 15 signals are supplied from the DSP 30 to the DSPs 32L and 32M through the terminal 53. The results of the convolution calculation processes of the DSPs 32L and

S

32M are supplied to the adding device 33 through the terminals 54 and 55, respectively. The adding device .i 20 33 adds the results of the convolution calculation processes of the DSPs 32L and 32M along with the results of the convolution calculation processes of the other DSPs 32A to 32K.

On the other hand, the DSPs 60A to perform the convolution calculation processes in parallel with those of the DSPs 32A to 32M shown in Fig. 9. Input digital audio signals are supplied from 1999 9A20B 13O 2If SONY ATSUGI W#4t' No. 0923 P. 34/63 0O@* Oe**

S

*5*S *5 4 the DSP 30 to the DSPs 60A to 60M through the terminal 51.

When digital audio signalas of four channels (channels 1 to 4) are processed with the option board the DSPs 32A to 32M perform convolution calculation processes for digital audio signals of channels 1 and 2, whereas the DSPs 60A to 60M perform convolution calculation processes for digital audio signals of channels 3 and 4. When digital audio signals at a sampling frequency of 96 kHz are handled, pairs of DSPs (for example a pair of the DSPs 32A and 60A, a pair of the DSPs 32B and 60B, and a pair of the DSPs 32M and 60M) that receive blocks with respective samples can perform respective convolution calculation processes in parallel at double speed.

The results of the convolution calculation processes of the DSPs 60A to 60M are supplied to the adding device 61. The added result of the adding device 61 is supplied to the DSP 62. As with the DSP 34, the DSP 62 performs an overflow process. The resultant signais are supplied to the DSP 30 through the terminal 52. The DSP 30 adjusts the ratio of a dry component and a wet component and the mixing ratio of signals of individual channels and supplies the resultant data to the output switcher 18.

The option board 50 also has a digital audio signal input terminal 63 and a digital audio signal 15 *5*S *r S SS S *5 S..2 19M9 9A206 I3B,215 SONY ATSUG fm1 No. 0923 P. 35/63 output terminal 64 corresponding to AES/EBU standard.

Signals of two channels (channel :3 and 4) are input to the input terminal 63. The input signals are supplied to the input switcher 12 through the terminal LikewiSe, output signals of two channels (channels 3 and 4) are supplied from the output switcher 18 to the option board 50 through the terminal 24 and then output from the output terminal 64. In this example, the terminals 63 and 64 are of cannon type.

Fig. 11 shows an example of a front panel 200 of the reverberator 1. Four mounting holes are formed at four corners of the front panel 200. With the four mounting holes, the reverberator 1 can be mounted to a rack. A power switch 201 Is disposed on the left of is1 the panel 200. Below the power switch 201, a CD-ROM loading portion 202 is disposed. A CD-ROM 45 is loaded to a CD-ROM drive 44 through the CD-ROM loading portion 202. With a switch 205, the CD-ROM 45 is loaded and unloaded to/from the CD-ROM drive 44 through the CD-ROM loading portion 202.

A display portion 203 is disposed at a nearly center position of the panel 200. The display portion 203 corresponds to the LCD 42 shown In Fig. 9. On the right of the display portion 203, a rotary encoder 204 is disposed. Below the display portion 203, function keys 206, 207, 208, and 209 are disposed. With the rotary encoder 204 and the function keys 206 to 209, 19994 9R2OE 13,21 SONY ATSUGI ~~YNo. 0923 P. 36/63 the user can select one of the functions of the reverberator 1 and input data thereto.

The display portion 203 displays various data corresponding to the selected function. In this example, the display portion 203 displays parameters corresponding to a selected reverberation type. The display portion 203 is largely separated into a displ.ay area 210 and a display area 211. The display area 210 visually displays parameters corresponding to the *:10 selected reverberation type. The display area 211 :***:displays parameter names and parameter values.

Data displayed In the display area 211 corresponds to the function switches 206 to 209 disposed below -the display portion 203. When one of 15 the function switches 206 to 209 is pressed, a parameter displayed above the function switch that has been pressed is selected. By turning the rotary encoder 204, the value of the selected parameter is varied. Another page can be displayed on the display portion 203. Ost another page, the value of another parameter can brt varied.

In the embodiment, the display area 210 displays ripples corresponding to a parameter that is being currently set. Thus, the user can visually know the effect of reverberation (spatial impression) corresponding to the parameter value. Figs. 12A to 12H and Figs. 13A to 13H show examples of ripples displayed 19994 9K00E 1 A 2N SONY ATSUGI i '1ty No, 0923 P. 37/63 -in the display area 210. As the reverberation time is prolonged, the number of wavers of ripples is increased in the order from Figs. 12A to 12H to Figs. 13A to 13H.

In this example, the ripples are displayed in 16 levels of the minimum value to the maximum value of the reverberation time. The ripples in 16 levels are proportional to the reverberation time. Ripple display data is stored in the CD-ROM 45. When the reverberator I. gets started, ripple display data Is read from the *:10 CD-ROM 45 and stored in the RAM of the controller Alternatively, the ripple display data may be prestored In the ROM of the controller 40. Once the parameter value of the reverberation time is set, ripples are displayed corresponding to the parameter 15 value that has been set.

When ripples are displayed in the display area 210. the user can visually know the effect of the reverberation that has been set. In other words, the user can visually know the spatial impression of the reverberation with ripples displayed in the display area 210.

In the example, ripples are displayed in the upper right direction of the display area 210.

However, ripples may be displayed with a different pattern. Figs. 18A, 18B, and 18C show other examples of ripples displayed in the display area 210. The center point and spreading direction of ripples can be 1994~ 9K05E 1AB225 SONY ATSUGI No. 0923 P. 38/63 freely set. The center position of ripples can be set at the left end of the display area 210 (see Fig. 18A).

Alternatively, the center position of ripples can be set at the center of the display area 210 (see Fig.

18B). A section of ripples may be displayed in the display area 210 (see Fig. 18C). In addition, the shape of ripples can be varied corresponding to the selected reverberation type. In this example, ripples are displayed as a still pattern. However, when a .:10 plurality of pages of ripple display data is prepared for each parameter and they are successively displayed, ripples can be displayed as an animated pattern.

Next, convolution calculation processes for impulse response data performed by the DSPs 32A to 32M 15 and the DSPs 60A to 60M will be described. In this example, for simplicity, only convolution calculation processes performed by only the DSPs 32A to 32K without the use of the option board 50 will be described.

Fig. 15 shows a process performed by each of the DSPs 32A to 32K. Impulse response data is read from for example the CD-ROM 45 under the control of the controller 40 and supplied to the DSPs 32A to 32K. The impulse response data that Is read from the CD-ROM is stored to the DRAMs of the DSPs 32A to 32K. Each of the DSPs 32A to 32K divides impulse response data at predetermined intervals on time axis corresponding to the process block sizes assigned thereto.

19994 9A20B IA23 SONY ATSUGI ~iW 4"t No. 0923 P. 3 9/633 10 rr r For simplicity, a DSP 32 that represents all the DSPs 32A to 32K will be described. The data unit of impulse response data processed by the DSP 32 is denoted by N. In this example, since the DSP 32A performs a convolution calculation process for impulse response data of 128 points, the data unit N is 128.

In the following description, one word corresponds to data of one sample of a digital audio signal. Thus, one word has a time period of (1/sampling frequency).

The number of quantizing bits of digital data is 24 bits.

Input data supplied to the DSP 32 is divided as- block data of N words. Thus, the time period for the first N words is the time period for inputting the data. The input data of N words is stored to the DRAM of the DSP 32. In the time period of the next N words, a convolution calculation process is performed for impulse response data corresponding to input data of N words stored in the DRAM. After the convolution calculation process is completed, the result of the process for N words is output. Thus, in the process for N words, output data is delayed by 2N words to input data.

Fig. 16 shows the process of the DSP 32 in detail. The DSP 32 performs a convolution calculation process for impulse response data by known recursive convolution overlap save method.

1999 9A2OE 1AD235 SONY ATSUGI ~B No. 0923 P. 40/63 10 In other words, as shown in Fig. 16, an n-tb block BOB and an (n-1)-th block 80A that immediately precedes the block 805 are supplied every N words on time axis. The n-th block 8OB and the (n-1)-th block BOA are converted into frequency element data 81 composed of a real part 81A of (N+I1) words and an imaginary part 81B of words by DFT (Discrete Fourier Transform) method.

On the other hand, real data 82A and zero data 82B of impulse response data 82 have been converted into frequency element data 83 composed of a real part 83A of words and an imaginary part 83B of words by DFT method.

The real part and imaginary part of frequency element data 81 of the input data and the real part and imaginary part of the frequency element data 83 of the impulse response are multiplied, respectively. The multiplied results of the samue frequency components are added. Namely, filter process (convolution calculation process) is performed. Thus, frequency element data 84 composed of a real part 84A of words and an imaginary part of words is obtained. The IDFT process that is an inverse.process of the DFT process is performed for the frequency element data 84. Thus, data 86 of 2N words on time axis is obtained.

As the results of the IDFT process, as represented by data 85, 86. and 87 shown in Fig. 16.

1999 9KOB IA323t SONY ATSUGI ~fiN".4Y No. 0923 P. 41/63 10 *o *oco 1

-L

blocks of 2N words are obtained at intervals of N words. The first N word portions 85A, 86A, and 87A of the data 85, 86, and 87 are discarded. Thus, output data of (n-l)-th block, n-th block, and (n+l)-th block is obtained. Consequently, the n-th output data is delayed by 2 blocks against the n-th input data.

When the block size is increased, a convolution calculation process is performed for more impulse response data. Thus, a longer reverberation time can be obtained. However, as described above, an output block is delayed by two blocks against an input block. Consequently, when the size of each block is increased, the delay time of an output component of the reverberation process adversely becomes long. To solve such a problem, according to the embodiment of the present invention, a process for obtaining a desired reverberation time is performed in parallel for a plurality of blocks each of which is composed of a predetermined number of points (words).

Figs. 17 and 18 show a convolution calculation process according to the embodiment of the present invention. In the convolution calculation process, a digital audio signal is divided to a plurality of blocks. For example, a convolution calculation process for 216 words (256 k words) is considered. In this case, a convolution calculation process is performed for a digital audio signal with 19994 9A200 1AB20~ SONY ATSUGJ *I~ No. 0923 P. 42/63 a. a* a.

15 impulse response data of 256 k words (256 k points).

When the sampling frequency is 48 kliz, a reverberation time of around 5.3 seconds is obtained. When the sampling frequency is 44. 1 kHz, a reverberation timne Of around 5.9 seconds is obtained.

As shown in Fig. 17, the impulse response data of 256 k words is divided into two portions. The temporally earlier portion of the two portions is further divided into two portions. In such a manner, the earlier portion on time axis is successively divided into two portions. The later portion on time axis is successively divided into two portions. Thus, two blocks with the same size are formed.

Fig. 18 is an enlarged view showing a top portion A of 8 k words shown in Fig. 17. Likewise, the portion A is divided into two portions. The first 256words portion is divided into two blocks each of which has 128 words. A convolution calculation process is performed for impulse response data of the two blocks.

Thus, the reverberation component is delayed by 256 words of the first portion. However, when the sampling frequency is 48 kHz, the delay is as small as 5 msec.

Thus, it does not adversely affect reverberation.

In the case of words (256 k words), a pair of two 2 7 words (128 words) blocks, a pair of two words (256 woi~ds) blocks, a pair of two 29 words (512 words) blocks, a pair of two 210 words (I k words) 1999 9K00U 1Ah20 SONY ATSUGI f f No. 0923 P. 43/63 15 blocks, d pair of two 211 words (2 kc words) blocks, a Pair of two 21-2 words (4 kc words) blocks, a pair of two 2 13 words (8 kc words) blocks, a pair of two 214 words (16 k words) blocks, a pair of two 215 words (32 kc words) blocks, and a pair of two 216 words (64 kc words) blocks (namely, pairs of two 24 words blocks) are formed.

Each of the DSPs 32A to 32K performs a convolution calculation process for the relevant pair with the same block size. In other words, as shown in Figs. 17 and 18, the DSPs 32A to 32K divide their input data as follows. The DSP 32A divides the input data into blocks each of which is composed of 125 words.

The DSP 32B divides the input data into blocks each of which is composed of 256 words. The DSP 32C divides the input data into blocks each of which is composed of 512 words. The DSP 32D divides the Input data into blocks each of which Is composed of 1 k words. The DSP 32E divides the input data into blocks each of which Is composed of 2 k words. The DSP 32F divides the input data into blocks each of which is composed of 4 Ic words. The DSP 32G divides the input data into blocks each of which is composed of 8 kc words. The DSP 32H divides the input data into blocks each of which Is composed of 16 k words. The DSP 321 divides the input data Into blocks each of which is composed of 32 kc words. Each of the DSPs 32J and 32K divides the input 1999 9A20B 13A240 SONY ATSUGI ffi4y No. 0923 P. 44/63 00 0 15 data into blocks each of which is composed of 64 k words.

For a convolution calculation process for blocks in the range from 128 words to 32 k words, each DSP performs the process for a pair of blocks with the same block size on time division basis.

In other words, each of the DSPs 32A to 32K performs a convolution calculation process for divided block data with relevant impulse response data. The second pair member of each pair is delayed by one block against the first pair member. Thus, each of the DSPs 32A to 32K successively outputs two blocks with the same block size. The adding device 33 adds the output blocks of the DSPs 32A to 32K and generates reverberation data 88.

When input data is successively'processed by the DSPs 32A to 32K in their assigned periods and the results are added, reverberation can be added to the successive data.

Fig. 19 shows an example of the structure of a convolution calculation filter 70 used in each of the DSPs 32A to 32K. The convolution calculation filter performs a convolution calculation process. The convolution calculation filter 70 is accomplished by a predetermined program supplied from the controller to the DSPs 32A to 32K. Referring to Fig. 23, a digital audio signal is input from a terminal 71. The 19994 9A200 13B25 SONY ATSUGI fi 4 No. 0923 P. 45/63 input digital audio signal is supplied to a DFT circuit 72. The DFT circuit 72 converts the digital audio signal on time axis into frequency element data.

Output data of the DFT circuit 72 is supplied to a multiplying device 74 and a delaying circuit 73.

The delaying circuit 73 delays the input digital audio signal by N words. In other words, the DSPs 32A, 32B, 32C, 32D, 32E, 32F, 32G, 32H, 321, and 32K have delay amounts of N 128, 256, 512, 1 k, 2 k, 10 4 k, 8 k, 16 k, 32 k, and 64 k, respectively. Data delayed by the delaying circuit 73 is supplied to a multiplying device 76.

The multiplying device 74 receives a filter coefficient A from a terminal 75. The filter 15 coefficient A is impulse response data that has been processed by DFT method. The multiplying' device 74 multiplies the output data of the DFT circuit 72 by the relevant frequency element of the filter coefficient A.

Likewise, the multiplying device 76 performs the same process as the multiplying device 74. In other words, the multiplying device 76 receives a filter coefficient B from a terminal 77. The filter coefficient B is impulse response data that has been processed by DFT method. The multiplying device 76 multiplies the output data of the delaying circuit 73 by the relevant frequency element of the filter coefficient B.

The multiplied results of the multiplying 19994 9MEO 13A25 SONY ATSUGI Y No. 0923 P. 46/63 10

S

S

S.

*SS.SS

S

S

*5*S *5 5 S

S

15 S S

S

devices 74 and 76 are added by an adding device 78.

The added result is supplied to an IDFT circuit 79.

The IDFT circuit 79 converts the frequency element data into data on time axis and outputs the resultant data from a terminal Since the convolution calculation filter performs a convolution calculation process for two blocks of -data of which one block is delayed by N words (namely, one block) against the other block and outputs data of two blocks. As was described with reference to Fig. 16, the first pair member of each pair is discarded.- Fig. 20 shows a process on time axis performed by the convolution calculation filter shown In Fig. 19. The left end and right end of Fig.

20 show input data and output data, respectively. It is assumed that In Fig. 20, time passes downwards.

Referring to Fig. 20, a plurality of filters 70 are shown. However, in reality, these processes are performed by one filter 70 at different timings. Thus, the result of the DFT process at the preceding timing is delayed by th~e delaying circuit 73. The delayed result is used for the filter process at the next timing. Consequently, output data delayed by two blocks against input data is successively obtained.

Fig. 21 Is a functional block diagram showing an outline of parallel processes of the DSPs 32A to 1999 9206 13A~260 SONY ATSUGI ~f~iW No. 0923 P, 47/63 r .o 5 S*

J

32K. Input data is supplied to the DSPs 32A to 32K in parallel. The DSPs 32A, 32B, 32C, 32D, 32E, 32F, 32G, 32H, 321, 32J, and 32K perform convolution calculation processes for N 128 points, N 256 points, N 512 points. N 1 k points, N 2 k points, N 4 k points, N 8 k points, N 16 k points, N 32 k points, and K 64 k points, respectively. Each of the calculated results of the DSPs 32A to 32K is delayed by 2 N words.

The delayed results are supplied to an adding device 22.

For example, the DSP 32A divides input data into blocks each of which is composed of N 128 words, performs a convolution calculation process for the divided blocks, and outputs the calculated result that has been delayed by 2N words against the input data.

Thereafter, the DSP 32A receives the next blocks each of which is composed of N words and repeats the same process for the blocks. This process applies to each of the DSPs 32B to DSP 32K.

Next, as features of the present invention, an editing process and a combining process for impulse responses will be described. As described above, data of a plurality of impulse responses and additional data corresponding thereto are read from the CD-ROM 45 to the reverberation adding apparatus. For example, a plurality of types of impulse responses collected by a steel-plate echo apparatus, one type of impulse 1999 9H20B I26 SONY ATSUGI 4* 4 fY No. 0923 P. 48/63

C

a U U. U U V 15

U

U

response collected from a hall, and additional data corresponding to each impulse response are read from the CD-ROM 45 to the reverberation adding apparatus.

As described above, as the additional data, data that represents a reverberation time of each impulse response, start/end point values of a direct sound and an initially reflected sound (for example, the value of an input ploint is "1" 1 and a start point value of each reverberation are used. Impulse responses are edited and combined by a software process of the controller Next, with reference to Figs. 22, 23, 24, and 25, as a first example of the impulse response editing process, a process for removing a direct sound and an Initially reflected sound will be described- When a direct sound and an initially reflected sound are contained in a reverberation, a process as a simulator of a hail is performed. Thus, a direct sound and an initially reflected sound should be removed from a reverberation. Fig. 22 shows original impulse response data of a hall. The original impulse response data is read from the CD-ROM 44. Referring to Fig. 22, the sampling frequency of the impulse response data Is 96 kHz and 64 k points of 512 kc points are shown. Fig. 23 is an enlarged view showing an initial response portion (4 k points) of the impulse response data shown in Fig.

22.

1999 9K206 13B26 SONY ATSUGI No, 0923 P. 49/63 10

S

'9 *9

S

*9 S. 15

S

Figs. 24 and 25 show impulse response data of which impulse response components corresponding to a direct sound and an initially reflected sound are removed from original impulse response data, respectively. Wh~en Impulse response components corresponding to a direct sound and an initially reflected sound are set to the direct sound and the initially reflected sound can be removed from the original impulse response data. Fig. 24 shows an impulse response component for 64 kc points against the impulse response data shown in Fig. 22. Fig. 25 is an enlarged view showing an impulse response component for 4 kc points against the impulse response component shown in Fig. 24. In these drawings, upper impulse response components and lower impulse response components correspond to L channel and R channel, respectively.

As described above, since additional data corresponding to the impulse response data shown in Fig. 22 contains information of start/end points of impulse response components corresponding to a direct sound and an initially reflected sound, the positions of the impulse response components corresponding to the direct sound and the initially reflected sound can be specified. When a fade-in operation is performed from the specified position, the initially reflected sound can be removed. However, a direct sound and an initially reflected sound can be removed without need e.g.

9SS* *5 *9

S.

1999 9K200 1 3B,2 75 SONY ATSUGI No, 0923 P. 50/63 to use the additional information. In other words, a portion after the maximum value of a waveform of an envelop of an impulse response until the slope of the envelop is inverted to plus is estimated as impulse response components corresponding to a direct sound or an initially reflected sound. Since a direct sound largely affects an reverberation, only the direct sound may be remioved.

Next, with reference to Figs. 26, a second example of the impulse response editing process will be described. In the second example, the balance of a direct sound, an initially reflected sound, and a ago 0reverberation is adjusted. Fig. 26A shows original s.:*.impulse response data. In Fig. 26A, the start point of 15 a direct sound and an initially reflected sound (denoted by Ni). the end point thereof (denoted by N2), the start point of a reverberation (denoted by N3), and a reverberation time (denoted by To) are represented as additional data.

With the additional data, original impulse response data is separated into Impul~se response components corresponding to a direct sound and an initially reflected sound (see Fig. 268) and an impulse response component corresponding to a reverberation (see Fig. 26C). In the example shown in Figs. 26.

since the impulse response components corresponding to the direct sound and the initially reflected sound are 1999 9A206 13 ,27 SONY ATSUGI iW~t t4 No. 0923 P. 51/63 1O S 25 oooo o very large against the impulse response component corresponding to the reverberation, the levels of the impulse response components corresponding to the direct sound and the initially reflected sound are lowered.

The resultant impulse response components are combined with the impulse response component corresponding to the reverberation. Fig. 26D shows the combined impulse response data. Alternatively, the level of the impulse response component corresponding to the reverberation can be raised. As another alternative method, the balance of impulse response components may be adjusted using a cross fade method. In other words, impulse response components corresponding to a direct sound and an initially reflected sound are faded out, whereas an impulse response component corresponding to a reverberation is faded in. The fade-in/out curves may be straight lines or exponential curves.

Next, with reference to Figs. 27 and 28, a third example of the impulse response editing process will be described. In the third example, the reverberation time is adjusted. In the case of a steel-plate echo apparatus, when the reverberation time is varied, the frequency characteristic of the reverberation adding apparatus is also varied. Fig. 27 shows an example of measured results of frequency characteristics of different reverberation times at 0.2 sec steps (1.5 sec, 1.75 sec. 2.0 sec, 3.5 sec).

1999$ 9A206 13DM285t SONY ATSUGI ~i '4 No. 0923 P. 52/63 10 15 1 Thus, to add a reverberation equivalent to that of a steel-plate echo apparatus to an audio signal, impulse responses corresponding to respective reverberation times are required. Thus, since the data amount for the impulse responses becomes huge, the storage capacity of the memory adversely increases.

To solve such a problem, as shown in Figs.

28, impulse responses corresponding to representative reverberation times is provided. Impulse responses corresponding to other reverberation times are obtained by calculations. In reality, the range of a reverberation time is divided into a plurality of time zones. In each time zone, an impulse response corresponding to a desired reverberation time is obtained with the representative impulse responses.

Fig. 28A shows an impulse response at reverberation time To 4 sec. Fig. 28B shows an impulse response at reverberation time To 3 sec. Fig. 28C shows an impulse response at reverberation time To 2 sec.

With an representative impulse response at To 4 sec, impulse responses corresponding to reverberation times 2.25 sec. 2.5 sec, and 2.75 sec are obtained. In addition, with a representative impulse response at Tr 2 sec, impulse responses at 1.25 sec, 1.5 sec, and 1.75 sec are obtained. Thus, in addition to the three representative impulse responses, impulse responses corresponding to nine types of reverberation times can lee 19994 9U20B 13BW28 SONY ATSUGI 4 'JNo. 0923 P. 53/63 be obtained.

Next, a calculating method for obtaining an impulse responsG In a short reverberation time using a representative impulse response will be described. In this case, an original. impulse response is multiplied by an exponential function so as to shorten a reverberation time. The shortened reverberation time is given a&s follows.

Trev /TO) (-60 Tx) -60 ((To Tx) /To -Tx)] 60 (To Tx) ITo Tx)] where Trev: shortened reverberation time; To: reverberation time of original imp ulse response data; and Tx: exponential. reverberation time multiplied by :15 impulse response data.

Thus, the following relation is satisfied.

Trev m(To s Tx) (To Tx) The exponential reverberation time can be :expressed as follows.

Trev *(To Tx) To Tx (Trev -To) 9 Tx =-Trev To Tx (Trev To) (To Trev) Next, an exponential function of which the reverberation time becomes Tx is obtained.

X -(Fs 9 Tx) -60 dB 10 (Fs *Tx) log X -log (10 -3 log X -3 (Fs *TX) 1999 9 206 13 KH 199~ 92O~1Th8~ SONY ATSUGI 1iWY No, O923 P. 54/63 10 15 x 10 -3 (Fs Tx)]I where X ^(Ps exponential function multiplied by impulse response data is an operator that represents a power); and Fs; Sampling frequency.

The resultant exponential function is multiplied by representative impulse response data.

Thus, the reverberation time can be adjusted to Tx- Next, with reference to Figs. 29, a fourth example of the impulse response editing process will be described. In the fourth example, a plurality of different impulse responses are combined and thereby a new impulse response is generated. Fig. 29A shows an -Impul.se response of a steel-plate echo apparatus. Fig.

29D shows an impulse response of a hall. In the impulse responses of the steel-echo apparatus and the hail, with additional data, a point NI at which impulse response components corresponding to a direct sound and an Initial~ly reflected sound end is obtained. In the examples shown in Figs. 29A and 29B, the point NI of the impulse response corresponding to the direct sound Is the same as that corresponding to the directly reflected sound. Alternatively, the point Ni of the impulse response corresponding to the direct sound may be different from that corresponding to the directly reflected sound.

The impulse response components corresponding to the direct sound and the initially reflected sound 1999 9NOBJ 1AD 29 SONY A'ISUGI f~t No. 0923 P. 55/63 are removed from the Impulse response data of the steel-plate echo apparatus shown in Fig. 29A. Thus, the impulse response component corresponding to the reverberation is separated as shown in Fig. 29C. in addition, the impulse response component corresponding to the reverberation of the impulse response of the hall shown in Fig. 29B is removed. Thus, the impulse response components corresponding to the direct sound and the initially reflected sound are separated as shown in Fig. 29D. By combining these impulse response components, a combined impulse response as shown in Fig. 29E is generated. Instead of the separating *process and the combining process, two impulse response components may be combined by a cross fade method using is 1 a straight line or an exponential function.

Next, with reference to Figs. 30 and 31, a fifth example of the impulse response editing process will be described. In the fifth example, initially :5 :reflected sounds different from an original impulse response are combined. In Fig. 30, DL1, DL2......and DLn are delay devices with different delay amounts and connected in parallel. An initially reflected sound x0 with an original impulse response shown in Fig. 31A Is supplied to the delay devices DLU to DLn.

Output signals of the delay devices DL1 to DLn are supplied to multiplying devices MP1 to MPn, respectively. The multiplying devices MP1 to MPn 1999 9K06 IAe29 SONY ATSUGI iWt 4 NY No. 0923 P. 56/63 10 rr r r r r r multiply the output signals of the delay devices DS1 to DLn by coefficients kl to kn, respectively. The multiplying devices MP1 to MPn generate impulse response components xl to xn corresponding to the initially reflected sound whose level has been adjusted at the positions corresponding to the delay amounts of the delay devices DL1 to DLn, respectively. The impulse response xO of the original initially reflected sound and the generated impulse response components xl to xn corresponding to the initially reflected sound are added by an adding device AD. Thus, as shown in Fig. 31B. the adding device AD outputs an impulse response component corresponding to an initially reflected sound that is different from the original impulse response. The process of the block diagram shown in Fig. 30 is executed by software of the controller As a sixth example of the impulse response editing process, the value of a pre-delay that is a time period after an original audio signal is input until a reverberation is generated. In other words, the value of a pre-delay may be set to 0 or a negative value.

The above-described impulse response editing/combining process can be performed by a switch operation of the reverberation adding apparatus or a rotating operation of a rotary encoder. In this case, .19994 920B 138A29 SONY ATSUGI ~fSt4tY No. 0923 P. 57/63 10 *eeo

S..

9 additional data may be displayed on an LCD display portion. Parameters of reverberation time of an impulse response that has been edited may be displayed.

In addition, an impulse response may be displayed.

In the embodiment, the impulse response collecting apparatus 97 is independent from the reverberator 1. However, it should be noted that the present invention is not limited to such a structure.

In other words, the reverberator 1 may have a measurement signal generating portion 90, a synchronously adding portion 94, and an impulse response converting portion 95. The measurement signal generating portion 90 generates a TSP signal. These portions can be composed of a CPU and several peripheral parts. Alternatively, the DSP 30 and DSP 34 of the reverberator 1 may be used. When the reverberator 1 has a function for collecting impulse response data, the user can obtain an original sound effect.

In the embodiment, the convolution calculation process for impulse response data is performed by hardware such as DSPs 32A to 32K.

However, it should be noted that the convolution calculation process may be performed by software.

Likewise, the processes of the DSPs 30 and 34 may be performed by software.

As described above, according to the present 19994 9 200 138WO5 SONY ATSUGI t f No. 0923 P. 58/63

S

invention, since a sound effect is obtained by performing a convolution calculation process for a collected impulse response and audio data, d natural, high quality result is obtained. In addition, since an original impulse response is edited or combined with another one, various sound effects can be obtained as the user desires. in other words, according to the present invention, by editing an impulse response or combining impulse response components, the following effects can be obtainled.

Firstly, a desired reverberation time can be obtained by an adjustment. In this case, with an impulse response corresponding to a representative reverberation time, a desired reverberation time van be obtained by a calculation. Thus, the data mount of an impulse response can be decreased. Secondly, impulse response components corresponding to a direct sound and initially reflected sounds can be removed. Their levels can be adjusted; A desired initially reflected sound can be composed. Thirdly, by combining different impulse responses, a new impulse response can be generated. Fourthly, a pre-delay can be adjusted to a desired value. Since an impulse response can be edited and impulse responses can be combined, the operational characteristics of a conventional digital reverberation adding apparatus can be accomplished. In addition, a natural, high quality reverberation can be added to an 1999 9 208 13BW30N SONY ATSUGI ~i "No. 0923 P. 59/63 original audio signal.

Having described a specific preferred embodiment of the present Invention with reference to the accompanying drawings, it Is to be understood that the invention is not limited to that precise embodiment, and that various changes and modifications may be affected therein by one skilled In the art without departing from the scope or the spirit of the invention as defined In the appended claims-

Claims (11)

1. A sound effect adding apparatus for performing a convolution calculation process for an input digital audio signal and impulse response data and thereby adding a sound effect to the input digital signal, comprising: impulse response editing means for editing an original impulse response; and convoluting means for performing a convolution calculation process for the edited impulse response and a digital audio signal.

2. The sound effect adding apparatus as set forth in claim 1, wherein a plurality of different impulse 15 responses are reproduced from a record medium.

3. The sound effect adding apparatus as set forth in claim 1, wherein said impulse response editing means edit the original impulse response by using a 20 incidental information that represents characteristic of the original impulse response.

4. The sound effect adding apparatus as set forth in claim 1, wherein the incidental information includes for example data that represents a reverberation hours of the original impluse response, point values of start/end of a direct sound and an initial reflected 1999 9K2OB 1331 SONY ATSUGI f-ft" No. 0923 P. 61/63 10 15 sound and a start point value of a reverberation.

The sound effect adding apparatus as set forth in claim 1, wherein the original impulse response is composed of a first impulse response and a second impulse response, the first impulse response corresponding to a direct sound and an initial reflected sound just after the digital audio signal is inputted the second impulse response corresponding to a reverberation after the first impulse response.

6. The sound effect adding apparatus as set forth in claim wherein said impulse response editing means removes at least an impulse response component corresponding to the direct sound from the first impulse response.

7. The sound effect adding apparatus as set forth in claim wherein said impulse response editing means adjusts the value of an impluse response component corresponding to at least the direct sound of the first impulse response and compositing the said second impluse response for an ajustment.

8. The sound effect adding apparatus as set forth in claim wherein said impulse response editing means generates an impulse response corresponding to a desired 19994 920B 1331 SONY ATSUGI ft4"t No. 0923 P. 62/63 1 a. a 1 4O *4s* a reverberation hours by using a plurality of representative impulse response components corresponding to different reverberation hours of the second impulse response by a calculation.

9. The sound effect adding apparatus as set forth in claim wherein said impulse response editing means combines a plurality of original impulse responses so as to generate a new impulse response.

10. The sound effect adding apparatus as set forth in claim wherein said impulse response editing means adjusts time called pre-delay after the said impulse response is input until a reverberation is generated. a.* a Irg r

11. A sound effect adding apparatus substantially as described herein with reference to any one of the embodiments, as that embodiment is described in the accompanying diagrams. DATED this Twenty-third Day of September, 1999 Sony Corporation Patent Attorneys for the Applicant SPRUSON FERGUSON *t 0 S *0 [R:\LIBK]01659.doc.jIs