JP2015079131A - Acoustic signal processing device and acoustic signal processing program - Google Patents

Abstract

PROBLEM TO BE SOLVED: To provide an acoustic signal processing device capable of reducing an arithmetic processing quantity, and an acoustic signal processing program.SOLUTION: A delay amount adjustment part 7 adjusts a delay time difference of a plurality of reflection voices V1, V2, ..., Vk to an integer multiple of a time corresponding to an FFT shift size. An acoustic block selection part 9 selects an acoustic block corresponding to a direct voice V0 and selects acoustic blocks corresponding to the plurality of reflection voices V1, V2, ..., Vk on the basis of the adjusted delay time difference. A convolution operation part 10 uses the selected acoustic blocks and a selected divided HRTF block to execute a convolution operation, in frequency domains, on the direct voice V0 and the plurality of reflection voices V1, V2, ..., Vk and performs complex vector addition on a result of the convolution operation. A time domain conversion part 11 successively converts the arithmetic result of the convolution operation part 10 into acoustic signals in time domains through IFFT.

Description

  The present invention relates to an acoustic signal processing device and an acoustic signal processing program for outputting an acoustic signal for reproducing sound in an acoustic space.

  Various techniques for reproducing the acoustic effect in an acoustic space such as a concert hall or a theater in a listening room have been developed (see Patent Documents 1 to 3).

  The sound radiated from the sound source in the acoustic space reaches the listener directly and reaches after being reflected once or a plurality of times by the wall or ceiling of the acoustic space. The plurality of reflected sounds arrive at the listener with a delay time corresponding to the length of each sound ray path with respect to the direct sound. In order to reproduce the acoustic effect of the acoustic space in the listening room, a plurality of reflected sounds having the same delay time as the plurality of reflected sounds in the acoustic space are reproduced. In the reflected sound extraction apparatus described in Patent Document 1, a sound field is reproduced by convolving a plurality of reflected sounds and music signals stored in advance. In the reverberation imparting device described in Patent Literature 2, the direction from the sounding point to the sound receiving point is specified as the direction of the sounding point, and an impulse response and a sound effect to which a sound effect is reflected reflecting the direction of the specified sounding point. A convolution operation with the signal is performed. In the reverberation imparting device described in Patent Document 3, an impulse response is specified from a sound ray synthesis vector obtained according to the directivity characteristics of the sound generation point and the sound reception point, and the impulse response is convolved with the acoustic signal. .

  In order to reproduce the reflected sound in the virtual acoustic space, it is conceivable to perform a convolution operation between the acoustic signals of a plurality of reflected sounds having different delay times and the head-related transfer function.

JP-A-5-46193 Patent No. 4062959 Japanese Patent No. 4464064

  As described above, in order to reproduce a plurality of reflected sounds in an actual sound space or a virtual sound space, a convolution operation between a plurality of sound signals having different delay times and an impulse response or a head-related transfer function is performed. There is a need to do.

  However, since there are a large number of acoustic signals corresponding to a large number of reflected sounds, the processing amount of the convolution calculation increases. In that case, it is necessary to use an arithmetic processing device capable of high-speed operation so that the arithmetic processing is not delayed with respect to the input of the real-time acoustic signal. This increases the cost and makes it difficult to reduce the size of the system. On the other hand, when a relatively inexpensive arithmetic processing device is used, the sound reproduction accuracy has to be lowered so that the arithmetic processing is not delayed by the real-time input of the acoustic signal.

  An object of the present invention is to provide an acoustic signal processing device and an acoustic signal processing program capable of reducing the amount of calculation processing.

  (1) The acoustic signal processing device according to the present invention is delayed from the first sound by the first sound radiated from the first sound source and arriving at the sound receiving point and the at least one second sound source. An acoustic signal processing apparatus that outputs an acoustic signal representing a sound obtained by mixing at least one second sound arriving at a sound receiving point, and calculating a delay time difference between the first sound and the second sound. An acoustic signal in the frequency domain is obtained by performing time-frequency conversion sequentially while shifting the original acoustic signal representing the first sound radiated from the first sound source by a certain shift amount on the time axis. A first conversion unit to be obtained, an adjustment unit that adjusts the delay time difference calculated by the calculation unit to an integral multiple of a time corresponding to a shift amount of time-frequency conversion, and a frequency domain obtained by the first conversion unit. A first signal portion corresponding to the first sound from the acoustic signal; And selecting a second signal portion corresponding to the second sound from the acoustic signal in the frequency domain obtained by the first converter based on the delay time difference adjusted by the adjustment unit; A first convolution operation between the first sound transfer function from one sound source to the sound receiving point and the first signal portion selected by the selection unit, and a second sound transfer from the second sound source to the sound receiving point. A second convolution operation between the function and the second signal portion selected by the selection unit is performed in the frequency domain, and an addition result of the first and second convolution operations is added; Is converted to a time domain acoustic signal.

  In this acoustic signal processing device, a delay time difference between a first sound corresponding to the first sound source and at least one second sound corresponding to at least one second sound source is calculated. Further, the calculated delay time difference is adjusted to an integral multiple of the time corresponding to the shift amount of the time-frequency conversion.

  The original acoustic signal representing the first sound is sequentially time-frequency converted while being shifted by a certain shift amount on the time axis, thereby obtaining an acoustic signal in the frequency domain. A first signal portion corresponding to the first sound is selected from the acoustic signal in the frequency domain, and a second signal portion corresponding to the second sound is selected based on the adjusted delay time difference. A first convolution operation between the first acoustic transfer function and the first signal portion and a second convolution operation between the second acoustic transfer function and the second signal portion are performed in the frequency domain, and the first and second The result of the convolution operation of 2 is added. The result of the addition is converted into an acoustic signal in the time domain.

  In this case, since the delay time difference between the first signal portion and the second signal portion in the acoustic signal in the frequency domain is an integral multiple of the time corresponding to the shift amount of the time-frequency conversion, the second signal portion The first signal portion already obtained by the previous time-frequency conversion can be used. Therefore, time-frequency conversion for obtaining the second signal portion is unnecessary. In addition, since the addition of the results of the first and second convolution operations is performed in the frequency domain, the sound block frequency domain is changed to the time domain per signal part (first or second signal part) of the original acoustic signal. An acoustic signal in the time domain can be obtained by a single conversion. Thereby, the number of calculations can be reduced. As a result, it is possible to reduce the amount of processing in the arithmetic processing for outputting the acoustic signal representing the sound arriving at the sound receiving point.

  (2) The first conversion unit sequentially obtains a unit block of the first number of samples from the original sound signal, and high-speeds an acoustic signal of the second number of samples including the unit block and larger than the first number of samples. Fourier transform, and the first transform unit, the computation unit, and the second transform unit perform fast Fourier transform, first and second convolution operations, and time domain acoustic signals by the overlap save method or the overlap add method. Conversion is performed, and the shift amount of the fast Fourier transform may be equal to the number of samples of the unit block.

  In this case, by reducing the size of the unit block, it is possible to reduce the error due to the adjustment of the delay time difference and the delay time in the convolution calculation. As a result, the sound arriving at the sound receiving point can be reproduced with high accuracy.

  (3) The first acoustic transfer function includes a plurality of first divided transfer functions, and the plurality of first divided transfer functions are the first acoustic response characteristics in the time domain from the first sound source to the sound receiving point. The plurality of first division response characteristics obtained by the division are obtained by fast Fourier transform, and the second acoustic transfer function includes a plurality of second division transfer functions, and a plurality of second division transfer functions The function is obtained by fast Fourier transforming a plurality of second divided response characteristics obtained by dividing the second acoustic response characteristic in the time domain from the second sound source to the sound receiving point. , Selecting a number of first signal portions according to the number of divisions of the plurality of first division transfer functions, and selecting a plurality of second signal portions according to the number of divisions of the plurality of second division transfer functions. The calculation unit selects a plurality of first divided transfer functions and a plurality of first divisions selected by the selection unit. The second convolution of the first convolution and a plurality of second divided transfer function and a plurality of second signal portion selected by the selection unit of the signal portion may be performed in the frequency domain.

  In this case, the size of each first signal portion is reduced, and the size of each second signal portion is reduced. This reduces the number of computations when fast Fourier transforming the time domain acoustic signal. Therefore, it is possible to further reduce the processing amount in the arithmetic processing for outputting the acoustic signal representing the sound arriving at the sound receiving point.

  In addition, since the size of the unit block can be reduced, it is possible to reduce the error due to the adjustment of the delay time difference and the delay time in the convolution calculation. Thereby, the sound arriving at the sound receiving point can be reproduced with higher accuracy.

  (4) The first sound is a direct sound that arrives at the receiving point without being reflected from the first sound source, and the second sound is a reflected sound that is reflected while being reflected from the first sound source, The second sound source may be a virtual sound source that virtually radiates reflected sound.

  In this case, it is possible to reproduce the sound that arrives at the sound receiving point in an actual acoustic space or a virtual acoustic space.

  (5) The acoustic signal processing program according to the present invention is delayed from the first sound by the first sound radiated from the first sound source and arriving at the sound receiving point and the at least one second sound source. An acoustic signal processing program executable by a computer to output an acoustic signal representing a sound obtained by mixing at least one second sound arriving at a sound receiving point, the first sound and the second sound Frequency by performing a time-frequency conversion while shifting the original sound signal representing the first sound radiated by the first sound source by a certain shift amount on the time axis. A process for obtaining an acoustic signal in the region, a process for adjusting the calculated delay time difference to an integral multiple of a time corresponding to the shift amount of the time-frequency conversion, and a first corresponding to the first sound from the acoustic signal in the frequency domain Select the signal part of A process of selecting a second signal portion corresponding to the second sound from the frequency domain acoustic signal based on the adjusted delay time difference, a first acoustic transfer function from the first sound source to the sound receiving point, and The first convolution operation with the selected first signal portion and the second convolution operation between the second sound transfer function from the second sound source to the sound receiving point and the selected second signal portion with the frequency are performed. The computer executes a process of performing addition of the results of the first and second convolution operations in the area and a process of converting the result of the addition into an acoustic signal in the time domain.

  According to this acoustic signal processing program, the delay time difference between the first signal portion and the second signal portion in the frequency domain acoustic signal is an integral multiple of the time corresponding to the shift amount of the time-frequency conversion. As the second signal part, the first signal part already obtained by the previous time-frequency conversion can be used. Therefore, time-frequency conversion for obtaining the second signal portion is unnecessary. In addition, since the addition of the results of the first and second convolution operations is performed in the frequency domain, one signal portion (first or second signal portion) of the original acoustic signal is 1 from the frequency domain to the time domain. An acoustic signal in the time domain can be output by the conversion of times. Thereby, the number of calculations can be reduced. As a result, it is possible to reduce the amount of processing in the arithmetic processing for outputting the acoustic signal representing the sound arriving at the sound receiving point.

  ADVANTAGE OF THE INVENTION According to this invention, it becomes possible to reduce the processing amount in the arithmetic processing for outputting the acoustic signal showing the sound which arrives at a sound receiving point.

It is a functional block diagram which shows the structure of the acoustic signal processing apparatus which concerns on one embodiment of this invention. It is a schematic diagram which shows virtual acoustic space. It is a block diagram which shows an example of the hardware constitutions of the acoustic signal processing apparatus of FIG. It is explanatory drawing of the head impulse response of a time domain, and the head-related transfer function of a frequency domain. It is a schematic diagram showing a plurality of sets of divided HRTF blocks stored in the HRTF database. It is explanatory drawing of the original acoustic signal of a time domain, and the acoustic block of a frequency domain. It is a figure which shows the head impulse response corresponding to a direct sound and a reflected sound, a head-related transfer function, the delay amount before adjustment, the delay amount after adjustment, and the number of delay blocks. It is a figure which shows the convolution calculation of the division | segmentation HRTF block and acoustic block in a frequency domain. It is a figure which shows the joining of the acoustic signal in a time domain. It is a flowchart which shows the acoustic signal process performed by the acoustic signal processing apparatus of FIG. It is a flowchart which shows the detail of a convolution calculation process. It is a figure which shows the convolution calculation of the division | segmentation HRTF block and acoustic block in the frequency domain in the convolution calculation process which concerns on a reference form. It is a flowchart which shows the detail of the convolution calculation process which concerns on a reference form. It is explanatory drawing of the original sound signal of a time domain at the time of using a division | segmentation overlap add method, and the sound block of a frequency domain. It is a figure which shows the joining of the acoustic signal in the time domain at the time of using a division | segmentation overlap add method.

  Hereinafter, an acoustic signal processing device and an acoustic signal program according to embodiments of the present invention will be described in detail with reference to the drawings.

(1) Functional Configuration of Acoustic Signal Processing Device FIG. 1 is a functional block diagram showing a configuration of an acoustic signal processing device according to an embodiment of the present invention. FIG. 2 is a schematic diagram showing a virtual acoustic space. FIG. 3 is a block diagram showing an example of a hardware configuration of the acoustic signal processing apparatus of FIG.

  The acoustic signal processing device 100 in FIG. 1 outputs an acoustic signal representing a sound arriving at a sound receiving point in a virtual acoustic space (hereinafter referred to as a virtual space). Here, an example of the virtual space will be described with reference to FIG.

  In FIG. 2, the main sound source S0 and the sound receiving point R are arranged in the virtual space 300. The virtual space 300, the main sound source S0, and the sound receiving point R are virtually created on a computer program. Sound is emitted from the main sound source S0 in the three-dimensional directions of front and rear, left and right, and upper and lower. The sound radiated from the main sound source S0 reaches the sound receiving point R as a direct sound V0 and is reflected once or a plurality of times by the wall or ceiling of the virtual space 300, and a plurality of reflected sounds V1 are received at the sound receiving point R. , V2, V3, V4, ..., Vk. Here, k is a natural number and represents the number of reflected sounds. In FIG. 2, the directions of the plurality of reflected sounds V1, V2, V3, V4,..., Vk are represented in a two-dimensional direction, but the directions of the plurality of reflected sounds V1, V2, V3, V4,. It may be expressed in a three-dimensional direction.

  The plurality of reflected sounds V1, V2, V3, V4,..., Vk can be regarded as equivalently emitted from the virtual sound sources S1, S2, S3, S4,. The virtual sound sources S1, S2, S3, S4,..., Sk are located on a straight line from the sound receiving point R in the direction opposite to the incident direction of the reflected sounds V1, V2, V3, V4,. The distance between the sound receiving point R and the virtual sound sources S1, S2, S3, S4, ..., Sk is such that the reflected sounds V1, V2, V3, V4, ..., Vk reach the sound receiving point R from the main sound source S0. Equal to the length of the path to

  Hereinafter, the time until the sound radiated from the main sound source S0 reaches the sound receiving point R as the direct sound V0 is referred to as a delay amount. Similarly, the time until the sound radiated from the main sound source S0 reaches the sound receiving point R as the reflected sounds V1, V2,. The delay amount of the reflected sounds V1, V2,..., Vk is larger than the delay amount of the direct sound V0. The difference between the delay amount of the reflected sounds V1, V2,..., Vk and the delay amount of the direct sound V0 is called a delay time difference.

  A head-related transfer function (HRTF) in the frequency domain is obtained in advance for each direction of sound arriving at the receiving point R. That is, a plurality of head related transfer functions corresponding to a plurality of directions are obtained in advance. Here, when the direction of the sound arriving at the sound receiving point R is represented in a three-dimensional direction, head related transfer functions respectively corresponding to a plurality of three-dimensional directions are obtained in advance. The head-related transfer function corresponding to the direction of arrival of the direct sound V0 at the sound receiving point R indicates a sound transfer characteristic from the main sound source S0 to the sound receiving point R. The head-related transfer functions corresponding to the arrival directions of the reflected sounds V1, V2,..., Vk at the sound receiving point R indicate the sound transfer characteristics from the virtual sound sources S1, S2,. . These head-related transfer functions are used to calculate an acoustic signal representing a sound arriving at the sound receiving point R as will be described later.

  In FIG. 1, an acoustic signal processing apparatus 100 includes a room shape instruction unit 1, a main sound source position instruction unit 2, a head related transfer function database (hereinafter referred to as HRTF database) 3, and a head related transfer function block selection unit (hereinafter referred to as “head transfer function block selection unit”). 4) (referred to as an HRTF block selector). The acoustic signal processing device 100 includes a virtual sound source position calculation unit 5, a delay amount calculation unit 6, a delay amount adjustment unit 7, a delay block number calculation unit 8, and an acoustic block selection unit 9. Furthermore, the acoustic signal processing device 100 includes a convolution operation unit 10, a time domain conversion unit 11, an acoustic signal output unit 12, an acoustic signal input unit 13, a frequency domain conversion unit 14, and a frequency domain acoustic buffer 15. The entire acoustic signal processing apparatus 100 operates at the same sampling frequency. The overall sampling frequency of the acoustic signal processing apparatus 100 is denoted as sampling frequency fs. The sampling frequency fs is 48 kHz, for example.

  The room shape instructing unit 1 outputs room data instructing the shape of the virtual space (hereinafter referred to as a room shape). For example, the room shape instructing unit 1 outputs room data indicating the room shape drawn by the user using an input device such as a mouse on the screen, or is selected by the user from a plurality of room shapes prepared in advance. Outputs room data indicating the room shape. Or the room shape instruction | indication part 1 may output room data dynamically on a program. For example, in a video game, the program may select appropriate room data depending on the position of the character. In this case, a part of the video game program corresponds to the room shape instruction unit 1.

  The main sound source position instruction unit 2 outputs position data indicating the position of the main sound source S0 in the virtual space. For example, the main sound source position instruction unit 2 outputs position data indicating the position of the main sound source S0 in a virtual space having a room shape drawn on the screen by the user. Alternatively, the main sound source position instruction unit 2 may dynamically output position data on the program. For example, the program may output position data indicating the position of the character in the video game. In this case, a part of the video game program corresponds to the main sound source position instruction unit 2. The position data of the main sound source S0 includes, for example, vector data representing the direction from the sound receiving point R to the main sound source S0 and the distance from the sound receiving point R to the main sound source S0.

  Based on the room data output from the room shape instruction unit 1 and the position data output from the main sound source position instruction unit 2, the virtual sound source position calculation unit 5 has a plurality of reflected sounds V1 and V2 that arrive at the sound receiving point R. ,..., Vk, and the positions of a plurality of virtual sound sources S1, S2,. The virtual sound source position calculation unit 5 outputs position data indicating the positions of the plurality of virtual sound sources S1, S2,. The position data of the virtual sound sources S1, S2,..., Sk is, for example, the direction from the sound receiving point R to the virtual sound sources S1, S2,. It consists of vector data representing the distance. In addition, the virtual sound source position calculation unit 5 calculates the amplitude attenuation amount of the plurality of reflected sounds V1, V2,..., Vk with respect to the direct sound V0. The amplitude attenuation amount is calculated for each reflected sound V1, V2,..., Vk based on the length (distance) of the sound path, the number of reflections, the sound absorption coefficient of each reflecting surface, and the like. The calculation process of the amplitude attenuation amount may be performed so as to vary depending on the frequency band of the sound.

  The delay amount calculation unit 6 calculates the delay amount of the direct sound V0 based on the position data output from the main sound source position instruction unit 2, and a plurality of delays based on the position data output from the virtual sound source position calculation unit 5. The delay amounts of the reflected sounds V1, V2,. Here, the difference between the delay amount of the plurality of reflected sounds V1, V2,..., Vk and the delay amount of the direct sound V0 is referred to as a delay time difference.

  The delay amount adjusting unit 7 includes a plurality of reflected sounds V1 such that a delay time difference between the plurality of reflected sounds V1, V2,..., Vk is an integral multiple of a time determined by the sampling frequency fs and the FFT (Fast Fourier Transform) shift size. , V2,..., Vk are adjusted. The FFT shift size will be described later. Specifically, the delay amount of each reflected sound is adjusted so that the delay time difference between the reflected sounds is an integral multiple of the time obtained by dividing the FFT shift size by the sampling frequency fs. In this case, an integer is selected so that an error between the delay amount of the reflected sounds V1, V2,..., Vk after adjustment and the delay amount of the reflected sounds V1, V2,.

  The delay block number calculation unit 8 calculates the number of delay blocks for a plurality of reflected sounds V1, V2,. Here, the number of delay blocks is the number of unit blocks (frames) corresponding to the adjusted delay time difference. A unit block is a sample of an acoustic signal that is processed at one time (that is, an acoustic signal processing unit). In the present embodiment, the unit block consists of N samples. N is a natural number.

  The acoustic signal input unit 13 inputs a time domain acoustic signal. For example, the acoustic signal input unit 13 converts an analog acoustic signal given to an acoustic input terminal from an external device or a microphone into a digital acoustic signal at a sampling frequency fs. Alternatively, the acoustic signal input unit 13 inputs a digital acoustic signal stored in a storage medium such as an optical disk, a magnetic disk, or a memory card. Hereinafter, the time-domain acoustic signal input by the acoustic signal input unit 13 is referred to as an original acoustic signal. The sampling frequency of the original sound signal is fs.

  The frequency domain conversion unit 14 sequentially converts the original acoustic signal input from the acoustic signal input unit 13 into a signal portion of the frequency domain acoustic signal by FFT (Fast Fourier Transform). Hereinafter, the signal portion of the frequency domain acoustic signal is referred to as an acoustic block. The acoustic blocks converted by the frequency domain converting unit 14 are sequentially stored in the frequency domain acoustic buffer 15.

  Based on the number of delay blocks calculated by the delay block number calculation unit 8, the acoustic block selection unit 9 generates a direct sound V 0 and a plurality of reflected sounds V 1, V 2,... From the acoustic block stored in the frequency domain acoustic buffer 15. , Vk is selected.

  On the other hand, in the HRTF database 3, a plurality of sets of divided head related transfer functions (hereinafter referred to as divided HRFT blocks) in the frequency domain are stored in advance. Details of the divided HRTF block will be described later. A plurality of sets of divided HRTF blocks are prepared in advance corresponding to a plurality of directions of sound arriving at the sound receiving point R. When the direction of the sound arriving at the sound receiving point R is represented by a three-dimensional direction, the plurality of sets of divided HRTF blocks respectively correspond to the three-dimensional direction.

  Based on the position data output from the main sound source position instructing unit 2 and the virtual sound source position calculating unit 5, the HRTF block selecting unit 4 receives the direct sound V0 and the plurality of direct sounds V0 from a plurality of sets of divided HRTF blocks stored in the HRTF database 3. The divided HRTF blocks corresponding to the reflected sounds V1, V2, V3,.

  The convolution operation unit 10 uses the acoustic block selected by the acoustic block selection unit 9 and the divided HRTF block selected by the HRTF block selection unit 4 to perform direct sound V0 and a plurality of reflected sounds V1, V2,. The convolution operation in the frequency domain is performed, and the result of the convolution operation is added as a complex vector. In this case, the convolution operation unit 10 adjusts the amplitude of each frequency component in the acoustic block based on the amplitude attenuation amount calculated by the virtual sound source position calculation unit 5.

  The time domain conversion unit 11 sequentially converts the calculation results of the convolution calculation unit 10 into time domain acoustic signals by IFFT (Inverse Fast Fourier Transform).

  The acoustic signal output unit 12 outputs an acoustic signal having the sampling frequency fs converted by the time domain conversion unit 11. For example, the acoustic signal output unit 12 converts a digital acoustic signal having a sampling frequency fs into an analog acoustic signal, and outputs the analog acoustic signal to a headphone or a speaker through an acoustic output terminal. Thereby, sound is generated from the headphone or the speaker.

  In the present embodiment, a convolution operation in the frequency domain is performed by an overlap-save method using divided HRTF blocks. Hereinafter, the overlap save method using the divided HRTF block is referred to as a divided overlap save method.

(2) Hardware Configuration of Acoustic Signal Processing Device FIG. 3 is a block diagram showing an example of the hardware configuration of the acoustic signal processing device 100.

  3 includes a CPU (Central Processing Unit) 110, a ROM (Read Only Memory) 120, a RAM (Random Access Memory) 130, a storage device 140, a display device 150, an input device 160, and an output device 170. including.

  The ROM 120 is composed of, for example, a non-volatile memory, and stores computer programs such as a system program and an acoustic signal processing program. The RAM 130 is composed of, for example, a volatile memory, is used as a work area for the CPU 110, and temporarily stores various data. The CPU 110 performs acoustic signal processing described later by executing an acoustic signal processing program stored in the ROM 120 on the RAM 130. In this case, the function of each component in FIG. 1 is realized.

  The storage device 140 includes a storage medium such as a hard disk, an optical disk, a magnetic disk, or a memory card. The storage device 140 includes the HRTF database 3 and the frequency domain acoustic buffer 15 shown in FIG. The acoustic signal processing program may be stored in the storage device 140. Further, for example, when the audio signal processing device 100 of FIG. 1 is configured as a part of a video game program, the video game program may be stored in the storage device 140.

  The acoustic signal processing program in the present embodiment may be provided in a form stored in a computer-readable recording medium and installed in the ROM 120 or the storage device 140, or in a form distributed via a communication network. It may be provided and installed in the ROM 120 or the storage device 140.

  The display device 150 includes a liquid crystal display device, an organic EL (electroluminescence) display device, a plasma display device, or the like. The input device 160 includes a mouse, a keyboard, and an acoustic input terminal. The input device 160 may be a video game controller.

  The display device 150 and the input device 160 are used, for example, for the user to indicate the room shape and the position of the main sound source on the screen. The display device 150 and the input device 160 may be integrated as a touch panel.

  The output device 170 includes a sound output terminal and headphones. The output device 170 may include a speaker. An acoustic signal obtained by acoustic signal processing is output from the acoustic output terminal of the output device 170.

  The acoustic signal processing apparatus 100 may include a DSP (Digital Signal Processor) instead of the CPU 110, or may include a DSP in addition to the CPU 110. Also, some or all of the components in FIG. 1 may be configured by hardware such as an electronic circuit.

(3) Head-related transfer function FIG. 4 is an explanatory diagram of a time-domain head impulse response and a frequency-domain head-related transfer function.

  A time-domain head-head impulse response (HRIR;) corresponding to the direct sound V0 is divided into M parts (hereinafter referred to as divided HRIR blocks). M is a natural number. In the example of FIG. 4, the head impulse response h0 is divided into four divided HRIR blocks h0, 0, h0, 1, h0, 2, h0, 3 on the time axis. Each divided HRIR block h0,0, h0,1, h0,2, h0,3 consists of N samples. The sampling frequency of the head impulse response is equal to the sampling frequency fs of the original sound signal.

  0 of N samples are added after the divided HRIR block h0,0, and 2N samples including 0 are converted into divided HRTF blocks H0,0 in the frequency domain by FFT. Similarly, divided HRTF blocks H0, 1, H0, 2, H0, 3 in the frequency domain are obtained using the divided HRIR blocks h0, 1, h0, 2, h0, 3, respectively.

  Note that N samples of 0 may be added before the divided HRIR blocks h0,0, h0,1, h0,2, h0,3, respectively.

  Similarly, the head impulse response in the time domain corresponding to each direction of the reflected sounds V1, V2,..., Vk is divided into M divided HRIR blocks, and M divided HRIR blocks are divided into M pieces by FFT. Converted to frequency domain split HRTF blocks.

  FIG. 5 is a schematic diagram showing a plurality of sets of divided HRTF blocks stored in the HRTF database 3.

  As shown in FIG. 5, the HRTF database 3 stores in advance k sets of divided HRTF blocks corresponding to k directions. The divided HRTF blocks H0, 0, H0, 1, H0, 2, H0, 3 correspond to the direction of the direct sound V0. The divided HRTF blocks H1, 0, H1, 1, H1, 2, H1, 3 correspond to the direction of the reflected sound V1. The divided HRTF blocks H2,0, H2,1, H2,2, H2,3 correspond to the direction of the reflected sound V2. The divided HRTF blocks Hk, 0, Hk, 1, Hk, 2, Hk, 3 correspond to the direction of the reflected sound Vk.

(4) Acoustic Block FIG. 6 is an explanatory diagram of the original acoustic signal in the time domain and the acoustic block in the frequency domain. In FIG. 6, time elapses from right to left.

  In the original sound signal VIN, the unit block vn is currently input. The unit blocks vn-1, vn-2, vn-3, and vn-4 are unit blocks that are input one time before, two times before, three times before, and four times before, respectively. The size of each unit block vn, vn-1, vn-2, vn-3, vn-4 is N samples.

  A signal portion xn composed of unit blocks vn and vn−1 is converted into an acoustic block Xn in the frequency domain by FFT. Similarly, the signal part xn-1 composed of the unit blocks vn-1 and vn-2 is converted into an acoustic block Xn-1 in the frequency domain by FFT, and the signal part xn-2 composed of the unit blocks vn-2 and vn-3. Is converted to a frequency domain acoustic block Xn-2 by FFT, and a signal portion xn-3 composed of unit blocks vn-3 and vn-4 is converted to a frequency domain acoustic block Xn-3 by FFT. The acoustic blocks Xn, Xn-1, Xn-2, and Xn-3 are sequentially stored in the frequency domain acoustic buffer 15 of FIG.

  Here, the size of the signal portion processed by one FFT is called the FFT size. In the example of FIG. 6, the FFT size is 2N samples. Also, the shift amount between the unit block that is the object of each FFT on the time axis and the unit block that is the object of the previous FFT is referred to as the FFT shift size. In the example of FIG. 6, the FFT shift size SS is N samples and is equal to the size of the unit block. In this case, the FFT size is twice the FFT shift size SS. The relationship between the FFT size and the FFT shift size SS is not limited to this example, and the FFT size may be a size other than twice the FFT shift size SS (for example, four times).

(5) Adjustment of delay amount FIG. 7 shows head impulse response, divided HRTF block, delay amount before adjustment, delay amount after adjustment, and number of delay blocks corresponding to direct sound V0 and reflected sounds V1, V2,. FIG. In FIG. 7, M1, M2,... Mk are integers.

  The head sound response h0 in the time domain and a set of divided HRTF blocks H0, 0, H0, 1, H0, 2, H0, 3 in the frequency domain correspond to the direct sound V0. The delay amount before adjustment of the direct sound V0 is d0, the delay amount after adjustment is also d0, and the number of delay blocks is zero.

  The reflected sound V1 corresponds to a head impulse response h1 in the time domain and a set of divided HRTF blocks H1, 0, H1, 1, H1, 2, H1, 3 in the frequency domain. The delay amount before adjustment of the reflected sound V1 is d1. The delay amount adjustment unit 7 in FIG. 1 adjusts the delay amount of the reflected sound V1 to d0 + M1 × SS / fs. In this case, the number of delay blocks is M1.

  Similarly, the head sound response h2 in the time domain and a set of divided HRTF blocks H2,0, H2,1, H2,2, H2,3 in the frequency domain correspond to the reflected sound V2. The delay amount before adjustment of the reflected sound V2 is d2, the delay amount after adjustment is d0 + M2 × SS / fs, and the number of delay blocks is M2. Further, the reflected sound Vk corresponds to a head-time impulse response hk in the time domain and a set of divided HRTF blocks Hk, 0, Hk, 1, Hk, 2, Hk, 3 in the frequency domain. The delay amount before adjustment of the reflected sound Vk is dk, the delay amount after adjustment is d0 + Mk × SS / fs, and the number of delay blocks is Mk.

  In this example, the delay time differences between the reflected sounds V1, V2,..., Vk are adjusted to M1 × SS / fs, M2 × SS / fs, and Mk × SS / fs, respectively. That is, the delay time difference between the reflected sounds V1, V2,..., Vk is adjusted to an integral multiple of the time corresponding to the FFT shift size SS.

(6) Convolution calculation in frequency domain FIG. 8 is a diagram showing a convolution calculation of a divided HRTF block and an acoustic block in the frequency domain. In FIG. 8, time elapses from right to left.

  The left end of the time axis is the portion of the original sound signal VIN that is currently input. At present, as shown in FIG. 6, the 2N sample portion of the original sound signal VIN is converted into the sound block Xn by FFT. The acoustic blocks Xn-1, Xn-2,..., Xn-12 are already stored in the frequency domain acoustic buffer 15 of FIG.

  In the example of FIG. 8, the delay time difference DL1 of the reflected sound V1 is three times the time corresponding to the FFT shift size SS, and the number of delay blocks M1 is 3. The delay time difference DLk of the reflected sound Vk is nine times the time corresponding to the FFT shift size SS, and the number of delay blocks Mk is nine.

  The divided HRTF blocks H0,0, H0,1, H0,2, H0,3 corresponding to the direct sound V0 are selected from a plurality of sets of divided HRTF blocks stored in the HRTF database 3 of FIG. Also, the divided HRTF blocks H1, 0, H1, 1, H1, 2, H1, 3 corresponding to the reflected sound V1 are selected, and the divided HRTF blocks Hk, 0, Hk, 1, Hk, 2 corresponding to the reflected sound Vk are selected. , Hk, 3 are selected.

  For the direct sound V0, a convolution operation of the divided HRTF blocks H0, 0, H0, 1, H0, 2, H0, 3 and the acoustic blocks Xn, Xn-1, Xn-2, Xn-3 is performed in the frequency domain. A convolution operation result Y0 is obtained. For the reflected sound V1, the divided HRTF blocks H1, 0, H1, 1, H1, 2, H1, 3 and the acoustic blocks Xn-3, Xn-4, Xn (because the delay block number M1 is 3) in the frequency domain. A convolution operation with -5 and Xn-6 is performed, and a convolution operation result Y1 is obtained. For the reflected sound Vk, the divided HRTF blocks Hk, 0, Hk, 1, Hk, 2, Hk, 3 and the acoustic blocks Xn-9, Xn-10, Xn in the frequency domain (since the delay block number Mk is 9). A convolution operation with -11 and Xn-12 is performed, and a convolution operation result Yk is obtained. Details of the convolution operation will be described later.

  The convolution calculation result Y1 is multiplied by a gain corresponding to the amplitude attenuation amount calculated for the reflected sound V1 by the virtual sound source position calculation unit 5 of FIG. Similarly, the convolution calculation result Yk is multiplied by a gain corresponding to the amplitude attenuation calculated for the reflected sound Vk. Thereby, the amplitudes of the convolution calculation results Y1,..., Yk are adjusted. When the amplitude attenuation amount is 0, the gain is 1. The convolution calculation result Y0 and the amplitude adjustment convolution calculation results Y1,..., Yk are added as complex vectors, and the addition result is converted into an acoustic signal yn in the time domain by IFFT.

  FIG. 9 is a diagram showing stitching of acoustic signals in the time domain. As shown in FIG. 9, the N samples in the first half of the acoustic signal yn obtained by the current process are discarded. The N samples in the latter half of the acoustic signal yn are joined to the N samples in the latter half of the acoustic signal yn-1 obtained in the previous process. By sequentially performing these operations, the acoustic signal VOUT is sequentially output.

  In addition, when N samples of 0 are added before each divided HRIR block at the time of calculation of each divided HRTF block in the frequency domain shown in FIG. 4, the acoustic signal yn obtained in this processing is The N samples in the latter half are discarded, and the N samples in the first half of the acoustic signal yn are joined with the N samples in the first half of the acoustic signal yn-1 obtained in the previous processing.

(7) Overall Operation of Acoustic Signal Processing Device FIG. 10 is a flowchart showing acoustic signal processing performed by the acoustic signal processing device 100 of FIG. The acoustic signal processing in FIG. 10 is performed by the CPU 110 in FIG. 3 executing the acoustic signal processing program stored in the ROM 120 or the storage device 140.

  The room shape instructing unit 1 in FIG. 1 outputs room data for instructing the room shape (step S1). The main sound source position instruction unit 2 outputs position data indicating the position of the main sound source S0 in the virtual space having the instructed room shape (step S2).

  Next, the virtual sound source position calculation unit 5 calculates the positions of the plurality of virtual sound sources S1, S2,..., Sk based on the room data and the position data of the main sound source S0 (step S3). Thereby, position data indicating the positions of the virtual sound sources S1, S2,.

  The delay amount calculation unit 6 calculates the delay amounts of the direct sound V0 and the reflected sounds V1, V2,..., Vk based on the position data of the main sound source S0 and the virtual sound sources S1, S2,. ). The delay amount adjusting unit 7 adjusts the delay time difference between the plurality of reflected sounds V1, V2,..., Vk to an integral multiple of the time corresponding to the FFT shift size (= SS / fs) (step S5). The delay block number calculation unit 8 calculates the number of delay blocks for a plurality of reflected sounds V1, V2,..., Vk based on the adjusted delay time difference.

  Based on the position data output from the main sound source position instructing unit 2 and the virtual sound source position calculating unit 5, the HRTF block selecting unit 4 receives the direct sound V0 and the plurality of direct sounds V0 from a plurality of sets of divided HRTF blocks stored in the HRTF database 3. The divided HRTF blocks corresponding to the reflected sounds V1, V2,..., Vk are selected (step S6).

  The convolution operation unit 10, the time domain conversion unit 11, the acoustic signal output unit 12, and the frequency domain conversion unit 14 perform a convolution operation process (step S7).

  FIG. 11 is a flowchart showing details of the convolution operation processing. The variable n in FIG. 11 means the current process, and the value of the variable n is incremented by 1 from 0. M represents the number of divisions of the head-related transfer function (number of divided HRTF blocks), and k represents the number of reflected sounds. M1,..., Mk represent the number of delay blocks for the reflected sounds V1,.

  In the initial state, the value of the variable n is 0 (step S11). The frequency domain converter 14 in FIG. 1 converts the signal portion xn of the original acoustic signal VIN having the sampling frequency fs into the acoustic block Xn by FFT (step S12). The signal portion xn includes a unit block vn currently acquired from the original sound signal VIN and a unit block vn-1 acquired last time (see FIG. 6). Moreover, the frequency domain conversion part 14 stores the acoustic block Xn in the frequency domain acoustic buffer 15 (step S13). The acoustic block Xn is sequentially stored in the frequency domain acoustic buffer 15 as the value of the variable n increases in step S34 described later.

  In steps S14 to S19, the convolution calculation result Y0 for the direct sound V0 is calculated. In steps S20 to S25, the convolution calculation result Y1 for the reflected sound V1 is calculated, and in steps S26 to S31, the convolution calculation result Yk for the reflected sound Vk is calculated. Steps S14 to S19, steps S20 to S25, and steps S26 to S31 are executed in parallel.

  First, the convolution operation unit 10 sets the value of the variable m to the initial value 0 (steps S14, S20, S26), and sets the convolution operation results Y0, Y1,... Yk to the initial value 0 (steps S15, S21, S21). S27). Next, the convolution operation unit 10 performs complex vector multiplication of the acoustic block Xn-m and the divided HRTF blocks H0, m, and calculates Y = Xn-m * H0, m as a convolution operation result (step S16). Next, the convolution operation unit 10 adds the current convolution operation result Y to the previous convolution operation result Y0 by a complex vector (step S17). Thereafter, 1 is added to the variable m (step S18), and it is determined whether or not the variable m is larger than M−1 (step S19). Until the variable m becomes M-1, the processes of steps S16 to S19 are repeated. Thereby, Y0 = Xn * H0, 0 + Xn-1 * H0, 1 + Xn-2 * H0, 2+... + Xn-M + 1 * H0, M-1 is calculated. Here, “*” means complex vector multiplication, and “+” means complex vector addition. In the example of FIG. 8, since M = 4, Y0 = Xn * H0, 0 + Xn-1 * H0, 1 + Xn-2 * H0, 2 + Xn-3 * H0,3 are calculated.

  In the above convolution calculation, the acoustic blocks Xn−1, Xn−2,..., Xn−M + 1 have already been calculated in the previous processing and stored in the frequency domain acoustic buffer 15.

  Similarly, in steps S22 to S25, Y1 = Xn−M1 * H1, 0 + Xn−1−M1 * H1,1 + Xn−2M1 * H1,2 +... + Xn−M + 1−M1 * H1, M−1 is calculated. The Here, M1 is the number of delay blocks of the reflected sound V1. In the example of FIG. 8, since M = 4 and M1 = 3, Y1 = Xn−3 * H1, 0 + Xn−4 * H1,1 + Xn−5 * H1, + Xn−6 * H1,3 is calculated. .

  In the above convolution calculation, the acoustic blocks Xn-M1, Xn-1-M1, Xn-2-M1,..., Xn-M + 1-M1 have already been calculated in the previous processing and stored in the frequency domain acoustic buffer 15. .

  In steps S28 to S31, Y1 = Xn-Mk * Hk, 0 + Xn-1-Mk * Hk, 1 + Xn-2-Mk * Hk, 2+... + Xn-M + 1-Mk * Hk, M-1 are calculated. Here, Mk is the number of delay blocks of the reflected sound Vk. In the example of FIG. 8, since M = 4 and Mk = 9, Y1 = Xn-9 * Hk, 0 + Xn-10 * Hk, 1 + Xn-11 * Hk, 2 + Xn-12 * Hk, 3 are calculated. .

  In the above convolution calculation, the acoustic blocks Xn-Mk, Xn-1-Mk, Xn-2-Mk,..., Xn-M + 1-Mk are already calculated in the previous processing and stored in the frequency domain acoustic buffer 15. Yes.

  The time domain transforming unit 11 performs complex vector addition on the convolution calculation results Y0, Y1,..., Yk in the frequency domain, and converts the result of the complex vector addition into an acoustic signal yn in the time domain having the sampling frequency fs by IFFT (step S32). ). The acoustic signal output unit 12 outputs the time domain acoustic signal yn (step S33). Thereafter, the value of the variable n is incremented by 1 (step S34), and the processes of steps S12 to S34 are performed. As described above, the first half of the acoustic signal yn is discarded, and the remaining second half is joined to the second half of the acoustic signal yn-1 obtained in the previous process.

(8) Effects of the Embodiment According to the acoustic signal processing apparatus 100 according to the present embodiment, an integer of time in which the delay time difference between the reflected sounds V1, V2,..., Vk and the direct sound V0 corresponds to the FFT shift size. Since the adjustment is performed twice, the acoustic block corresponding to the direct sound V0 that has already been calculated can be used as the acoustic block corresponding to the reflected sounds V1, V2,. Therefore, FFT for obtaining acoustic blocks corresponding to the reflected sounds V1, V2,. Further, the convolution calculation results Y0, Y1,... Yk for the direct sound V0 and the reflected sounds V1, V2,..., Vk are added in the frequency domain, so that the time domain acoustic signal VOUT is obtained by one IFFT. Can be obtained. On the other hand, when the convolution calculation results are added in the time domain, it is considered that (k + 1) times of IFFTs are required for one FFT.

  As a result, the number of calculations in the convolution calculation process can be reduced. As a result, it is possible to reduce the amount of processing in the arithmetic processing for outputting the acoustic signal VOUT.

  Further, since the divided overlap save method using divided HRTF blocks is used, the size of the unit block can be reduced. Thereby, the number of multiplications in FFT and IFFT can be reduced. Therefore, it is possible to further reduce the processing amount in the arithmetic processing for outputting the acoustic signal VOUT.

  Further, since the FFT shift size is equal to the size of the unit block, it is possible to reduce the error due to the adjustment of the delay time difference and the delay time in the convolution calculation by reducing the size of the unit block. Thereby, the sound arriving at the sound receiving point R can be reproduced with higher accuracy.

  As a result, the cost and size of the acoustic signal processing apparatus 100 can be reduced without reducing the sound reproduction accuracy.

(9) Comparison of the number of operations (a) Number of operations in the present embodiment and the reference embodiment Hereinafter, the number of operations in the convolution operation processing according to the present embodiment is compared with the operation circuit in the convolution operation processing according to the reference embodiment.

  In the acoustic signal processing in the reference form, the delay amount of the reflected sounds V1, V2,..., Vk is not adjusted. Therefore, the delay time difference between the reflected sounds V1, V2,..., Vk is not an integral multiple of the time corresponding to the FFT shift size.

  FIG. 12 is a diagram showing a convolution operation between the divided HRTF block and the sound block in the frequency domain in the convolution operation processing according to the reference embodiment. In FIG. 12, time elapses from right to left.

  In the example of FIG. 12, the delay time difference dl1 of the reflected sound V1 and the delay time difference dlk of the reflected sound Vk are not integer multiples of the time corresponding to the FFT shift size SS. The acoustic block X0, n, X0, n-1, X0, n-2, X0, n-3 corresponding to the direct sound V0 is calculated by the FFT of the original acoustic signal VIN, and the acoustic block X1 corresponding to the reflected sound V1 , N, X1, n-1, X1, n-2, X1, n-3 and the acoustic block Xk, n, Xk, n-1, Xk, n-2, Xk, n-3 corresponding to the reflected sound Vk Need to be calculated respectively.

  For the direct sound V0, divided HRTF blocks H0,0, H0,1, H0,2, H0,3 and acoustic blocks X0, n, X0, n-1, X0, n-2, X0, n- in the frequency domain. 3 is performed, and an acoustic signal Y0 in the frequency domain is obtained. For the reflected sound V1, the divided HRTF blocks H1, 0, H1, 1, H1, 2, H1, 3 and the acoustic blocks X1, n, X1, n-1, X1, n-2, X1, n− are divided in the frequency domain. 3 is performed, and an acoustic signal Y1 in the frequency domain is obtained. For the reflected sound Vk, the divided HRTF blocks Hk, 0, Hk, 1, Hk, 2, Hk, 3 and the acoustic blocks Xk, n, Xk, n-1, Xk, n-2, Xk, n- in the frequency domain. A convolution operation with 3 is performed.

  FIG. 13 is a flowchart showing details of the convolution calculation processing according to the reference embodiment.

  In the initial state, the value of the variable n is 0 (step S51). In steps S52 to S59, the convolution calculation result Y0 for the direct sound V0 is calculated. In steps S60 to S67, the convolution calculation result Y1 for the reflected sound V1 is calculated, and in steps S68 to S75, the convolution calculation result Yk for the reflected sound Vk is calculated.

  For the direct sound V0, the signal portion x0, n of the original sound signal VIN is converted into the sound block X0, n by FFT (step S52), and the sound block X0, n is stored in the frequency domain sound buffer 15 (step S53). . For the reflected sound V1, the signal parts x1 and n of the original sound signal VIN are converted into sound blocks X1 and n by FFT (step S60), and the sound blocks X1 and n are stored in the frequency domain sound buffer 15 (step S60). S61). Similarly, for the reflected sound Vk, the signal part xk, n of the original sound signal VIN is converted to the sound block Xk, n by FFT (step S68), and the sound block Xk, n is stored in the frequency domain sound buffer 15 ( Step S69).

  In steps S54 to S59, Y0 = X0, n * H0, 0 + X0, n-1 * H0, 1 + X0, n-2 * H0, 2+... + X0, n−m * H0, m is calculated for the direct sound V0. . In steps S62 to S67, Y1 = X1, n * H1, 0 + X1, n-1 * H1, 1 + X1, n-2 * H1, 2... + X1, n−m * H1, m is calculated for the reflected sound V1. . In steps S68 to S75, Yk = Xk, n * Hk, 0 + Xk, n-1 * Hk, 1 + Xk, n-2 * Hk, 2+... + Xk, n−m * Hk, m are calculated for the reflected sound Vk. .

  The processing in steps S76 to S78 is the same as the processing in steps S31 to S33 in FIG.

  Here, the number of calculations in the convolution calculation process according to the embodiment of FIG. 11 is compared with the number of calculations in the convolution calculation process according to the reference form of FIG.

  The size of the unit block is N samples, the division number of the head-related transfer function is M, and the number of main sound sources and virtual sound sources is k. In this case, the number of samples to be subjected to FFT is 2N.

  When the number of multiplications in the FFT and the number of multiplications in the IFFT is OA, and the number of multiplications in the complex vector product including the loop is OB, the multiplication numbers OA and OB are expressed by the following equations.

OA = 2 × (2N) × log2 (2N)
OB = M × 4 × N
The number of computations PI in the embodiment of FIG.

PI = OA + k × OB + OA
The number of calculations PR in the reference form of FIG.

PR = k × (OA + OB) + OA
Assuming that the size N of the unit block is 32 samples, the division number M of the head-related transfer function is 4, and the number k of the main sound source and the virtual sound source is 100, the multiplication times OA and OB are as follows.

OA = 2 * (2 * 32) * log2 (2 * 32) = 768
OB = 4 × 4 × 32 = 512
As a result, the number of operations PI in the embodiment of FIG.

PI = 768 + 100 × 512 + 768 = 52736
On the other hand, the number of operations PR in the reference form of FIG.

PR = 100 × (768 + 512) + 768 = 128778
The ratio between the calculation number PI and the calculation number PR is calculated as follows.

PI / PR = 52736 / 128778≈0.4
Therefore, according to the convolution operation processing according to the present embodiment, the number of operations is reduced by about 60% compared to the convolution operation processing according to the reference embodiment. As the number of virtual sound sources (the number of reflected sounds) increases, the effect of reducing the number of computations becomes more prominent.

  In the comparison of the number of operations described above, the number of additions and reading / writing with respect to the buffer are not considered.

(B) Number of Calculations in Time Domain Convolution Calculation Processing Next, the number of calculations in acoustic signal processing using time domain convolution calculation processing is calculated.

  The number of operations OT in the time domain convolution operation processing is expressed by the following equation.

OT = k × M × N2
When the size N of the unit block is 32 samples, the division number M of the head related transfer function is 4, and the number k of the main sound source and the virtual sound source is 100, the number of operations OT is as follows.

OT = 100 × 4 × 322 = 409600
As a result, the number of operations PI in the embodiment of FIG.

  The ratio between the number of operations PI in the convolution operation processing according to the present embodiment and the number of operations OT in the time domain convolution operation processing is calculated as follows.

PI / OT = 52736 / 409600≈0.13
Therefore, according to the convolution calculation process according to the present embodiment, the number of calculations is reduced by about 87% compared to the time domain convolution calculation process. As the number of virtual sound sources (the number of reflected sounds) increases, the effect of reducing the number of computations becomes more prominent.

(C) Error due to adjustment of delay amount When the FFT shift size SS is 32 samples, the error of the delay amount due to adjustment of the delay time difference between the reflected sounds V1, V2,... This is the time corresponding to the sample. When the sampling frequency is 48 kHz, the delay amount error is calculated as follows.

16/48000 [Hz] ≈0.00033 [sec] = 0.33 [msec]
The distance error corresponding to the delay amount error is calculated by the following equation.

0.00033 [sec] × 340 [m / sec] ≈0.11 [m] = 11 [cm]
It is considered that the change in reflected sound when the size of the virtual space changes by about 11 cm has little influence on the sound image localization and the sound spread.

  When the FFT shift size SS is 16 samples, the distance error corresponding to the delay amount error is about 5.6 cm, and the influence on the sound image localization and the sound spread is further reduced.

(10) Other Embodiments (a) In the above embodiment, the division overlap save method is used for the convolution operation processing, but the present invention is not limited to this. For example, an overlap-add method using divided HRTF blocks may be used for convolution calculation processing. Hereinafter, the overlap add method using the divided HRTF block is referred to as a divided overlap add method.

  FIG. 14 is an explanatory diagram of a time-domain original sound signal and a frequency-domain sound block when the divided overlap add method is used. In FIG. 14, time elapses from right to left.

  In the divided overlap add method, in the original sound signal VIN, 0 of N samples is added to the unit block vn of N samples currently input, and the signal portion xn of 2N samples is converted into the sound block Xn by FFT. Similarly, 0 of N samples is added to the unit block vn-1, and a signal portion xn-1 of 2N samples is converted into an acoustic block Xn-1 by FFT. Further, 0 of N samples is added to the unit block vn-2, and a signal portion xn-2 of 2N samples is converted into an acoustic block Xn-2 by FFT. Further, 0 of N samples is added to the unit block vn-3, and a signal portion xn-3 of 2N samples is converted into an acoustic block Xn-3 by FFT. Also in this case, the FFT shift size SS is N samples. The convolution operation in the frequency domain is the same as when the division overlap save method is used.

  FIG. 15 is a diagram showing stitching of acoustic signals in the time domain when the divided overlap add method is used. As shown in FIG. 15, the N samples in the first half of the acoustic signal yn obtained by the current processing and the N samples in the second half of the acoustic signal yn-1 obtained by the previous processing are added. By sequentially performing these operations, the acoustic signal VOUT is sequentially output.

  (B) In the above embodiment, the head-related transfer function in the frequency domain is divided into a plurality of divided HRTF blocks. However, the present invention is not limited to this. The present invention is also applied when the division number M of the head-related transfer function is 1. The overlap save method when the division number M is 1 is a normal overlap save method, and the overlap add method when the division number M is 1 is a normal overlap save method.

  Here, it is assumed that the unit block of the acoustic signal in the time domain consists of, for example, 128 samples. In the normal overlap-save method, 128 samples of 0 are added to the 128-sample time-domain head impulse response, and a total of 256 samples are converted into a frequency-domain head-related transfer function by FFT. Also, a 256-sample signal portion composed of the 128-sample acoustic signal input this time and the 128-sample acoustic signal input last time is converted into a frequency-domain acoustic block by FFT. Thereafter, the head-related transfer function in the frequency domain and the acoustic block in the frequency domain are multiplied by a complex vector, and the multiplication result is converted into an acoustic signal in the time domain of 256 samples by IFFT. Finally, half of the time domain acoustic signal is discarded to obtain the remaining 128 samples of the acoustic signal. The 128-sample acoustic signal obtained this time is connected to the 128-sample acoustic signal obtained last time.

  The normal overlap add method differs from the normal overlap save method in the following points. A 128-sample 0 is added to the 128-sample time-domain sound signal input this time, and a 256-sample signal portion including 0 is converted into a frequency-domain sound block by FFT. The 256-sample time-domain acoustic signal obtained by IFFT is added so as to overlap the 256-sample time-domain acoustic signal obtained last time by 128 samples.

  (C) In the above embodiment, the present invention is used to reproduce sound in a virtual space, but the present invention is not limited to this. The present invention can also be applied to a reverberation imparting device for reproducing sound in an actual acoustic space. In this case, a frequency domain acoustic transfer function obtained by performing FFT on the impulse response is used instead of the frequency domain head related transfer function.

  (D) In the above embodiment, the acoustic signal input unit 13 inputs the original acoustic signal VIN and the acoustic signal output unit 12 outputs the acoustic signal yn. However, the present invention is not limited to this. The acoustic signal input unit 13 may input an original acoustic signal in a file format such as a WAV file, and the acoustic signal output unit 12 may output an acoustic signal in a file format such as a WAV file. The present invention can also be applied to an acoustic simulation apparatus for performing acoustic simulation.

  (E) In step S33 of FIG. 11, the acoustic signal output unit 12 may delay and output the acoustic signal yn by the delay amount d0 of FIG.

  (F) In the above embodiment, the acoustic signal processing of FIG. 10 and FIG. 11 is performed for the entire frequency band of the original audio signal VIN, but the present invention is not limited to this. For example, the entire frequency band of the original audio signal VIN may be divided into a high band and a low band, and the above acoustic signal processing may be performed for each of the high band and the low band.

  (G) In the above embodiment, FFT is used as time-frequency conversion for converting an original sound signal in the time domain into an acoustic block in the frequency domain, but the present invention is not limited to this. As time-frequency conversion, other orthogonal transforms such as Laplace transform, Z transform, and Mellin transform may be used. Moreover, in the said embodiment, although IFFT is used as frequency-time conversion for converting the addition result of the convolution calculation result of a frequency domain into the acoustic signal of a time domain, this invention is not limited to this. Other inverse orthogonal transforms such as inverse Laplace transform, inverse Z transform or inverse Merin transform may be used as the frequency-time transform.

  (H) In the above embodiment, the entire acoustic signal processing apparatus 100 operates at the same sampling frequency fs, but is not limited to this. A part of the acoustic signal processing apparatus 100 may operate at a sampling frequency different from the sampling frequency fs by appropriately performing a sampling frequency conversion process.

  (I) In the above embodiment, a plurality of sets of divided HRTF blocks are stored in the HRTF database 3, but for example, a plurality of sets of divided HRTF blocks are stored in a server or the like on the Internet. A plurality of sets of divided HRTF blocks may be downloaded from a server or the like. In this case, the acoustic signal processing apparatus 100 may not include the HRTF database 3.

  (J) Although the single acoustic signal processing apparatus 100 has been described in the above embodiment, a pair of acoustic signal processing apparatuses 100 for the left ear and the right ear may be provided. In this case, some of the components shown in FIG. 1 may be commonly used for the left ear and right ear acoustic signal processing apparatuses 100.

(11) Correspondence between each constituent element of claim and each part of the embodiment Hereinafter, an example of correspondence between each constituent element of the claim and each part of the embodiment will be described, but the present invention is limited to the following example. Not.

  In the above embodiment, the main sound source S0 is an example of the first sound source, the virtual sound sources S1, S2,..., Sk are examples of the second sound source, and the sound receiving point R is an example of the sound receiving point. The direct sound V0 is an example of the first sound, and the reflected sounds V1, V2,..., Vk are examples of the second sound.

  The delay amount calculation unit 6 is an example of a calculation unit, the HRTF database 3 is an example of a storage unit, the delay amount adjustment unit 7 is an example of an adjustment unit, and the acoustic block selection unit 9 is an example of a selection unit, The frequency domain conversion unit 14 is an example of a first conversion unit, the convolution calculation unit 10 is an example of a calculation unit, and the time domain conversion unit 11 is an example of a second conversion unit.

  The divided HRTF blocks H0, 0, H0, 1, H0, 2, H0, 3 are examples of the first acoustic transfer function or the plurality of first divided transfer functions, and the divided HRTF blocks H1, 0, H1, 1, H1,2, H1,3, divided HRTF blocks H2,0, H2,1, H2,2, H2,3 and divided HRTF blocks Hk, 0, Hk, 1, Hk, 2, Hk, 3 are the second sound. It is an example of a transfer function or a plurality of second divided transfer functions, a head impulse response h0 is an example of a first acoustic response characteristic, and head impulse responses h1, h2,..., Hk are second acoustic responses. This is an example of characteristics, and divided HRIR blocks h0, 0, h0, 1, h0, 2, h0, 3 are examples of a plurality of first divided response characteristics.

  The original acoustic signal VIN is an example of the original acoustic signal, the FFT shift size SS is an example of a constant shift amount, and the acoustic blocks Xn, Xn−1,..., Xn−M + 1 are examples of the first signal portion. , Acoustic blocks Xn-M1, Xn-1-M1,..., Xn-M + 1-M1 and acoustic blocks Xn-Mk, Xn-1-Mk,. The acoustic signal VOUT is an example of a time domain acoustic signal, N samples are examples of the first number of samples, and 2N samples are examples of the second number of samples.

  As each constituent element in the claims, various other elements having configurations or functions described in the claims can be used.

  The present invention can be used to reproduce sound that arrives at a sound receiving point in an acoustic space.

DESCRIPTION OF SYMBOLS 1 Room shape instruction | indication part 2 Main sound source position instruction | indication part 3 HRTF database 4 HRTF block selection part 5 Virtual sound source position calculation part 6 Delay amount calculation part 7 Delay amount adjustment part 8 Delay block number calculation part 9 Acoustic block selection part 10 Convolution calculation part DESCRIPTION OF SYMBOLS 11 Time domain conversion part 12 Acoustic signal output part 13 Acoustic signal input part 14 Frequency domain conversion part 15 Frequency domain acoustic buffer 100 Acoustic signal processing apparatus 110 CPU
120 ROM
130 RAM
140 Storage Device 150 Display Device 160 Input Device 170 Output Device 300 Virtual Space

Claims (5)

A first sound radiated by the first sound source and arriving at the sound receiving point and at least one first sound radiated by at least one second sound source and delayed from the first sound and arriving at the sound receiving point An acoustic signal processing apparatus that outputs an acoustic signal representing a sound obtained by mixing two sounds,
A calculation unit for calculating a delay time difference between the first sound and the second sound;
A first transform that obtains a frequency domain acoustic signal by sequentially time-frequency transforming an original acoustic signal representing the first sound emitted from the first sound source while shifting the original acoustic signal by a certain shift amount on the time axis. And
An adjustment unit that adjusts the delay time difference calculated by the calculation unit to an integral multiple of a time corresponding to the shift amount of the time-frequency conversion;
The first signal portion corresponding to the first sound is selected from the frequency domain acoustic signal obtained by the first conversion unit, and based on the delay time difference adjusted by the adjustment unit, the first A selection unit that selects a second signal portion corresponding to the second sound from the frequency domain acoustic signal obtained by the conversion unit;
A first convolution operation between the first sound transfer function from the first sound source to the sound receiving point and the first signal portion selected by the selection unit, and from the second sound source to the sound receiving point. An arithmetic unit that performs a second convolution operation between the second acoustic transfer function and the second signal portion selected by the selection unit in the frequency domain and adds the results of the first and second convolution operations When,
An acoustic signal processing apparatus comprising: a second conversion unit that converts a result of addition by the calculation unit into an acoustic signal in a time domain. The first conversion unit sequentially obtains a unit block of the first number of samples from the original sound signal, and performs high-speed operation of the sound signal of the second number of samples that includes the unit block and is larger than the first number of samples. Fourier transform
The first conversion unit, the calculation unit, and the second conversion unit may perform the fast Fourier transform, the first and second convolution operations, and the time domain acoustic signal by an overlap save method or an overlap add method. Conversion to
The acoustic signal processing apparatus according to claim 1, wherein a shift amount of the fast Fourier transform is equal to the number of samples of the unit block. The first acoustic transfer function includes a plurality of first divided transfer functions, and the plurality of first divided transfer functions is a first acoustic response in a time domain from the first sound source to the sound receiving point. A plurality of first divided response characteristics obtained by dividing the characteristics are obtained by fast Fourier transform,
The second acoustic transfer function includes a plurality of second divided transfer functions, and the plurality of second divided transfer functions are second acoustic responses in a time domain from the second sound source to the sound receiving point. A plurality of second division response characteristics obtained by characteristic division are obtained by fast Fourier transform,
The selection unit selects a first signal portion having a number corresponding to the number of divisions of the plurality of first division transfer functions, and a second number corresponding to the number of divisions of the plurality of second division transfer functions. Select the signal part of
The calculation unit includes the first convolution calculation of the plurality of first division transfer functions and the plurality of first signal portions selected by the selection unit, and the plurality of second division transfer functions and the selection. The acoustic signal processing device according to claim 2, wherein the second convolution operation with a plurality of second signal portions selected by the unit is performed in a frequency domain. The first sound is a direct sound that arrives at the sound receiving point without being reflected from the first sound source, and the second sound is a reflected sound that is reflected while being reflected from the first sound source. The acoustic signal processing apparatus according to claim 1, wherein the second sound source is a virtual sound source that virtually radiates the reflected sound. A first sound radiated by the first sound source and arriving at the sound receiving point and at least one first sound radiated by at least one second sound source and delayed from the first sound and arriving at the sound receiving point An acoustic signal processing program executable by a computer to output an acoustic signal representing a sound mixed with the sound of two,
A process of calculating a delay time difference between the first sound and the second sound;
A process of obtaining an acoustic signal in the frequency domain by performing time-frequency conversion while shifting the original acoustic signal representing the first sound radiated by the first sound source by a certain shift amount on the time axis;
A process of adjusting the calculated delay time difference to an integral multiple of a time corresponding to the shift amount of the time-frequency conversion;
A first signal portion corresponding to the first sound is selected from the frequency domain acoustic signal, and a first corresponding to the second sound is selected from the frequency domain acoustic signal based on the adjusted delay time difference. A process of selecting two signal parts;
A first convolution operation between the first acoustic transfer function from the first sound source to the sound receiving point and the selected first signal portion and a second convolution from the second sound source to the sound receiving point. A process of performing a second convolution operation of the acoustic transfer function and the selected second signal portion in the frequency domain, and adding the results of the first and second convolution operations;
A process of converting the result of the addition into an acoustic signal in a time domain,
An acoustic signal processing program to be executed by the computer.

Images