JP5168373B2 - Audio signal processing method, sound field reproduction system - Google Patents

Description

  The present invention relates to an audio signal processing method suitable for reproducing a sound field of one environment in another environment. Also, a sound comprising a recording device that records information on a recording medium, and an audio signal processing device that generates a reproduction audio signal for reproducing a sound field based on the information recorded on the recording medium. The field reproduction system.

JP 2002-186100 A

For example, when viewing contents such as movies and music, reverberation is added to give the reproduced sound a sense of reality.
As the reverberation adding process, a so-called digital reverb method is known. This digital reverb system generates a lot of delay information, which is a random delay time with respect to the original sound, and further reduces the volume as the delay time becomes longer, or applies feedback at places where the delay time is longer to reduce the reverberation time. Signal processing such as taking a long time is performed. As a result, a reverberation effect can be artificially generated for the original sound. However, since the parameters for generating the delay information are set based on the audibility of the operator who sets the parameters, this setting work is complicated. In addition, there is no concept of localizing the original sound because it artificially creates reverberation, and it is not excellent in reproducing the sound field.

On the other hand, as a technique for actually measuring an impulse response in a sound field space and obtaining a reverberation effect based on spatial information such as localization of a sound source, for example, the technique described in Patent Document 1 is known. Yes.
In the technique described in Patent Document 1, for example, as shown in FIG. 1, a measurement speaker 3 is arranged as a sound source in a measurement environment (measurement sound field) 1 such as a hall. Then, an impulse response measurement audio signal such as a TSP (Time Streched Pulse) signal or the like is supplied to the measurement speaker 3, and a plurality of measurement microphones 4 (a To p), the measurement sound output from the measurement speaker 3 is input. In this case, for example, in the measurement microphone 4a, as indicated by an arrow in FIG. 1, the direct sound from the measurement speaker 3 and the reflected sound output from the measurement speaker 3 and reflected in the hall as the measurement environment Can be detected. Although not shown, this is the same for the other measurement microphones 4b, 4c, and 4d.
Therefore, a transfer function corresponding to each of the measurement microphones 3 to 4 is measured by measuring an impulse response including reverberation based on the audio signals detected by the measurement microphones 4 (a to d). Can be requested.

If such a transfer function is used, as shown in FIG. 3, for example, in an environment in which the speakers 8 (ap) are arranged in the same positional relationship as each measurement microphone 4 in the case of FIG. The sound field in the measurement environment can be reproduced.
That is, when the transfer functions corresponding to the respective arrangement positions of the respective measurement microphones 4 are obtained as described above, the audio signals to be reproduced are convolved using these transfer functions, respectively, so that the arrangement of the speakers 8 is obtained. An audio signal to be output from the position is obtained. Therefore, by outputting these audio signals from the speakers 8 arranged at corresponding positions, the reverberation effect similar to that in the measurement environment of FIG. 1 can be obtained in the space surrounded by the speakers 8.
Since such a method uses a transfer function actually measured, it has excellent sound field reproducibility during reproduction. At the same time, the sound image localization in the reproduction sound field becomes clearer.

  It is important to note that the measurement microphone 4 (ap) in the measurement environment of FIG. 1 and the speaker 8 (ap) in the reproduction environment of FIG. 3 are arranged in a geometrically equivalent positional relationship. It is to make it. By doing in this way, the localization of the sound source of the measurement sound field (sound image localization) can be clearly reproduced in the region (closed curved surface) surrounded by the speaker 8 in the reproduction environment, and the sound field of the measurement environment Can be clearly reproduced.

By the way, in the method described in Patent Document 1, the measurement speaker 3 that outputs the measurement signal in the measurement environment is omnidirectional, thereby enabling sound to be emitted from one point to the entire space. In this way, information (spatial information of the measurement environment) for expressing a sound caused by the size of the measurement space, the material of the wall, floor, ceiling, geometric structure, and the like is measured.
However, in practice, it can be assumed that the directivity of the sound source to be reproduced as a virtual sound image at the position where the measurement speaker 3 is arranged is also reproduced. In such a case, if the sound field is reproduced based on the result of measuring the impulse response using the non-directional measuring speaker 3 as described above, the directivity of the sound source cannot be reproduced. It will be.

Therefore, in the present invention, in view of the above problems, the audio signal processing method is as follows.
That is, the recording step of recording the sound from the sound source with a directional microphone from a plurality of directions so as to surround a required sound source.
Further, the directional speaker arranged at the virtual sound image position outside the first closed curved surface is directed toward a plurality of directions opposite to the plurality of directions in which the sound from the sound source is recorded in the recording step. A first sound generation step for sound generation;
A first measurement step of measuring sounds in the plurality of directions sounded in the first sound generation step at a plurality of positions on the first closed curved surface;
Further, based on the sound measured by the first measurement step, the first transfer function group corresponding to each of the plurality of positions on the first closed curved surface from the virtual sound image position is determined by the plurality of the plurality of transfer functions. It has the 1st transfer function production | generation process produced | generated for every direction.
Further, each of the plurality of positions on the first closed curved surface is subjected to arithmetic processing based on the first transfer function group in a corresponding direction with respect to the audio signal recorded and input in the recording step. The reproduction audio signals for the plurality of directions are respectively obtained as reproduction audio signals corresponding to the above, and the reproduction audio signals are added for each of the plurality of positions on the first closed curved surface . A first reproduction audio signal generation step for obtaining a first reproduction audio signal corresponding to each of the plurality of positions on the closed curved surface.

As described above, the first transfer function group obtained based on the sound produced by directing the directional speaker toward a plurality of directions performs arithmetic processing on the input sound signal, thereby obtaining the sound. The first reproduction audio signal can include information on the direction of the sound produced by the directional speaker.
Therefore, if these first reproduction audio signals are respectively output from reproduction speakers arranged in a geometrically equivalent positional relationship with a plurality of positions on the first closed curved surface, for example, these reproduction speakers will In the enclosed space, the sound field (reverberation and sound image localization) of the measurement environment can be reproduced in the form of the directivity of the virtual sound source.

  As described above, according to the present invention, the reproduction of the sound field of the measurement environment can be realized in the form of expressing the directivity direction of the virtual sound source.

It is a schematic diagram for demonstrating a measurement environment. It is the block diagram shown about the basic composition of the reproduction system of the reproduction sound in a reproduction environment. It is a schematic diagram for demonstrating reproduction environment. It is the figure which showed typically the mode for the measurement in a measurement environment in the case of reproducing a some virtual sound image position. It is the figure shown about the structure of the reproduction signal production | generation apparatus corresponding to the case where a some virtual sound image position is reproduced. It is the figure which showed typically about the reproduction environment in the case of reproducing a some virtual sound image position. It is the figure which showed typically the mode of the measurement in a measurement environment in the case of reproducing the sound field in a 2nd closed curved surface. It is the block diagram which showed the structure of the reproduction signal generation apparatus in the case of performing the sound field reproduction by the 2nd closed curved surface. It is a figure explaining the reverberation sound field at the time of making the inside of a 2nd closed curved surface into a listening position in a reproduction environment, and a sound image localization. It is the figure which showed typically the mode of the measurement environment in the case of reproducing a specific directivity direction. It is a schematic diagram for demonstrating the method of reproducing a specific directivity direction in reproduction | regeneration environment. It is the figure which showed typically the mode of measurement in the measurement environment in the case of simulating a performance form. It is a block diagram shown about the structure of the reproduction signal production | generation apparatus corresponding to the case where the performance form is simulated. It is a data structure figure which shows the example of a data structure of Direction corresponding | compatible information. It is the figure which showed typically the mode of a measurement environment in the case of reproducing two sound sources, Rch and Lch, for one virtual sound image position. It is a block diagram shown about the structure of the reproduction signal production | generation apparatus corresponding to the case where two sound sources, Rch and Lch, are reproduced for one virtual sound image position. It is a figure for demonstrating the recording method of the sound source in the case of performing sound field reproduction in consideration of the directivity of a sound source and the sound emission characteristic for every direction. It is a block diagram which shows about the structure of the reproduction signal production | generation apparatus corresponding to the case where sound field reproduction which considered the directivity of a sound source and the sound emission characteristic for every directivity direction is performed. It is a figure for demonstrating the method of encircling a sound source and recording audio | voice. It is the figure which showed typically the mode of the measurement in a measurement environment in the case of performing the sound field reproduction corresponding to the case where sound recording was performed by three-dimensionally enclosing a sound source. It is the figure which showed typically the mode of the ambience recording in a measurement environment. It is a block diagram shown about the structure of the reproduction signal production | generation apparatus corresponding to the case where sound field reproduction is performed using ambience. It is a figure for demonstrating the measuring method in a measurement environment in the case of performing the sound field reproduction according to a camera angle. It is the figure which showed about the work process which should be performed by the production side in the sound field reproduction system as embodiment, and the structure of a recording device. It is the block diagram which showed the structure of the reproduction signal production | generation apparatus in the sound field reproduction system as embodiment. It is the data structure figure shown about the data structure example of the angle / Direction / transfer function correspondence information.

Hereinafter, modes for carrying out the invention (hereinafter referred to as embodiments) will be described. The description will be made in the following order.

<1. Basic configuration>
1-1. Reproduction of one sound image position 1-2. Reproduction of a plurality of sound image positions 1-3. Sound field reproduction on the second closed surface <2. Sound field reproduction as an embodiment>
2-1. Reproduction of directivity direction of sound source 2-2. Simulation of performance form 2-3. Reproduction of stereo effector 2-4. Reproduction of directivity of sound source and sound emission characteristics for each direction 2-5. Addition of ambience data 2-6. Sound field reproduction according to the camera viewpoint 2-7. Configuration example of sound field reproduction system

In this specification, unless otherwise specified, “calculation processing based on a transfer function” for an audio signal means that the transfer function is subjected to convolution integration processing for the audio signal, or the transfer function is set as a filter coefficient. It means that the audio signal is filtered by an FIR (Finite Impulse Response) filter.

<1. Basic configuration>
1-1. Reproduction of one sound image position

FIG. 1 is a diagram schematically showing a measurement environment for sound field reproduction.
The sound field reproduction technique described in “1. Basic configuration” is a technique used as a basis for realizing the sound field reproduction as the present embodiment described later. It is also described in “JP-A-2002-186100”, which is an earlier application.

In FIG. 1, a measurement environment 1 is a sound field to be reproduced in a reproduction environment described later. In this case, for example, it may be considered as a concert hall or a live venue.
The measurement environment 1 includes, for example, measurement microphones (microphones) 4a, 4b, 4c, 4d, 4e, 4f, 4g, 4h on a circumference having a radius R_bnd at a position not close to the wall of the measurement environment 1. 4i, 4j, 4k, 4l, 4m, 4n, 4o, 4p are arranged.
Hereinafter, the circumference having such a radius R_bnd is referred to as a first closed curved surface 10.

Each measurement microphone 4 (ap) has its directivity directed outward in the normal direction of the first closed curved surface 10. In the following description of the present specification, the arrow shown on the microphone indicates the directivity.
Further, the measurement speaker 3 is arranged as a virtual sound source at a position having a radius R_sp from the center of the first closed curved surface 10. A measurement signal is supplied from the measurement signal reproducing unit 2 to the measurement speaker 3. As this measurement signal, a TSP (Time Streched Pulse) signal for impulse response measurement described later is output.
Note that since the measurement speaker 3 is provided to reproduce a virtual speaker in a reproduction environment described later, it is desirable that the directivity and the frequency characteristic assume an audibility to the listener in the reproduction environment.

In the measurement environment 1, the measurement signal TSP is supplied to the measurement speaker 3 so that the measurement sound output from the measurement speaker 3 is input to each measurement microphone 4 (ap). FIG. 1 schematically shows a voice path from the measurement speaker 3 to the measurement microphone 4a.
The sound signal detected by each measurement microphone 4 (ap) is supplied to an impulse response measurement device (not shown), and here, based on the sound pressure of the sound detected by each measurement microphone 4, the measurement is performed. Impulse responses corresponding to the respective measurement microphones 4 (ap) are measured from the speaker 3. The impulse response may be about 5 to 10 seconds for a large hole or the like, but may be shorter in a small hole or a hole with little sound. By this measurement, a transfer function based on each impulse response can be obtained. That is, FIG. 1 shows a voice path when the transfer function Ha corresponding to the measurement microphone 4a is obtained. Although not shown, the transfer functions Hb to Hp corresponding to the measurement microphone 4b to the measurement microphone 4p can be similarly obtained.
The impulse response measurement may be performed for each measurement microphone, or may be performed simultaneously for all the measurement microphones 4 (ap). Further, the measurement signal is not limited to the TSP signal, and pseudo random noise, a music signal, or the like may be used.
In the following description, the transfer function from the measurement speaker to the measurement microphone in the measurement environment 1 is represented by “H”.

  In this way, in the measurement environment 1, the transfer functions Ha, Hb, Hc, Hd... Hp corresponding to the respective measurement microphones 4a, 4b, 4c, 4d. By using these transfer functions Ha to Hp, the sound field of the measurement environment 1 can be reproduced in an environment (reproduction environment) different from the measurement environment 1.

FIG. 2 shows a configuration of a reproduction sound reproduction system (reproduction signal generation device) in a reproduction environment.
In the reproduction signal generation device 5, the sound reproduction unit 6 is configured to output an arbitrary sound signal S. The audio signal S output from the audio reproduction unit 6 is supplied to the calculation units 7a, 7b, 7c, 7d... 7n, 7o, 7p. Transfer functions to which the same subscripts (alphabetical characters) are attached to the individual calculation units 7 (ap) are transferred to the respective computation units 7 (ap) in correspondence with the measurement microphones 4a to 4p as described above. H is set, and each arithmetic unit 7 performs arithmetic processing based on the set transfer function H on the supplied audio signal S. As a result, the reproduction signals SHa, SHb, SHc, SHd... SHn, SHo, SHp, in which the impulse response corresponding to the transfer function is convoluted with the audio signal S from each arithmetic unit 7 (ap). Is output.
As described above, the operation of each calculation unit 7 can also be realized by an FIR filter in which each set transfer function (impulse response) is set as a filter coefficient. The same applies to all “calculation units” described later.

  Each reproduction signal SH (ap) is supplied to reproduction speakers 8a, 8b, 8c, 8d... 8n, 8o, 8p arranged in the reproduction environment. Thereby, each reproduction speaker 8 (ap) outputs sound based on the reproduction signal SH (ap) based on the transfer function H (ap) in the measurement environment 1.

FIG. 3 is a schematic diagram for explaining the reproduction environment.
The reproduction environment 11 is, for example, an anechoic room or a studio with little reverberation.
The reproduction speaker 8 (ap) shown in FIG. 2 is arranged in the reproduction environment 11. In this case, the reproduction speaker 8 (ap) corresponds to the arrangement position of the measurement microphone 4 (ap) shown in FIG. 1 and is on the outer periphery of the first closed curved surface 10 formed with the radius R_bnd. Arranged toward the inside. That is, the arrangement positions of the reproduction speakers 8 (ap) and the measurement microphones 4 (ap) with the same subscript (alphabet) correspond to each other.
The first closed curved surface 10 in the measurement environment 1 and the first closed curved surface 10 in the reproduction environment 11 are closed curved surfaces that exist in different spaces, but here they are geometrically equivalent formed by the same radius. For convenience, the same reference numerals are attached.

  Then, the reproduction signal SH (ap) is supplied from these reproduction speakers 8 (ap) to the output as shown in FIG. The listener who is present can feel as if the sound field when the audio signal S is reproduced from the measurement speaker 3 shown in FIG.

Here, when there is no sound source in a certain closed curved surface, in order to accurately reproduce the sound field in another place, the sound pressure and normal direction of the outer periphery of the closed curved surface are reproduced between the original sound field and the reproduced sound field. It is known that the particle velocities need to be matched (refer to “Literature: Electronic systems and digital processing” edited by the Institute of Electronics, Information and Communication Engineers (Corona)). Specifically, an infinite number of bidirectional microphones are installed on a closed curved surface, and the sound pressure and particle velocity at each installation point are measured. For this reason, innumerable measurement microphones are installed outward in the normal direction in the first closed curved surface 10 in the measurement environment 1, and innumerable corresponding to these measurement microphones in the first closed curved surface 10 in the reproduction environment 11. By arranging a single speaker for reproduction, when the inner side of the first closed curved surface 10 in the reproduction environment 11 is set as the viewing position, the listener is positioned in the same manner as when the listener is in the first closed curved surface 10 of the measurement environment 1. A virtual sound image can be perceived at the position of the measurement speaker 3 that is not in the reproduction environment 11. That is, at any listening position inside the first closed curved surface 10 of the reproduction environment 11, a sound field feeling equivalent to that of the measurement environment 1 can be obtained on the outside.
However, as described above, it is difficult to actually realize this because it requires an infinite number of microphones and reproduction speakers. Therefore, the present applicant pays attention to the fact that sound pressure and particle velocity components are included in the output of a directional microphone, for example, a unidirectional microphone, and for reproducing a finite number of directional microphones and the corresponding number. Experiments have confirmed that almost the same sound effects can be obtained with speakers.

In this way, the sound field in the measurement environment 1 such as a hall can be reproduced in the reproduction environment 11 that is an anechoic chamber or the like.
Here, according to this, if the impulse response is measured once in the measurement environment 1 as shown in FIG. 1, the measurement data (transfer function) is used thereafter, so that the reproduction environment 11 or the like is used. In an environment other than the measurement environment 1, the sound field of the measurement environment 1 can be simulated at any time.
In this case, according to the configuration shown in FIG. 2, since the sound reproduced in the sound field reproduced in this way can be any sound, any sound is reproduced in the measured hall. Can be reproduced as being performed (any performance was performed).

1-2. Reproduction of multiple sound image positions

In the above description, one sound image position is reproduced in the reproduction environment 11 based on the result of measuring the impulse response from one measurement speaker 3 to each measurement microphone 4 (a to 4p) in the measurement environment 1. However, by applying this technique, it is possible to reproduce a plurality of measurement speakers 3 arranged as shown in FIG. 4 below, that is, a plurality of sound image positions.

  4, first, in this case, in the measurement environment 1 in which measurement microphones 4a to 4p similar to those in FIG. 1 are arranged, a plurality of measurement speakers 3-1, 3-2, 3-3 as shown in the figure. 3-4 are arranged at different locations on the outside of the first closed curved surface 10, respectively. Here, the arrangement position of the measurement speaker 3-1 is represented as Position1, and the arrangement position of the measurement speaker 3-2 is represented as Position2. Similarly, the arrangement positions of the measurement speaker 3-3 and the measurement speaker 3-4 are Position 3 and Position 4, respectively.

The measurement in the measurement environment 1 in this case is performed by supplying the measurement signal TSP to each measurement speaker 3. At this time, each measurement microphone 4 (ap) detects an output audio signal for each measurement speaker 3. The audio signal for each of these measurement speakers 3 obtained by each measurement microphone 4 is again supplied to an impulse response measurement device (not shown), whereby each measurement speaker 3 (3-1 to 3-4) is supplied. To the measurement microphones 4 (a to p) corresponding to the impulse responses are measured, and based on the results, transfer functions corresponding to the measurement microphones 3 to the measurement microphones 4 can be obtained. .
For example, in FIG. 4, a path for obtaining a transfer function Ha-1 corresponding to the measurement speaker 3-1 to the measurement microphone 4a and a transfer function Hb-1 corresponding to the measurement speaker 3-1 to the measurement microphone 4b is obtained. This is shown schematically. Similarly, a path for obtaining a transfer function Ha-3 corresponding to the measurement microphone 3-3 to the measurement microphone 4a and a transfer function Ho-3 corresponding to the measurement speaker 3-3 to the measurement microphone 4o. Is also shown schematically.

In this way, according to the measurement signal TSP output for each measurement speaker 3, the transfer functions Ha-1 to Hp-1 corresponding to the measurement microphones 4a to 4p are measured. Transfer function Ha-2 to Hp-2 corresponding to each measurement microphone 4a-p from the measurement speaker 3-2, and transfer function Ha-3 corresponding to each measurement microphone 4a to p from the measurement speaker 3-3. Transfer functions Ha-4 to Hp-4 corresponding to Hp-3 and the measurement microphones 3-4 to the measurement microphones 4a to 4p can be obtained.
The measurement of the impulse response in this case is preferably performed by outputting the measurement signal TSP for each measurement speaker 3 so that the sound from the measurement speakers 3 of different Positions is not mixed. In addition to arranging a plurality of measurement speakers 3, one measurement speaker 3 can be sequentially arranged in each position.

FIG. 5 shows reproduction for performing sound field reproduction based on these transfer functions Ha-1 to Hp-1, Ha-2 to Hp-2, Ha-3 to Hp-3, Ha-4 to Hp-4. 2 shows a configuration of a reproduction signal generation device 15 that generates a sound signal for use (also simply referred to as a reproduction signal).
For example, the reproduction signal generation device 15 employs a configuration corresponding to a case where different sounds are output for each of a plurality of sound image positions (Position 1 to Position 4). For this purpose, a total of four audio reproducing units 6-1, 6-2, 6-3, and 6-4 corresponding to each Position are provided.
Also in this case, each audio reproduction unit 6 can output an arbitrary audio signal S. Here, the audio signal S output from each audio reproduction unit 6 is shown as audio signals S1, S2, S3, and S4 corresponding to the respective Position numbers.

In this case, four sets corresponding to each of Position 1 to Position 4 are provided as the calculation unit 7. That is, arithmetic units 7a-1 to 7p-1 corresponding to Position 1, arithmetic units 7a-2 to 7p-2 corresponding to Position 2, arithmetic units 7a-3 to 7p-3 corresponding to Position 3, and arithmetic units corresponding to Position 4 7a-4 to 7p-4 are provided.
For the computing units 7a-1 to 7p-1, transfer functions Ha-1 to Hp- obtained according to outputs from the measurement speakers 3-1 (Position 1) to the respective measurement microphones 4 as shown in the figure. 1 is set. These arithmetic units 7a-1 to 7p-1 perform arithmetic processing based on the set transfer function H on the audio signal S1 input from the audio reproducing unit 6-1, respectively, and reproduce signals SHa-1 to SHp. -1 is output. Thereby, first, a reproduction signal for reproducing the sound image position of the measurement speaker 3-1 (Position 1) is obtained.
For the calculation units 7a-2 to 7p-2, transfer functions Ha-2 to Hp-2 obtained according to outputs from the measurement speaker 3-2 (Position 2) to each measurement microphone 4 are obtained. These arithmetic units 7a-2 to 7p-2 perform arithmetic processing based on the set transfer function H for the audio signal S2 input from the audio reproducing unit 6-2, respectively, and reproduce the signal SHa-2. ~ SHp-2 is output. Thus, a reproduction signal for reproducing the sound image position of the measurement speaker 3-2 (Position 2) is obtained.
Similarly, in the calculation units 7a-3 to 7p-3, transfer functions Ha-3 to Hp-3 obtained according to the measurement speaker 3-3 (Position 3) are set. An arithmetic process based on the set transfer function H is performed on the input audio signal S3, and reproduction signals SHa-3 to SHp-3 are output. As a result, a reproduction signal for reproducing the sound image position of the measurement speaker 3-3 (Position 3) is obtained.
Furthermore, the calculation units 7a-4 to 7p-4 are set with transfer functions Ha-4 to Hp-4 obtained in accordance with the measurement speaker 3-4 (Position 4), and are input from the sound reproduction unit 6-4, respectively. The reproduction signal SHa-4 to SHp-4 is output by performing arithmetic processing based on the set transfer function H for the audio signal S4 to be performed. Accordingly, a reproduction signal for reproducing the sound image position of the measurement speaker 3-4 (Position 4) is obtained.

The adders 9a to 9p are provided in a one-to-one relationship with the reproduction speakers 8a to 8p. The calculation units 7a-1 to 7p-1, the calculation units 7a-2 to 7p-2, and the calculation unit 7a-3. ˜7p-3, among the computation units 7a-4 to 7p-4, the output from the computation unit 7 with the corresponding subscript (alphabet) is input and added to the corresponding subscript. To the reproduction speaker 8.
That is, for example, the adder 9a inputs four reproduction signals SHa-1, SHa-2, SHa-3, and Sha-4 from the arithmetic units 7a-1, 7a-2, 7a-3, and 7a-4, These are added and supplied to the reproduction speaker 8a. As a result, the reproduced sound corresponding to the route from all the positions shown in FIG. 4 to the measurement microphone 4a can be output from the speaker 8a.
Further, the adder 9p inputs the four reproduction signals SHp-1, SHp-2, SHp-3, SHp-4 from the arithmetic units 7p-1, 7p-2, 7p-3, 7p-4, and these Are added to the reproduction speaker 8p. As a result, the reproduced sound corresponding to the route from all the positions shown in FIG. 4 to the measurement microphone 4p can be output from the speaker 8p.
By adding the reproduction signals SH in the same manner in the adders 9b to 9o, the corresponding speakers 8b to 8o also correspond to the paths from all the positions to the corresponding measurement microphones 4 in the same manner. A reproduction signal can be output.
As a result, the listener inside the first closed curved surface 10 in the reproduction environment 11 surrounded by the reproduction speakers 8a to 8p is connected to each of the measurement speakers 4 (Position 1, Position 2, Position 3, Position 4) shown in FIG. It can be felt that the sound field when the sound is reproduced from each of them is reproduced in a pseudo manner outside the first closed curved surface 10. That is, it is possible to reproduce (localize and present) a sound image at each position of Position1, Position2, Position3, and Position4.

FIG. 6 schematically shows the reproduction environment 11 in this case.
The reproduction signal generation device 15 shown in FIG. 5 is configured so that different sounds can be input independently for each of Position1, Position2, Position3, and Position4. According to this, by inputting voices of different players such as vocals, drums, guitars, keyboards (keyboard instruments), etc. for each Position, as shown in FIG. Player 1), Position 2 drum (Player 2), Position 3 guitar (Player 3), Position 4 keyboard (Player 4), etc. You can present the sound of the appropriate Player at the appropriate Position.

1-3. Sound field reproduction on the second closed surface

Here, in the sound field reproduction method as described above, as the number of the measurement speakers 4 and the number of the reproduction speakers 8 in the reproduction environment 11 are increased, the sense of localization of the sound image (the sound field) Reproducibility) can be increased. According to this, the reproduction environment 11 is preferably an environment in which as many reproduction speakers 8 as possible can be arranged, but the actual reproduction environment may be, for example, a general home room. It is done.
In an environment such as a room in a general household, the number of speakers arranged is limited, and in addition to the limited number of speakers arranged in this way, the arrangement relationship of each speaker in each home is different. Is expected. Therefore, when sound field reproduction in a general home is assumed, measurement is performed for each of the arrangement relation / number of measurement microphones 4 according to each condition in a measurement environment such as a hall according to those conditions. Need to do.
However, according to this, for example, when it is necessary to correspond to the condition of the number and arrangement of new speakers, it is necessary to go to the target hole one by one and perform measurement with the arrangement of the measurement microphone according to the condition. In this respect, much labor and cost are required.

Here, as described so far, the fact that the sound field in the measurement environment 1 can be reproduced inside the first closed curved surface 10 in the reproduction environment 11 means that the second inside further on the first closed curved surface 10. Even on a closed curved surface, if a calculation process using a transfer function from each speaker on the first closed curved surface 10 is performed, a reproduction signal for reproducing the sound field of the measurement environment 1 can be obtained.
That is, this makes it possible to reproduce the sound field of the measurement environment 1 within the second closed curved surface.
According to this, for example, if the measurement in the target hall is performed once, the measurement for adapting to the actual reproduction environment such as the room in the home does not have to go to the target hall. For example, in a laboratory facility as the reproduction environment 11, it may be performed from each of the reproduction speakers 8 on the first closed curved surface 10 to each measurement microphone on the second closed curved surface 14.

For confirmation, the sound field reproduction on the first closed curved surface 10 in the reproduction environment 11 is not limited to being applied to a home room or the like in the experimental facility described above. Various applications can be assumed.
For example, as a live event, in addition to the actual performance of the artist in the venue (hall), a venue where a screen for displaying images at an actual live venue, such as a so-called film live, is arranged. In some cases, live audio is played.
In such a film live venue, a relatively large number of reproduction speakers 8 can be arranged (that is, many measurement microphones 4 can be arranged at the time of measurement). On the basis of this, if the reproduced sound is output from these reproduction speakers 8, the sound field just like an actual live venue can be reproduced. At this time, if the position for each player is determined in advance, measurement is performed in advance for each position at the actual venue, and at the time of reproduction, the sound of the player corresponding to that position is measured at that position. By performing arithmetic processing based on the result (transfer function), the sound image position of each player can also be accurately reproduced.

FIG. 7 is a schematic diagram for explaining an impulse response measurement technique when reproducing the sound field on the second closed curved surface in the first closed curved surface 10 as described above.
In order to simplify the description, an example is given in which only one measurement speaker 3 is arranged in the measurement environment 1 and only one sound image position is reproduced.
7, in this case, measurement microphones 13A, 13B, 13C, 13D, and 13E are arranged inside the first closed curved surface 10 in the reproduction environment 11. These measurement microphones 13 (A to E) may have an arrangement state corresponding to a reproduction speaker arranged in a reproduction environment (reproduction environment 20 to be described later) such as a home room, for example. The number and the arrangement relationship are not limited to those shown in the figure.
In this figure, a closed curved surface formed with these measurement microphones 13 (A to E) as the outer periphery is represented as a second closed curved surface 14. The inside of the second closed curved surface 14 is a listening position in a reproduction environment such as a home room.
Since the second closed curved surface 14 needs to be formed inside the first closed curved surface 10, when measuring in the measurement environment 1, the first closed curved surface 14 is taken into consideration in consideration of the width of the second closed curved surface 14. It is desirable to form the curved surface 10.
In this case, it is preferable to arrange as many measurement microphones 4 as possible in the hall, and obtain transfer functions H for as many points as possible on the first closed curved surface 10. As a result, in the measurement environment 1 → the reproduction environment 11, the target sound field can be reproduced with higher reproducibility, and higher reproducibility can be realized as adaptation to a reproduction environment such as a home room.

  In this case, as shown in the figure, the measurement signal reproduction device 2 outputs the measurement signal TSP for each reproduction speaker 8 (ap) arranged on the first flat curved surface 10, The impulse response corresponding to each of the measurement microphones 13 from the speaker 8 is measured. From these impulse responses, it is possible to obtain a transfer function in each path of each speaker 8 → each measurement microphone 13.

The transfer function from the reproduction speaker arranged on the first closed curved surface 10 to the measurement microphone arranged on the second closed curved surface 14 is represented by “E”.
For example, as shown in FIG. 7, the transfer function from the reproduction speaker 8a to the measurement microphone 13A is represented as Ea-A. The transfer function from the reproduction speaker 8b to the measurement microphone 13A is represented as Eb-A, and the transfer function from the reproduction speaker 8c is represented as Ec-A.
Although not shown, transfer functions from the reproduction speaker 8a to the remaining measurement microphones 13B to 13E are represented by Ea-B, Ea-C, Ea-D, Ea-E, and the above reproduction. Eb-B, Eb-C, Eb-D, Eb-E transfer functions from the speaker 8b for measurement to the measurement microphones 13B to 13E, and Ec− transfer functions from the reproduction speaker 8c to the measurement microphones 13B to 13E. B, Ec-C, Ec-D, Ec-E. Similarly, the transfer function E is expressed in such a manner that the lowercase alphabet indicates the measurement speaker 8 and the uppercase alphabet after the hyphen indicates the measurement speaker 13 and indicates the correspondence from the speaker to the microphone.

  Here, if the transfer function E obtained as described above is used, the sound field reproduced inside the first closed curved surface 10 can be reproduced in the second closed curved surface 14. As described above, since the sound field of the measurement environment 1 can be reproduced using the transfer function H inside the first closed curved surface 10 in the reproduction environment 11, according to this, even in the second closed curved surface 14. The sound field of the measurement environment 1 can be reproduced.

FIG. 8 shows the configuration of the reproduction signal generation device 19 for reproducing the sound field of the measurement environment 1 in the second closed curved surface 14 in this way.
In this figure, the reproduction speakers arranged in the actual reproduction environment 20 such as a home room are reproduction speakers 18A, 18B... 18E as shown.
First, the audio signal S from the audio reproduction unit 6 is input to each of the calculation units 7a to 7p in which the transfer functions Ha to Hp are set in the same manner as shown in FIG. As described above, the audio signals S are calculated based on the transfer functions Ha to Hp by the calculation units 7a to 7p, so that reproduction signals SHa to SHp corresponding to the reproduction speakers 8a to 8p are obtained.

Here, as can be seen with reference to FIG. 7, the output sound from the reproduction speaker 8 on the first closed curved surface 10 in this case is input to each microphone 13 on the second closed curved surface 14. The Accordingly, as the transfer function E, a number corresponding to the reproduction speakers 8a to 8p on the first closed curved surface 10 is obtained for each measurement microphone 13. That is, Ea-A, Eb-A ... Ep-A corresponding to the measurement microphone 13A, Ea-B, Eb-B ... Ep-B, measurement corresponding to the measurement microphone 13B. Corresponding to the microphone 13C, Ea-C, Eb-C... Ep-C are obtained. Further, Ea-D, Eb-D... Ep-D are obtained corresponding to the measurement microphone 13D, and Ea-E, Eb-E... Ep-E are obtained corresponding to the measurement microphone 13E. Is.
Accordingly, in order to obtain a reproduction signal corresponding to each position of each measurement microphone 13 on the second closed curved surface 14 (that is, the reproduction speaker 18 in the actual reproduction environment 20), the position of the measurement microphone 13 as shown in FIG. The arithmetic units 16a-A to 16p-A, 16B-a to 16B-p,... 16E-a to which the transfer functions E of a to p for the respective microphones 13 are set for each (A to E). 16E-p is provided.
Then, as shown in the figure, for these arithmetic units 16a-A to 16p-A, 16B-a to 16B-p ... 16E-a to 16E-p, the reproduction signal SH from the above-described arithmetic units 7a to 7p. ... To SHp, the reproduction signals SH with the corresponding subscripts are supplied, so that each calculation unit 16 performs calculation processing on the input reproduction signal SH based on the transfer function E set for each. To.

With such a configuration, the measurement microphones 13A to 13E are arranged in accordance with the respective paths of the measurement speakers 8a to 8p on the first closed curved surface 10 for each arrangement position (reproduction speakers 18A to 18E). A reproduction signal SHE calculated by the transfer function E can be obtained.
That is, for example, for the measurement microphone 13A (reproduction speaker 18A), the reproduction signals SHEA-a to SHEA-p calculated by the transfer function E corresponding to the path from each of the measurement microphones 8a to 8p. Is obtained. Similarly, for the measurement microphone 13B (reproduction speaker 18B), reproduction signals SHEB-a to SHEB-p processed by the transfer functions E corresponding to the respective paths from the measurement microphones 8a to 8p are provided. It can be obtained.
Similarly, the outputs from the arithmetic units 16a-C to 16p-C, 16D-a to 16D-p, and 16E-a to 16E-p are respectively SHEC-a to SHEC-p and SHED-a to SHED-p. , SHEE-a to SHEE-p.

Adders 17A, 17B... 17E are provided in a one-to-one relationship with reproduction speakers 18A, 18B.
As shown in the figure, adders 17A, 17B,... 17E are connected to SHEA-a to p from arithmetic units 16a-A to 16p-A and SHEB-a to p from 16a-B to 16p-B, respectively. SHEE-ap from -E to 16p-E are input, added, and supplied to the corresponding speaker 18 among the reproduction speakers 18A, 18B.
As understood from the above description, the reproduction signals SHE (ap) input to each of the adders 17 are processed based on the transfer function H and the transfer function E, respectively, and each measurement microphone 13 (reproduction). The reproduction signal corresponds to each speaker 18).
Therefore, as described above, the adders 17 add them and supply them to the corresponding reproduction speakers 18, so that the reproduction signals SHE for reproducing the sound field in the measurement environment 1 are respectively output from the reproduction speakers 18. (SHEA, SHEB... SHEE) is output. That is, in the actual reproduction environment 20 in which such a reproduction speaker 18 is arranged in the same manner as the measurement microphone 13 on the second closed curved surface 14, the sound field of the measurement environment 1 is generated within the second closed curved surface 14. It can be reproduced.

FIG. 9 shows the actual reproduction environment 20, the measurement environment 1 as a virtual sound field, and the first closed curved surface 10 when the sound field of the measurement environment 1 is reproduced by the second closed curved surface 14 in this way. This is shown schematically.
The reproduction speakers 18 (A to E) in the reproduction environment 20 are arranged on the second closed curved surface 14 having the same radius as the second closed curved surface 14 shown in FIG. A to E) are arranged in the same positional relationship. That is, each reproduction speaker 18 in the reproduction environment 20 is arranged in a geometrically equivalent positional relationship with each measurement microphone 13.
Then, as shown in the figure, these reproduction speakers 18 (A to E) are arranged on the second closed curved surface 14 so as to face inward, and from the reproduction speaker 18A, the reproduction signal SHEA and the reproduction speaker 18B. Is output from the reproduction speaker SHC, the reproduction signal SHEC from the reproduction speaker 18C, the reproduction signal SHED from the reproduction speaker 18D, and the reproduction signal SHEE from the reproduction speaker 18E. A person can feel a sound field equivalent to the sound field reproduced by the reproduction speaker 8 (ap) arranged on the first closed curved surface 10 indicated by a broken line. That is, the sound field of the measurement environment 1 indicated by the broken line (the sound image position of the reverberation / measurement speaker 3) can be perceived in a pseudo manner. Therefore, when the inside of the second closed curved surface 14 is set as the listening position, a reverberant sound field and sound image localization in the measurement environment 1 can be obtained. As a result, while being in a room at home, for example, a sound as content can be heard by a reverberant sound field such as a hall and sound image localization.

Here, the case where only one Position is assumed as an example in which only one measurement speaker 3 is arranged in the measurement environment 1 is illustrated. However, when a plurality of Positions are assumed, the number of positions increased. The configuration of the previous stage may be added to each adder 17 shown in FIG. That is, for example, assuming two Positions, Position1 and Position2, in addition to the configuration shown in FIG. 8, an audio playback unit 6 (6-2) for Position 2 and arithmetic units 7a to 7p (7a-2 to 7p) -2), arithmetic units 16A-a to 16A-p, 16B-a to 16B-p ... 16E-a to 16E-p (16A-a-2 to 16A-p-2, 16B-a-2 to 16B-p-2... 16E-a-2 to 16E-p-2) are added. In addition, reproductions output from the added arithmetic units 16A-a-2 to 16A-p-2, 16B-a-2 to 16B-p-2, ... 16E-a-2 to 16E-p-2 In this case, the adders 17A to 17E may be configured to input and add signals with the same subscripts (upper case alphabets).
However, in this case, the calculation units 7a to 7p and the calculation units 7a-2 to 7p-2 that process the reproduction signal S based on the transfer function H in the measurement environment 1 → the first closed curved surface 10 respectively set the transmissions. The function H (ab) is different. That is, for example, if the transfer function H set in the calculation units 7a to 7p is Ha-1 to Hp-1 corresponding to each measurement microphone 8 from Position 1, the transfer set in the calculation units 7a-2 to 7p-2. The function H measures Ha-2 to Hp-2 corresponding to each measurement microphone 8 from Position2.
With the above configuration, the outputs of the adders 17A to 17E include the sound image positions of Position 1 and Position 2 in the correspondence of the measurement environment 1 → the first closed curved surface 10, and the first closed curved surface 10 → the second closed curved surface 14. As a result, the reproduction signal SHE (A to E) adapted by the transfer function E is obtained. Thereby, each reproduction speaker 18 (A to E) can output a reproduction signal that can reproduce the sound image positions of Position 1 and Position 2, so that the second closed curved surface 14 can also listen to the inside thereof. The person can perceive the respective sound images at Position 1 and Position 2 in the measurement environment 1 as a virtual sound field.

<2. Sound field reproduction as an embodiment>
2-1. Reproduction of direction of sound source

Here, in the sound field reproduction method described so far, the measurement speaker 3 that outputs the measurement signal in the measurement environment 1 is omnidirectional, thereby releasing sound from one point to the entire space. It is possible to measure information (spatial information of the measurement environment 1) for expressing the sound caused by the size of the measurement space, the material of the wall, floor, ceiling, geometric structure, etc. It was.
However, in reality, a sound source having a directivity can be assumed as a sound source to be reproduced as a virtual sound image at the position where the measurement speaker 3 is arranged. In such a case, if the sound field is reproduced based on the result of measuring the impulse response using the non-directional measuring speaker 3 as described above, the directivity of the sound source cannot be reproduced. It will be.

Therefore, in the present embodiment, a directional speaker is used as a measurement speaker that outputs a measurement signal in the measurement environment 1, and based on the result of measuring the impulse response in the required direction, Reproduce the place.
FIG. 10 is a schematic diagram of the measurement environment 1 when reproducing a specific directivity direction of a sound source as sound field reproduction as the present embodiment.
10, in this case as well, in the measurement environment 1, the measurement microphones 4a to 4p are arranged outward on the first closed curved surface 10, respectively. In this case, the measurement signal TSP is output in a state in which the unidirectional measurement speaker 21 is directed in a specific direction as shown in the drawing as the directional measurement speaker. Thereafter, the impulse response corresponding to the measurement microphones 4a to 4p is measured in the same manner as before, and the transfer function H is obtained.
Here, the direction in which the measurement speaker 21 is directed in FIG. 10 is Direction2, and the arrangement position of the measurement speaker 21 is Position1.
The transfer functions H obtained for each of the measurement microphones 4 (ap) as described above in the state directed to Direction 2 are the transfer functions in the order of the measurement microphones 4a, 4b,. It is expressed as Ha-dir2, Hb-dir2, Hc-dir2,... Hp-dir2.

FIG. 11 schematically shows how the sound field of the measurement environment 1 is reproduced in the reproduction environment 11 based on the transfer function H thus obtained.
First, in this case, the line recording sound source (Player 1) 22 is assumed as the sound source. The line recording sound source 22 is a sound source directly recorded from a target player. For example, in a vocal, an electric signal detected by a microphone is captured. In an electric instrument such as a guitar or a keyboard, an audio output terminal is used. The electric signal from is directly taken in.
For confirmation, “Player” here corresponds to each of the virtual sound image positions (Positions) to be reproduced, as shown in FIG. For example, it corresponds to each performer such as vocals, drums, guitars and keyboards. Here, assuming that Player 1 is a vocal, the virtual sound image is indicated by a broken line.

Also in this case, in the reproduction environment 11, reproduction speakers 8 a to 8 p are arranged on the first closed curved surface 10 with the same positional relationship as the measurement microphones 4 a to 4 p in the measurement environment 1.
Then, an audio signal as line recording data from the line recording sound source 22 is converted into transfer functions Ha-dir2, Hb-dir2, Hc-dir2,... Hp-dir2 in which the sound source directing direction is added as described above. Calculation processing is performed based on each of them, and these are output from the corresponding reproduction speakers 8.
As a result, the listener in the first closed curved surface 10 can perceive that Player 1 is emitting sound in the direction indicated by the arrow in the drawing at the virtual sound image position of Position 1 in the measurement environment 1. it can. That is, in this way, in the reproduction environment 11, it is possible to reproduce a sound field when sound is emitted from the virtual sound image position of Position 1 in the measurement environment 1 toward a specific directivity direction.
In this case, as a configuration of a reproduction signal generation device that generates a reproduction signal to be output by each of the speakers 8a to 8p, the transfer function H set in each arithmetic unit 7 is different from that shown in FIG. Other than that, the configuration is the same.

2-2. Simulate performance

If a specific directivity direction can be expressed in this way, it is possible to simulate a performance form such as a player such as a vocal or a guitar looking back during performance or turning an instrument. The method will be described below.
FIG. 12 is a diagram schematically showing the state of the measurement environment 1 for realizing the performance form simulation in this way.
First, in this case, the impulse response is measured by directing the measurement speaker 21 in each direction around the virtual sound image position. Here, a speaker having a directivity of 60 ° is used as the measurement speaker 21, and six directions (Direction1, Direction2,..., Direction6) are defined as the direction of sound source.
Then, the impulse response when the measurement speaker 21 is directed to each direction is measured by the measurement microphones 4 (ap) arranged on the first closed curved surface 10 as shown in the figure, and the measurement speaker 21 is measured. → A transfer function H corresponding to each measurement microphone 4 is obtained for each direction.
At this time, the transfer function H to each measurement microphone 4 (ap) when Direction 1 is set is expressed as “Ha-dir1, Hb-dir1,... Hp-dir1”. Similarly, the transfer functions to each measurement microphone 4 (ap) when Direction 2, Direction 3, Direction 4, Direction 5, and Direction 6 are set to “Ha-dir2, Hb-dir2,... Hp-dir2,” “Ha”, respectively. -dir3, Hb-dir3 ... Hp-dir3 "," Ha-dir4, Hb-dir4 ... Hp-dir4 "," Ha-dir5, Hb-dir5 ... Hp-dir5 "," Ha-dir6 " , Hb-dir6... Hp-dir6 ”.

  In this way, by obtaining the transfer function H for each direction, the arithmetic processing for the input sound signal is performed while changing to the transfer function H by different direction as time passes, thereby directing the sound emitted from the sound source. The directions can be different sequentially. For example, by changing the transfer function H used for calculation processing to one corresponding to Direction1 → Direction2 → Direction3 ... → Direction6, the player at the virtual sound image position is Direction1 → Direction2 → Direction3 ... → Direction6 You can reproduce the state of sounding while rotating.

FIG. 13 shows a configuration of the reproduction signal generation device 25 in the case where such directivity control is performed.
This figure shows a configuration corresponding to a case where a plurality of Positions (Position 1 to Position 4) in the measurement environment 1 are reproduced as described with reference to FIGS.
As described above, when a plurality of Positions are assumed, the transfer function H is obtained by measuring the impulse response of the measurement speakers 21 (21-1 to 21-4) arranged in each Position by the same method as described in FIG. It can obtain | require based on the result of having measured.

In FIG. 13, since the reproduction signal generating device 25 in this case corresponds to a plurality of Positions (1 to 4) as described above, each Position is similar to the one shown in FIG. Each voice reproduction unit (6-1 to 6-4) and a calculation unit for each Position are provided.
Also in this case, the audio reproduction units corresponding to each Position (each Player) are shown as audio reproduction units 6-1, 6-2, 6-3, and 6-4 in order from Position 1. In addition, the calculation units for each Position are set to calculation units 26a-1 to 26p-1, 26a-2 to 26p-2, 26a-3 to 26p-3, and 26a-4 to 26p-4 in order from Position1.
Further, even in this case, adders 27a to 27p provided in a one-to-one relationship with the reproduction speakers 8a to 8p are provided. These adders 27a to 27p are arithmetic units 26a-1 to 26p-1, arithmetic units 26a-2 to 26p-2, arithmetic units 26a-3 to 26p-3, and arithmetic units 26a-4 to 26p-4. The outputs from the operation unit 26 with the corresponding subscript (alphabet) are input, added, and supplied to the corresponding reproduction speaker 8. As a result, each reproduction speaker 8 can output a reproduction signal representing the sound image position of each Position.

In addition, in this case, a controller 28 and a ROM 29 are provided as a configuration for sequentially changing and setting the transfer function H obtained for each direction as described above to control the directivity direction.
In the ROM 29, information on the transfer function H for each Position and each Direction obtained as a result of measurement in the measurement environment 1 in advance is stored as Direction corresponding information 29a.
FIG. 14 shows the data structure of the Direction correspondence information 29 a stored in the ROM 29. As shown in this figure, in this case, information on the transfer function H when the measurement speaker 21 is directed to each direction is obtained for each Position.
Here, regarding the transfer functions Ha to Hp corresponding to the respective measurement microphones 4 (a to p), the position is represented by a number immediately after that. Furthermore, each “Direction” is represented by the number “-dir” appended thereto. For example, the transfer function H to the measurement microphone 4a when the Position 1 measurement speaker 21 is directed in the direction 2 is expressed as “Ha1-dir2”, and the Position 3 measurement speaker 21 is directed in the direction 6 The transfer function H to the measurement microphone 4b is expressed as “Hb3-dir6”.

The controller 28 is issued from each Position by variably setting the transfer function H set in each calculation unit 26 based on the information of the transfer function H for each Position / Direction stored in the ROM 29. Controls the sound direction.
For example, when the directivity direction for Position 1 is rotated in the direction of Direction 1 → Direction 2 → Direction 3, the transfer functions Ha 1 -dir 1 to Hp 1 -dir 1 → Ha 1 -dir 2 to Hp 1 -dir 2 → Ha 1 -dir 3 to Hp 1 -dir 3 are read from the ROM 29. Are sequentially read out, and these are sequentially set to the arithmetic units 26a-1 to 26p-1. Thereby, the directivity direction about Position1 can be rotated in order from Direction1 → Direction2 → Direction3 with time.

Here, in order to express a smoother rotation, it is necessary to shorten the direction interval to be changed as much as possible. For this purpose, it is conceivable to define a finer division of the Direction and obtain the transfer function H for more Direction.
However, this increases the number of measurements and is not realistic, so when actually expressing the rotation, the transfer function H between adjacent directions is interpolated to generate a transfer function H for each direction with finer divisions. These are sequentially changed and set as time passes. According to this, smooth rotation can be expressed even when relatively few Direction are defined.

Here, the configuration for instructing the controller 28 about the change direction of the directivity direction is omitted, but such an instruction of the change direction of the directivity direction is provided, for example, by a user operation by providing an operation unit. Based on this, or by giving in advance instruction information for instructing which direction should be set at which timing on the reproduction time axis of the audio signal.
This also applies to the designation of a sound source (Position) that is a target of directivity control.

2-3. Stereo effector reproduction

By the way, in the description so far, it is assumed that the input audio signal is a monaural audio signal, but the input audio signal may be a stereo audio signal. For example, for an electric musical instrument such as an electric guitar, the output sound itself is monaural. However, when a so-called effector for effecting the sound is passed, the monaural signal may be processed and output as a stereo signal.
When it is desired to reproduce the effect by such an effector as it is, it is conceivable to reproduce two sound sources, Rch (channel) and Lch, for one virtual sound image position. Here, an example is given in which such reproduction of Rch and Lch is performed using the concept of the directivity direction of the sound source described so far.

FIG. 15 schematically shows the state of the measurement environment 1 when reproducing two sound sources of Rch and Lch for one virtual sound image position in this way.
Here, if each sound source is based on Rch and Lch, the directivity directions corresponding to these sound sources may be reversed, or at least not in the same direction. In this case, therefore, Direction 6 is defined for the directivity direction of the Rch sound source and Direction 2 is defined for the directivity direction of the Lch sound source as shown in the figure.
Correspondingly, measurement in this case includes impulse response to each measurement microphone 4 when the measurement speaker 21 is directed in the Direction 6 direction corresponding to the Rch sound source, and Direction 2 direction corresponding to the Lch sound source. The impulse response to each measurement microphone 4 when the measurement speaker 21 is directed to is measured, and thereby the respective transfer functions H corresponding to the Rch sound source and the Lch sound source are obtained.
Also in this case, if the position of the measurement speaker 21 is Position 1, the transfer function H obtained for each microphone 4 when directed to Direction 6 as described above is “Ha 1 -dir 6, Hb 1 -dir 6. Hp1-dir6 ”. Similarly, the transfer function H obtained for each microphone 4 when directed to Direction 2 is indicated as “Ha 1 -dir 2, Hb 1 -dir 2... Hp 1 -dir 2”.

FIG. 16 shows that a reproduction signal to be output from each reproduction speaker 8 (ap) is generated in the reproduction environment 11 when reproducing two sound sources of Rch and Lch for one virtual sound image position in this way. The structure of the reproduction signal generation apparatus 30 for this is shown.
A reproduction signal S from the audio reproduction unit 6 is input to the stereo effect processing unit 31. The stereo effect processing unit 31 performs digital effect processing such as a so-called flanger and digital delay on the input monaural audio signal to generate a stereo audio signal by Rch and Lch.
Although a configuration in which a stereo effector is incorporated is shown here, an Rch audio signal and an Lch audio signal obtained by stereo effect processing in an external effector may be directly input.

The calculation units 31a-R, 31b-R,... 31p-R have transfer functions Ha1-dir6, Hb1-dir6,. Is set. In addition, transfer functions Ha1-dir2, Hb1-dir2,... Hp1-dir2 obtained corresponding to the Lch sound source are set in the arithmetic units 31a-L, 31b-L,.
The calculation units 31a-R, 31b-R,... 31p-R perform calculation processing based on the transfer function H set for each input Rch audio signal. As a result, a reproduction signal to be output from each reproduction speaker 8 in order to reproduce the Rch sound source having the Direction 6 direction is obtained.
Similarly, the arithmetic units 31a-L, 31b-L,... 31p-L perform arithmetic processing based on the transfer function H set for each input Lch audio signal. A reproduction signal to be output from each reproduction speaker 8 in order to reproduce an Lch sound source having a directivity direction can be obtained.

The adders 32a to 32p respectively input and add the reproduction signals from the calculation unit 31 with the same subscript out of the calculation units 31a-R to 31p-R and 31a-L to 31p-L. The result is supplied to the reproduction speaker 8 with the same subscript.
In this way, the reproduction signal for reproducing the directivity direction of the Rch sound source and the reproduction signal for reproducing the directivity direction of the Lch sound source are added and output from the corresponding reproduction speaker 8, respectively. In the first closed curved surface 10 in the reproduction environment 11 in which the reproduction speakers 8 are arranged, the sound field of the measurement environment 1 can be reproduced in the form of expressing the directivity direction of the Rch sound source and the directivity direction of the Lch sound source. .

2-4. Reproduction of directivity of sound source and sound emission characteristics for each direction

Here, as a so-called acoustic musical instrument that is not an electric musical instrument such as a piano, a violin, and a drum, the directivity of each musical instrument and the sound emission characteristics for each directing direction are somewhat different. Strictly speaking, the directivity of each musical instrument (sound source) and the sound emission characteristics of each directivity direction individually affect the entire acoustic space such as a hall and shape the sound characteristics of the sound source. Therefore, when reproducing a virtual sound image as the sound source more realistically, it is effective to perform sound field reproduction in consideration of the directivity and sound emission characteristics for each directivity direction.

A method in which sound field reproduction is performed in such a manner in consideration of the directivity of the sound source and the sound emission characteristics for each directivity direction will be described with reference to FIGS.
FIGS. 17A and 17B are diagrams schematically showing how sound sources are recorded in this case. FIG. 17A shows a perspective view thereof, and FIG. 17B shows a top view thereof.
In this case, first, a sound source recording surface 35 that surrounds a required sound source 36 in a circular shape on a certain plane is defined. On the sound source recording surface 35, a plurality of recording microphones 37 (directional microphones) are arranged so as to surround the sound source 36. Also in this case, the direction of the arrow written on each microphone 37 indicates the directivity direction, and each microphone 37 is arranged in a direction facing the sound source 36 as can be seen from these arrows. By recording the sound from the sound source 36 for each of the plurality of directional microphones arranged in this way, each recorded sound can reflect the directivity of the sound source 36 and the sound emission characteristics for each direction. it can.
Here, an example is shown in which six recording microphones 37 each having a directivity of 60 ° are arranged on the sound source recording surface 35 and six directions of Direction 1 to Direction 6 are defined. As shown in the drawing, the recording microphone 37 of Direction 1 is the recording microphone 37-1, the recording microphone 37 of Direction 2 is the recording microphone 37-2, and the symbols of the recording microphone 37 are arranged by numbers below the hyphen. Indicates the direction.
Thus, by enclosing the sound source 36 from six directions, six directions are defined as directivity directions for the sound source 36 in this case. Furthermore, by recording the sound by the recording microphones 37 arranged in the six directions as described above, the recorded sound for each of the recording microphones 37 has a sound emission characteristic for each of the six directivity directions of the sound source 36. Each can be reflected.

According to this, in this case, if the sound recorded by each recording microphone 37 is emitted outward for each direction, the directivity of the sound source 36 and the sound emission characteristics for each direction are reproduced. can do.
That is, in FIG. 17, a directional speaker having the same directivity of 60 ° is disposed outwardly at the position of each recording microphone 37 disposed for each direction, and recording is performed by the corresponding recording microphone 37 from these speakers. By outputting the sound, the sound source 36 can be reproduced in a manner that reflects the directivity of the sound source 36 and the sound emission characteristics in each direction.
In this case, the recording of the sound source 36 by each recording microphone 37 is performed as close to the sound source 36 as possible (so-called “on” state) so as not to include spatial information at the recording site. Preferably, it is done.

In this way, sound is recorded from each direction so as to surround the sound source 36, and each recorded sound is output by a directional speaker arranged so as to have the same direction relationship. However, in this embodiment, the sound field as the measurement environment 1 in which such a sound source 36 is arranged is used as a reproduction environment 11 that is another environment. To try to reproduce.
Here, in order to express the directivity directions of Direction 1 to Direction 6 of the sound source 36 arranged in the measurement environment 1 in the reproduction environment 11, as described above, the transfer function H is obtained for each of these directions. That's fine. In this case, since the recorded sound from the sound source 36 exists for each direction, the recorded sound of each direction is convolved with the transfer function H of the same direction among the transfer functions H obtained for each direction. A reproduction signal for each direction can be obtained.

Here, since six directivity directions for the sound source 36 are defined, the method for obtaining the transfer function H for each direction is also in this case in the measurement environment 1 as described in FIG. What is necessary is just to measure and measure the impulse response corresponding to each measurement microphone 4 (ap) when the measurement speaker 21 is directed to each direction (Direction 1 to Direction 6).
That is, for example, if the position of the sound source 36 in the measurement environment 1 is Position 1 (Player 1), the transfer function Ha1-dir1, Hb1-dir1,... Hp1-dir1, and the transfer function Ha1- dir2, Hb1-dir2,... Hp1-dir2 are obtained. Similarly, transfer function Ha1-dir3, Hb1-dir3... Hp1-dir3 for Direction3, transfer function Ha1-dir4 for Direction4, Hb1-dir4 ... Hp1-dir4, transfer function Ha1-dir5 For Hb1-dir5... Hp1-dir5, Direction6, transfer functions Ha1-dir6, Hb1-dir6... Hp1-dir6 are obtained.

FIG. 18 shows a configuration of the reproduction signal generation device 40 corresponding to the case where sound field reproduction is performed in consideration of the directivity of the sound source and the sound emission characteristics for each directivity direction.
In this case, the audio reproduced by the audio reproducing unit 6 is an audio signal recorded for each direction. Here, the audio reproduction unit 6 that reproduces the audio recorded by the recording microphone 37-1 disposed in Direction 1 is referred to as an audio reproduction unit 6-1-1, and the recording microphone 37-2 disposed in Direction 2 The audio reproduction unit 6 that reproduces the audio recorded in is shown as an audio reproduction unit 6-1-2. Similarly, the audio reproduction unit 6 that reproduces the audio signal recorded by the recording microphones 37-3, 37-4, 37-5, and 37-6 is replaced with the audio reproduction units 6-1-3 and 6-1-4. 6-1-5, 6-1-6.
Note that the number after the first hyphen in the code attached to the sound reproduction unit in this case is a number corresponding to the case where the sound source 36 is assumed to be Player1 arranged in Position1, for example, Position2 In the case of Player 2 arranged in “2”, “2” is added. This also applies to the reference numerals of the parts described below.

Then, calculation units 41-1-1a to 41-1-1p and calculation unit 41-1- for processing the audio signals recorded for each direction based on the transfer function H obtained for each direction. 2a to 41-1-2p, computing units 41-1-3a to 41-1-3p, computing units 41-1-4a to 41-1-4p, computing units 41-1-5a to 41-1-5p, Arithmetic units 41-1-6a to 41-1-6p are provided.
The calculation units 41-1-1a to 41-1-1p include transfer functions H (Ha1-dir1 to Hp1) corresponding to the respective measurement microphones 4 (ap) when the measurement speaker 21 is directed to Direction1. -dir1) is set, and each processes the audio signal supplied from the audio reproduction unit 6-1-1 based on the set transfer function H. Thereby, first, with respect to the recorded audio of Direction 1, it is possible to obtain a reproduction signal to be output from each reproduction speaker 8 (ap) in order to express it as being emitted in the Direction 1 direction.
Also, transfer functions Ha1-dir2-Hp1-dir2 are set in the arithmetic units 41-1-2a to 41-1-2p, and the recorded audio signals of Direction 2 reproduced from the audio reproducing unit 6-1-2, respectively. By performing arithmetic processing based on the set transfer function H, it is possible to obtain a reproduction signal to be output from each reproduction speaker 8 in order to represent that the recorded audio of Direction 2 is emitted in the direction of Direction 2.
Similarly, calculation units 41-1-3a to 41-1-3p, calculation units 41-1-4a to 41-1-4p, calculation units 41-1-5a to 41-1-5p, calculation unit 41-1 The transfer functions Ha1-dir3-Hp1-dir3, transfer functions Ha1-dir4-Hp1-dir4, transfer functions Ha1-dir5-Hp1-dir5, and transfer functions Ha1-dir6-Hp1-dir6 are included in −6a to 41-1-6p. Are set, and the audio signals from the audio reproduction units 6-1-3, 6-1-4, 6-1-5, and 6-1-6 are processed based on the set transfer function H, respectively. As a result, Direction 3 recorded sound is calculated in the calculation units 41-1-3a to 41-1-3p, and Direction 4 recorded sound is calculated in the calculation units 41-1-4a to 41-1-4p, and the calculation units 41-1-5a to 41-41. Reproduction signals to be output from the respective reproduction speakers 8 (ap) are obtained for the recorded sound of Direction 5 at -1-5p and the recorded sound of Direction 6 at the calculation units 41-1-6a to 41-1-6p, respectively. .

The adders 42a, 42b,... 42p are provided in a one-to-one relationship with the reproduction speakers 8a, 8b,... 8p, and the reproduction signals from the computing units 41 with the corresponding subscripts are respectively provided. Input and addition are performed, and the result is supplied to the reproduction speaker 8 with the same subscript.
Thus, the reproduction signals to be output for each reproduction speaker 8 obtained for each direction as described above are added for each reproduction speaker 8 and output from the corresponding reproduction speaker 8. It becomes.

With such a configuration of the reproduction signal generation device 40, for example, the recorded voice in Direction 1 can be reproduced as output in the direction 1 direction in the measurement environment 1, and the recorded voice in Direction 2 is also measured in the measurement environment 1. Can be reproduced in the reproduction environment 11 as being output in the direction 2 direction. Similarly, recorded audio in other directions can also be reproduced as output in the direction of the corresponding direction.
As a result, the virtual sound image in the measurement environment 1 can be reproduced more realistically in the first closed curved surface 10 of the reproduction environment 11 in a manner that reflects the directivity of the sound source and the sound emission characteristics for each direction.

  Here, since six Direction is defined by using six recording microphones 37 having a directivity of 60 °, the transfer function H is obtained by measuring the same six Direction measurement results as in FIG. An example of use is given, but when a different number of directivity directions are defined, for example, by defining a directivity direction of 18 directions with a recording microphone 37 having directivity of 20 °, the transfer function H is defined for each defined directivity direction. Should be requested. Or, without measuring the transfer function H for all the directions actually defined, the transfer function H is obtained for a smaller number of directions, and the transfer functions H of adjacent directions among them are interpolated. The transfer function H corresponding to the directed Direction may be obtained. As a result, the number of measurements can be reduced.

Further, here, the case where the sound recording from the sound source is performed in a planar manner is illustrated, but for example, as shown in FIG.
19, Ru Tei example surrounding the sound source in a cylindrical shape.
In this case, the sound source is recorded by dividing the cylinder into upper, middle, and lower circular planes as shown in the figure, and arranging a plurality of recording microphones 51 on the respective circular planes.
Here, the upper, middle, and lower circular planes are shown as a circular plane 50-1, a circular plane 50-2, and a circular plane 50-3, respectively. Then, the recording microphone 51 arranged on the outer circumference of the upper circular plane 50-1 is 51-1, the recording microphone 51 arranged on the outer circumference of the middle circular plane 50-2 is 51-2, and the lower microphone plane 51 is arranged. The recording microphone 51 arranged on the outer periphery of the circular plane 50-3 is denoted as 51-3. In this case as well, on each circular plane, the recording microphone 51 uses a directivity of 60 °, and defines six directions from Direction 1 to Direction 6. In this case, the direction of each recording microphone 51 is represented by the number after the second hyphen. For example, the recording microphone 51 arranged in the upper stage 2 is represented as a recording microphone 51-1-2, and the recording microphone 51 arranged in the lower stage 6 is represented as 51-3-6.

In this way, when the sound source is three-dimensionally surrounded and voice recording is performed by the recording microphones 51, for example, assuming that the sound source is a human being, the sound of rubbing clothes, the sound generated by hand actions, footsteps, etc. In addition to the voice, it is possible to record a plurality of sound sources including the directivity and sound emission characteristics for each direction.
That is, the reproduction microphones having the same directivity of 60 ° are arranged outward at the geometrically equivalent arrangement position as the recording microphone 51 shown in FIG. By outputting the sound, it is possible to perceive that the person who is the recording target exists in the space surrounded by the circular planes 51-1 to 51-3.

FIG. 20 is a diagram schematically showing the state of the measurement environment 1 when the three-dimensionally enclosed sound source is reproduced in the reproduction environment 11.
First, since a three-dimensional reproduction is performed in this case, a three-dimensional space is assumed as the first closed curved surface 10. In this case, the first closed curved surface 10 is assumed to be a space with a rectangular parallelepiped, and the measurement microphone is arranged to face outward at a required position covering the first closed curved surface 10 with the rectangular parallelepiped. The three-dimensionally arranged measurement microphones are denoted as 53a to 53x as illustrated. This does not necessarily mean that the number of the measurement microphones 4 to be arranged is different from the case of the first closed curved surface 10 based on the conventional plane, and the measurement microphones 53 are a to p. You can also.
In the above description, the first closed curved surface 10 is circular. However, for the sake of convenience, the same reference numerals are given to closed solid curved surfaces.

And the measurement in this case assumes the circular planes 50-1, 50-2, 50-3 outside the first closed curved surface 10 by the above-mentioned solid, and is measured for each of the same directions at the time of recording on these circular planes. This is done by arranging a speaker 52 for use. That is, the measurement speaker 52 is arranged according to an arrangement relationship that is geometrically equivalent to the recording microphone 51 shown in FIG.
As the measurement speakers 52, directional speakers having a directivity of 60 ° are used. Also in this case, the reference numerals of the respective measurement speakers 52 indicate the numbers after the first hyphen and indicate the circular planes 50-1, 50-2, 50-3 to be arranged, and the numbers after the second hyphen. Represents one of Direction1 to Direction6.
Then, the measurement signal TSP is output from the measurement signal reproduction device 2 (not shown) for each measurement speaker 52, and at this time, the measurement microphones 53 a to 53 disposed on the first closed curved surface 10 are output. The impulse response (transfer function H) obtained corresponding to 53x is measured.
For example, here, the number of measurement microphones 53 on the first closed curved surface 10 is x and the number of measurement speakers 52 is 6 × 3 = 18, so that a total of 18 × x transfer functions H are obtained. It will be.

Although illustration is omitted, the reproduction environment 11 in this case assumes the first closed curved surface 10 of the same cube in response to the assumption that the first closed curved surface 10 is a cube in the measurement environment 1, and in this case also the measurement is performed. The reproduction speakers 8a to 8x are arranged in a geometrically equivalent arrangement relationship with the measurement microphone 53 arranged in the environment 1.
Then, the configuration of the reproduction signal generation apparatus for generating the reproduction signal to be output by these reproduction speakers 8a to 8x is basically the same as the configuration shown in FIG. In other words, in this case as well, if one circular plane 50 is viewed, the reproduction signals to be output from each reproduction speaker 8 by convolving each recorded sound with the corresponding transfer function H for the six directions from Direction1 to Direction6. Since there is no change in obtaining, it is assumed that the configuration for that purpose (that is, six sound reproduction units 6 and six sets of calculation units 41 (1a to 1p, 2a to 2p... 6a to 6p)) is further increased by two. Can be captured.
However, in this case, since the measurement microphone 53 is set to a to x, the transfer function H obtained for each measurement speaker 52 is set to Ha to Hx. In other words, as a set of calculation units 41 provided for each recorded voice, those set with Ha to Hx are provided. Similarly, since the reproduction speakers 8 include a to x, the adder 42 is provided with a to x corresponding to each reproduction speaker 8. Also in this case, each adder 42 inputs and adds the reproduction signals from the operation unit 41 with the same subscript, and supplies the result to the reproduction speaker 8 with the same subscript. Configure.

With such a configuration, it is assumed that the recording sound by each recording microphone 51 is emitted from each reproduction speaker 8 toward each direction in each circular plane 50-1, 50-2, 50-3. A reproduction signal that can be reproduced is output.
Thus, the listener in the first closed curved surface 10 in the reproduction environment 11 in which the reproduction speaker 8 is arranged has a human being as a recording target in this case in the cylindrical space as the virtual sound image position in the measurement environment 1. You can feel as you do. In other words, in the first closed curved surface 10 of the reproduction environment 11, reproduction can be performed so that a person as a recording target exists in the cylindrical space as the virtual sound image position of the measurement environment 1.

Such a method is preferably employed as a method of after-recording (after-recording) such as animation or CG. In other words, by recording a voice actor as a subject of recording in a cylindrical shape, recording including voices such as rubbing sounds and footsteps in addition to voice is performed. As the transfer function H to be used, for example, depending on the relationship between the position where the character exists in the scene and the space surrounding it, the measurement environment 1 is selected, and the virtual sound image position and the first closed position in the measurement environment 1 are selected. An arrangement relationship with the curved surface 10 is selected and measured.
Thereby, in the reproduction environment 11, it can reproduce as if the character exists in the cylindrical space as the assumed virtual sound image position in the assumed space.

In addition, although the example which encloses a sound source three-dimensionally as an example which encloses a sound source was shown here, it can also make it enclose, for example in a spherical form. That is, in this case, on the spherical surface surrounding the sound source, the recording microphone 51 is arranged for each arbitrarily defined direction, and audio recording is performed.
At this time, in the measurement environment 1, the measurement speaker 52 may be arranged on the equivalent spherical surface at a position that is geometrically equivalent to the arrangement position of each recording microphone 51, and the same impulse response may be measured. .
The reproduction signal generating apparatus has the same configuration as that described above when the recording microphone 51 and the measurement microphone 53 (that is, the measurement speaker 52 and the reproduction speaker 8) are arranged in the same number. It becomes composition.

Further, here, the case where a plurality of measurement speakers 52 are arranged and the impulse response is measured is illustrated, but the measurement of the impulse response in the measurement environment 1 is performed even if the plurality of measurement speakers 52 are not actually arranged. It is also possible to perform the measurement speaker 52 sequentially at each position on the outer periphery of each circular plane 50 toward each direction.
In this case, the number of measurements can be reduced by interpolating the transfer function H in the adjacent direction.

2-5. Add ambience data

Here, it is effective to add sounds (ambience) other than the performance sound generated at the venue, such as the cheering and applause of the audience, in order to more realistically reproduce the sense of presence in live events. Here, a method in the case of reproducing such a realistic sound field by adding such ambience will be described.

FIG. 21 schematically shows the state of the measurement environment 1 when recording such ambience.
First, in this case, the recording microphones 64a to 64p are arranged on the first closed curved surface 10 at the same number and the same position as when the impulse response is measured. As these recording microphones 64a to 64p, directional microphones are used.
Note that the recording microphone 64 and the measurement microphone 4 are given different symbols as microphones arranged at the same position on the first closed curved surface 10 of the same measurement environment 1, but the same microphone is used. It is good.

  In addition, as shown in the figure, a plurality of persons as extras are arranged at required positions outside the first closed curved surface 10 to emit predetermined sounds as ambience such as cheers and applause for each recording. The sound is recorded by the microphone 64. As a result, the recording microphones 64a to 64p can record ambience in a form including the spatial information of the measurement environment 1. Here, the ambience recorded audio signals respectively obtained by the recording microphones 64a, 64b,... 64p are referred to as ambience-a, ambience-b... Ambience-p.

  In the reproduction environment 11, the reproduction speakers 8a, 8b,..., 8p arranged on the first closed curved surface 10 output ambience-a, ambience-b,. A listener who is inside the curved surface 10 can feel that the audience is outside the first closed curved surface 10 in the measurement environment 1.

FIG. 22 shows a configuration of the reproduction signal generation device 60 in the case where such ambience is added.
This figure shows the configuration of a reproduction signal generation apparatus that is adapted for sound field reproduction in consideration of the directivity of the sound source and the sound emission characteristics for each directivity direction shown in FIG.
As shown in this figure, the ambience-a, ambience-b... Ambience-p recorded in the measurement environment 1 are reproduced and output by the reproduction units 61a, 61b. In this case, adders 62a to 62p are further provided after the adders 42a to 42p provided in a one-to-one relationship with the reproduction speakers 8a to 8p, and the reproducing units 61a, 61b,. 61p supplies ambience-a, ambience-b, ... ambience-p to these adders 62a, 62b,... 62p.

In this way, ambience-a, ambience-b... Ambience-p are added to the reproduction signal supply lines of the reproduction speakers 8a, 8b,. That is, the ambience-a, ambience-b, ... ambience-p recorded by the recording microphones 64a, 64b,... 64p in the measurement environment 1 are arranged at geometrically equivalent positions in the reproduction environment 11. The reproduction speakers 8a, 8b,..., 8p respectively output the first closed curved surface 10 inside.
Thereby, a listener who is inside the first closed curved surface 10 in the reproduction environment 11 can feel as if the audience in the measurement environment 1 is outside the first closed curved surface 10, giving a more realistic presence. Can do.

2-6. Sound field reproduction according to camera viewpoint

Up to now, the case where only the sound is reproduced in the reproduction environment 11 has been described. However, for example, as content containing a live event of a certain artist, AV (Audeo and Video) content may be considered. That is, in accordance with this, in the reproduction environment 11, live recorded audio and recorded video synchronized therewith are reproduced.
Here, as AV contents recorded with live video, there are those which are not captured from only one viewpoint (one angle) from the beginning but captured from a number of angles. In this way, when there is a video that captures a subject from a plurality of angles, it is possible to give a sense of reality corresponding to each camera angle by performing sound field reproduction corresponding to each angle.

FIG. 23 is a diagram schematically showing the method.
FIG. 23 shows a state of the measurement environment 1 as a hall where an event such as a live is actually performed, and FIG. 23A shows a state of video recording by the camera 65 when the live is actually performed. FIG. 23B schematically shows a state of measurement according to the camera angle. Here, assuming that there are a plurality of players on the stage 66, their positions are indicated by Position1 to Position4 in the figure.
For example, as shown in FIG. 23 (a), when the camera 65 captures an artist on the stage 66, the stage 66 at the same angle in the hall as the same measurement environment 1 shown in FIG. 23 (b). The impulse response at each position on the stage 66 is measured by the measurement microphones 68a to 68x arranged so as to capture the signal.
In FIG. 23B, as the first closed curved surface 10 in the measurement environment 1 in this case, a three-dimensional space is assumed as in FIG. 20, and the measurement microphones a to x are the same as those in FIG. 68 is arranged. The three-dimensional space formed by the first closed curved surface 10 is inclined with respect to the stage 66 at an angle equivalent to the camera angle in FIG. With respect to the measurement signal TSP output for each measurement speaker 67 (67-1 to 67-4) arranged for each measurement, the impulse response for each measurement microphone 68 (ax) is measured.
Thus, x × 4 transfer functions H corresponding to each measurement speaker 67 (1 to 4) to each measurement microphone 68 (ax) are obtained.

Then, at the time of reproduction in the reproduction environment 11, the reproduction audio signal is convoluted using these transfer functions H corresponding to the scene of the angle, and the reproduction signal obtained thereby is arranged for measurement in the measurement environment 1. What is necessary is just to output by the reproduction | regeneration speaker 8 (ax) arrange | positioned so that it may become geometrically equivalent positional relationship with the microphone 68 (ax).
Thus, when the viewer on the first closed curved surface 10 surrounded by the reproduction speakers 8 (a to x) in the reproduction environment 11 reproduces the video captured on the stage 66 at the angle shown in FIG. In addition, it is possible to feel the sound field when the stage 66 is viewed from the same angle.

Furthermore, sound field reproduction by such a method is performed for a plurality of assumed camera angles, so that the sound field can be perceived when the stage 66 is viewed at each angle for each camera angle. it can.
That is, in this case, the transfer function H is measured for a plurality of assumed angles by the same method as described in FIG. 23B, and the correspondence information between the camera angle and the transfer function H is obtained. Create.
At this time, for the video signal, information on the camera angle for each scene is embedded as, for example, metadata.
When reproducing the recorded video and audio, based on the angle information embedded in the video signal, the corresponding transfer function H is selected from the correspondence information of the angle and transfer function H, and this is calculated as needed. The playback audio signal is set and set for the unit, and the playback audio signal is processed and output from the reproduction speakers 8 (a to x). Thereby, for each of a plurality of camera angles in the video, the sound field can be perceived when the stage 66 is viewed at that angle.
It would be preferable to be able to reproduce such a sound field corresponding to each camera angle because it can increase entertainment.

Here, in measuring the transfer function H for each camera angle, a solid is assumed as the first closed curved surface 10, but it may be a plane instead.
In FIG. 23B, the measurement speaker that outputs the measurement signal TSP and the measurement microphone arranged on the first closed curved surface 10 are labeled with the measurement speaker 67 and the measurement microphone 68, respectively. However, these are equivalent to the measurement speaker 21 and the measurement microphone 4, respectively.

2-7. Configuration example of sound field reproduction system

The method and configuration for realizing the individual functions as the sound field reproduction system according to the present embodiment have been described above. The method for realizing these functions and the configuration of the entire reproduction system will be described below. To do.
Here, for convenience of explanation, the directivity of the sound source and the sound emission characteristics for each direction as described in FIGS. 17 to 20 are not considered, and the stereo effector described in FIGS. 15 to 16 is used. A configuration in the case of not corresponding to is described. An additional configuration in a case where these are also handled will be described later.
In addition, here, a configuration in the case of reproducing the sound field on the second closed curved surface 14 assuming a reproduction environment 20 as a home room or the like as a reproduction environment for actually reproducing the sound field will be described.
Furthermore, in this case, three players (Positions) to be presented as virtual sound image positions are Player1 to Player3, and six directions are defined as the direction of sound source in each Position.

First, as a premise, the sound field reproduction system according to the present embodiment records various audio / videos for creating AV contents including live video and its audio, and transmits the virtual sound image position. There are a production side for measuring functions and the like, and a user side for actually reproducing the sound field in the reproduction environment 11.
In this case, the production side records the recorded video / audio, transfer function, and the like on a required medium, and the user side uses a reproduction signal generation device (to be described later) to reproduce the sound field based on the information recorded on the medium. It is assumed that reproduction is performed.

FIG. 24 is a diagram showing work processes to be performed on the production side in this case and a configuration of a recording device 70 for recording information obtained by these work processes on the medium 77.
First of all, the recording device 70 in this case records the angle / Direction / transfer function correspondence information, the reproduction environment / transfer function correspondence information, the ambience data, and the line recording of each player from the information obtained in the work steps S1 to S4 in the figure. An angle / Direction / transfer function correspondence information generation unit 71, a reproduction environment / transfer function correspondence information generation unit 72, an ambience data generation unit 73, and a line recording data generation unit 74 of each player are generated. In addition, an angle information / direction instruction information adding unit 75 for adding angle information / direction instruction information to the recorded video obtained in the operation step S5 in the figure is provided.
Furthermore, each of the data obtained by the angle / direction / transfer function correspondence information generation unit 71, the reproduction environment / transfer function correspondence information generation unit 72, the ambience data generation unit 73, and the line recording data generation unit 74 of each player, A recording unit 76 is provided for recording the video data, to which the angle information / direction instruction information is added by the angle information / direction instruction information adding unit 75, on a medium 77 such as an optical disc recording medium.
The recording device 70 is realized by a personal computer, for example.

In FIG. 24, first, in work step S1, the transfer function H is measured for each Position while changing the angle / Direction. This is a work necessary to realize the control of the directivity direction of the virtual sound image described with reference to FIGS. 12 to 14 and the sound field reproduction according to the camera angle described with reference to FIG.
In this work step S1, in the measurement environment 1 such as a hall, directional measurement speakers 21 are arranged at each of the positions assumed as virtual sound image positions (in this case, three positions from Position 1 to Position 3). A predetermined number of measurement microphones 68 (measurement microphones 4) are arranged on the closed curved surface 10 in a predetermined arrangement relationship.
At this time, the measurement signal TSP is output from each measurement speaker 21 while changing the direction of the measurement speaker 21 to Direction1, Direction2,..., Direction6 for each Position. On the other hand, the measurement of the impulse response based on the detection result of the measurement signal TSP by each measurement microphone 68 corresponds to each assumed camera angle as shown in FIG. The measurement is performed while changing the angle of the first closed curved surface 10 in which the measurement microphone 68 is disposed.
As a result, a plurality of transfer functions H corresponding to the respective measurement microphones 68 are obtained for each Position and each Direction / Angle. That is, in this case, the transfer function H corresponding to each measurement microphone 68 is obtained by the number of Positions × Direction Numbers × the assumed number of angles.

Here, for convenience of explanation, the number of measurement microphones 68 (measurement microphones 4) arranged on the first closed curved surface 10 of the measurement environment 1 is not a to x shown in FIG. Suppose there is.
In this case as well, one measurement speaker 21 is arranged for each Position, but one measurement speaker 21 may be sequentially arranged in each Position to output the measurement signal TSP.

In the recording device 70, the angle / Direction / transfer function correspondence information generation unit 71 is based on the information of each transfer function H obtained by the work process S1 as shown in FIG. Generate function correspondence information.
That is, as shown in FIG. 26, for each Position assumed as a virtual sound image position, information on the transfer function H obtained for each measurement microphone 68 when each angle and each Direction is stored. Is.
Here again, the subscripts (ap) of the transfer function H indicate which of the measurement microphones 68 (ap) corresponds to. The number following this subscript represents the position. Furthermore, the following “ang” number indicates the angle, and the last “dir” number indicates the direction.

In FIG. 24, in work step S2, the transfer function E is measured while changing the number / arrangement of the measurement microphones 13 on the second closed curved surface.
In this work step S2, as shown in FIG. 7, the measurement microphones 68 (4) arranged on the first closed curved surface 10 in the measurement environment 1 on the first closed curved surface 10 in the reproduction environment 11 are arranged. The reproduction speaker 8 is arranged so as to obtain the same geometrical arrangement relationship as in FIG. Then, within the first closed curved surface 10 in the reproduction environment 11, the arrangement number and the arrangement relation assumed as the arrangement number and arrangement relationship of the reproduction speakers 18 in the actual reproduction environment (reproduction environment 20) are obtained. The impulse response of the measurement signal TSP output for each reproduction speaker 8 is measured while changing the arrangement number and arrangement relationship of the measurement microphones 13 arranged on the second closed curved surface 14. Thus, a transfer function E corresponding to each measurement microphone 13 is obtained for each pattern of the number of arrangements and the arrangement relation.
Note that, also in this operation step S2, only one measurement microphone 13 is used, and these can be sequentially arranged at assumed positions on the second closed curved surface 14 to measure the impulse response.

  The reproduction environment / transfer function correspondence information generating unit 72 uses the information on the transfer function E for each number of placement / relationships of the measurement microphones 13 obtained in the work step S2 as information on the number of placements / location relationships. Corresponding playback environment / transfer function correspondence information is generated.

Subsequently, in the work process S3, ambience recording is performed. That is, as shown in FIG. 21, a person as an extra is arranged outside the first closed curved surface 10 in the measurement environment 1, and a predetermined sound as ambience such as cheers or applause is emitted. This sound is recorded by each recording microphone 64 disposed at the same position as each measurement speaker 68 disposed on the first closed curved surface 10 in the previous work step S1.
That is, as described above, each ambience needs to be recorded at the position of each measurement microphone 68 arranged at the time of measuring the impulse response. Each of the recording microphones 64 used must be arranged at the same position as each of the measurement microphones 68 arranged at the time of measurement.
In this case, since the measurement microphones 68a to 68p are used as the measurement microphone 68 as described above, the recording microphones 64a to 64p are also used as the recording microphone 64. Note that the measurement microphone 68 and the recording microphone 64 have different reference numerals, but the same microphone can also be used.

  The ambience data generation unit 73 generates ambience data based on the recorded audio signal of each ambience recorded in the work process S3. That is, in this case, ambience data in which ambience-a to ambience-p respectively recorded by the recording microphones 64a to 64p are managed as individual data is generated.

  In work process S4, line recording of each player is performed. That is, for example, if the musical instrument played by the player is an electric musical instrument, an audio signal that is electrically output is recorded. Alternatively, in the case other than electric instruments such as vocals and drums, sound recording may be performed with a microphone arranged close to the sound source.

  The line recording data generation unit 74 of each player generates line recording data for each player based on each recording voice in the work process S4. That is, in this case, the line recording audio signals of Player 1 to Player 3 generate line recording data for each Player managed as individual data.

In work process S5, video recording is performed. That is, the video camera records an event that occurs in the measurement environment 1 such as a hall that is actually measured.
The angle information / direction instruction information adding unit 75 includes angle information for indicating which angle should be selected as the transfer function H for the recorded video data obtained in the work process S5, and the transfer function H. As a metadata, direction instruction information for instructing which direction should be selected for each player is added.
In this case, the angle information is determined by the person on the production side judging from which camera angle each scene corresponds to by actually reproducing the recorded video. The angle information / direction instruction information adding unit 75 adds angle information to the recorded video data based on each scene determined in this way and the corresponding angle information. Similarly, for the direction instruction information, if there is a scene where the player can look back when the recorded video is actually played back, the direction in which the person on the production side expresses the directing direction according to the movement of the player is displayed. Determine and decide. Then, the angle information / direction instruction information adding unit 75 adds the direction instruction information to the recorded video data so that the direction instruction information determined in this way is added to the designated scene.

The recording unit 76 includes an angle / direction / transfer function correspondence information generation unit 71, a reproduction environment / transfer function correspondence information generation unit 72, an ambience data generation unit 73, and each data obtained by the line recording data generation unit 74 of each player. The video data to which the angle information / direction instruction information is added by the angle information / direction instruction information adding unit 75 is recorded on the medium 77.
At this time, since there are a plurality of audio signals from ambience-a to ambience-p as the ambience data, they are recorded on the medium 77 so as to be recorded in separate tracks. Similarly, for the line recording data of each player, the line recording audio signal for each player is recorded on a separate track.

  For confirmation, the process numbers given to the work steps in FIG. 24 do not necessarily indicate the order in which the steps should be performed.

FIG. 25 shows the configuration of a reproduction signal generation device 80 used for sound field reproduction in the reproduction environment 20 on the user side.
Although not shown, the reproduction environment 20 in this case includes five reproduction speakers 18 arranged on the second closed curved surface 14 in the reproduction environment 20 shown in FIG. What is necessary is just to think that it was set as three of 18A, 18B, and 18C. In this case, since three virtual sound image positions, Position 1 to Position 3, are assumed, one virtual sound image position indicated as a measurement speaker 3 by a broken line in FIG. 9 is three.
Further, in this case, since the AV content recorded on the medium 77 is reproduced and the video is output, a display device for the AV content is arranged in the reproduction environment 20. Alternatively, on the outside, the second closed curved surface 14 may be disposed at a position on the virtual sound image position side when viewed from the viewer (listener). If the player is arranged on the virtual sound image position side in this way, the position of each player displayed on the screen and the direction of the virtual sound image position can be matched, and the display position of each player and the reproduced virtual sound image position can be matched. A sense of unity can be increased.
In FIG. 25, the display device is not shown.

25, the reproduction signal generation device 80 in this case includes arithmetic units 26a-1 to 26p-1, arithmetic units 26a-2 to 26p-2, and an arithmetic unit 26a similar to those described in FIG. -3 to 26p-3 are provided. That is, it is an arithmetic unit that can be changed and set for the transfer function H. Since Player 1 to Player 3 are assumed here, three sets up to Player 3 are provided out of the four sets up to Player 4 in FIG.
Then, among these calculation units 26, the outputs from the calculation units 26 with the corresponding subscripts are added, and reproduction corresponding to each reproduction speaker 8 (ap) on the first closed curved surface 10 is performed. Adders 27a to 27p for obtaining signals are provided.
Furthermore, adders 62a to 62p provided so as to have a one-to-one relationship with these adders 27a to 27p are provided. These adders 62a to 62p are the same as those shown in FIG. 22, and are provided to add audio signals as corresponding ambiences.

In the subsequent stage, calculation units 86A-a to 86A-p, calculation units 86B-a to 86B-p, and calculation units 86C-a to 86C-p are provided.
These calculation units 86 are basically arranged on the second closed curved surface 14 from each of the reproduction speakers 8 (ap) arranged on the first closed curved surface 10 in the same manner as shown in FIG. The transfer function E obtained corresponding to each of the measurement microphones 13 arranged in the above is set. Here, the reproduction speaker 18 on the second closed curved surface 14 is actually reproduced as a reproduction environment matching process. In order to correspond to the number of arrangements and the arrangement relationship, the corresponding transfer functions E are set to the respective arithmetic units 86 from the controller 83 described later.

The calculation units 86A-a to 86A-p, the calculation units 86B-a to 86B-p, and the calculation units 86C-a to 86C-p are respectively the same subscripts (ap) of the adders 62a to 62p. The output from the adder 62 marked with is input and processed based on the set transfer function E.
As a result, in the calculation units 86A-a to 86A-p, the measurement microphone 13A (for reproduction) on the second closed surface 14 from each of the reproduction speakers 8 (a to p) on the first closed surface 10 in the reproduction environment 11 is obtained. Reproduction signals (SHEA-a to SHEA-p) corresponding to the speakers 18A) are obtained, and the calculation units 86B-a to 86B-p receive from the reproduction speakers 8 (a to p) to the reproduction speakers 18B. Corresponding reproduction signals (SHEB-a to SHEB-p) are obtained. In addition, in the calculation units 86C-a to 86C-p, reproduction signals (SHEC-a to SHEC-p) corresponding to the reproduction speakers 18C to 18C are obtained.

  The adders 17A, 17B, and 17C have a one-to-one relationship with the reproduction speaker 18 (in this case, 18A, 18B, and 18C) arranged on the second closed curved surface 14 in the same manner as shown in FIG. It is provided as follows. The adder 17A inputs and adds the outputs from the arithmetic units 86A-a to 86A-p, and supplies the result to the reproduction speaker 18A. The adder 17B inputs and adds outputs from the arithmetic units 86B-a to 86B-p, supplies the result to the reproduction speaker 18B, and the adder 17C further calculates the arithmetic units 86C-a to 86C-. The outputs from each of p are input and added, and the result is supplied to the reproduction speaker 18C.

In this case, the reproduction signal generating device 80 reproduces various information recorded on the medium 77, and performs a control based on the information. The medium reading unit 81, the buffer memory 82, the controller 83, A memory unit 84 and a video reproduction system 85 are provided.
The media reading unit 81 reads various information recorded on the medium 77 loaded in the reproduction signal generating device 80 and supplies the information to the buffer memory 82. The buffer memory 82 performs buffering of read data and reading of the buffered data based on the control of the controller 83.

The controller 83 is constituted by a microcomputer and performs overall control of the reproduction signal generation device 80. The memory unit 84 comprehensively shows storage devices such as a ROM and a RAM provided in the controller 83. Although not shown, various control programs are stored in the controller 83, and the controller 83 stores each control unit based on the control program. Control is to be executed.
Here, as described above with reference to FIG. 24, for the media 77, the angle / direction / transfer function correspondence information, the playback environment / transfer function correspondence information, the ambience recording data, the line recording data of each player, and Video data in which angle information and direction instruction information are embedded is recorded.
The controller 83 causes the media reading unit 81 to read out the angle / Direction / transfer function correspondence information and the reproduction environment / transfer function correspondence information from these pieces of information, and the angle / Direction in the memory 84 as shown in FIG. Stored as transfer function correspondence information 84a and reproduction environment / transfer function correspondence information 84b.

Further, the controller 83 also causes the media reading unit 81 to read out the ambience recording data, the line recording data of each player, and the video data in which the angle information and the direction instruction information are embedded, and buffer this in the buffer memory 82. Let me.
As shown in the figure, from the buffer memory 82, ambience-a, ambience-b,... Ambience-p as ambience recording data are supplied to adders 62a, 62b,. Yes.
Similarly, with respect to the line recording data of each player, the recording audio signal of Player 1, the recording audio signal of Player 2, and the recording audio signal of Player 3 are calculated by calculation units 26a-1 to 26p-1, and calculation units 26a-2 to 26p-2. The arithmetic units 26a-3 to 26p-3 are supplied.
Further, the video data in which the angle information and the direction instruction information are embedded is supplied to the video reproduction system 85.

Here, the buffer memory 82 reads and buffers all ambience recording data recorded on the medium 77, line recording data of each player, and video data in which angle information and direction instruction information are embedded. The controller 83 may control the read operation of the buffer memory 82 so that the buffered data is continuously supplied to corresponding units.
However, in practice, since it takes a very long time to read and buffer all data from the medium 77 in this way, each data sequentially reads from the medium 77 only the necessary data amount in a time-sharing manner. The read operation of the buffer memory 82 can be controlled so as to sequentially supply this to each unit.

The video playback system 85 comprehensively shows the configuration of a playback system such as a compression / decompression decoder and an error correction processing unit for video data. The video reproduction system 85 displays video on a display device (not shown) arranged in the reproduction environment 20 by performing reproduction processing on the video data supplied from the buffer memory 82 by the compression / decompression decoder, the error correction processing unit, or the like. Is generated and supplied to the display device as the illustrated video output.
In this case, the video reproduction system 85 extracts angle information and direction instruction information added as metadata to the video data, and supplies them to the controller 83.

Based on the angle information and direction instruction information supplied from the video reproduction system 85 in this way and the angle / direction / transfer function correspondence information 84a stored in the memory unit 84 as described above, the controller 83 An angle / direction changing unit 83a for changing and setting the transfer function H to be set for each arithmetic unit 26 is provided.
As the angle / direction changing unit 83a, the transfer function H specified by the input angle information and the direction instruction information is read from the angle / direction / transfer function correspondence information 84a in the memory unit 84, and this is read out. Set for the arithmetic unit 26.
For example, if the angle information indicates “Angle 1” and the Direction instruction information indicates that Player 1 (Position 1) is Direction 1, Player 2 (Position 2) is Direction 2, and Player 3 (Position 3) indicates Direction 6. 26, from the angle / direction / transfer function correspondence information 64a shown in FIG. 26, “Ha1-ang1-dir1-Hp1-ang1-dir1” for Player1, and “Ha2-ang1-dir2-Hp2-ang1-” for Player2. For “dir2” and Player3, “Ha3-ang1-dir6 to Hp3-ang1-dir6” are read. The arithmetic units 26a-1 to 26p-1 for "Ha1-ang1-dir1-Hp1-ang1-dir1" and the arithmetic units 26a-2 to 26p for "Ha2-ang1-dir2-Hp2-ang1-dir2" -2, “Ha3-ang1-dir6 to Hp3-ang1-dir6” are set in the arithmetic units 26a-3 to 26p-3.
By such an operation of the angle / direction changing unit 83, each time the new angle / direction is instructed to each computing unit 26 as angle information and direction instruction information, the transfer function H corresponding to the angle / direction is sequentially generated. Is set. This makes it possible to reproduce the sound field according to the angle change and to control the directivity direction of the designated player.

  For confirmation, the angle / direction changing unit 83a shows the function of the controller 83 as a block, and is actually realized by software processing of the controller 83. The same applies to the reproduction environment matching processing unit 83b described below.

The controller 83 also determines the actual number of speakers 18 for reproduction based on the reproduction environment / transfer function correspondence information 84 b stored in the memory unit 84 and the arrangement pattern information 84 c stored in advance for the same memory unit 84. And a reproduction environment matching processing unit 83b for performing a reproduction environment matching process for setting the transfer function E suitable for the arrangement relationship.
Here, the arrangement pattern information 84c is information indicating a pattern of the number of arrangement / arrangement relations of the reproduction speakers 18 set in advance as corresponding to the reproduction signal generation device 80. In reproduction environment matching processing section 83 b, on the basis of such arrangement pattern information 84c pattern information of the arrangement number and arrangement relationship shown in, are stored in association with the appropriate pattern from the playing environment and the transfer function correspondence information 84b Transfer functions E (Ea-A to Ep-A, Ea-B to Ep-B, and Ea-C to Ep-C) are read out and set in the corresponding computing units 86, respectively.
As a result, a transfer function E corresponding to the number of arrangements / relationships of the reproduction speakers 18 in the actual reproduction environment 20 is set in each computing unit 86. As a result, the number of arrangements of the reproduction speakers 18 in the actual reproduction environment 20 is set. -Appropriate sound field reproduction according to the arrangement relationship can be performed.

  In addition, when the reproduction signal generation device 80 can cope with the arrangement number / arrangement relationship of a plurality of patterns, an operation unit may be provided so that the user can select a corresponding pattern from the patterns. .

Here, as described above, the configuration of the sound field reproduction system described above is a configuration in which the directivity of the sound source and the sound emission characteristics for each directivity direction are not considered, and the stereo effector is not supported. However, in the case of a configuration corresponding to these, the recording device 70 and the reproduction signal generation device 80 may be configured as follows.
As an example here, the sound field reproduction that considers the directivity of the sound source and the sound emission characteristics in each direction only for Player 1 is performed, and the line recording data of Player 2 is through a stereo effector. Is illustrated.
In this case, on the production side, in the work process S4, as for Player1, the sound is recorded by the recording microphone 37 arranged so as to surround from the six directions of Direction1 to Direction6 as shown in FIG. As for Player 2, the line recording data passed through the stereo effector is input to the recording device 70.
That is, in this case, the line recording data generation unit 74 of each player generates six recording data of Direction 1 to Direction 6 for Player 1, and generates two recording data of Lch and Rch for Player2. Is done. The recording unit 76 records these recorded data on the medium 77.

In this case, the reproduction signal generation device 80 has six pieces of recorded data of Player 1 corresponding to Direction 1 to Direction 6, and the calculation unit 26 performs an operation of inputting the recorded data corresponding to Direction 1 of Player 1 respectively. In addition, arithmetic units 26a-1-2 to 26p-1-2 for inputting recorded data corresponding to Direction 2 are added to the units 26a-1-1 to 26p-1-1. Similarly, for the recorded data of Direction3, Direction4, Direction5, and Direction6, the calculation units 26a-1-3 to 26p-1-3, the calculation units 26a-1-4 to 26p-1-4, and the calculation unit 26a-1 -5 to 26p-1-5 and arithmetic units 26a-1-6 to 26p-1-6 are added.
In this case, the angle / direction changing unit 83a includes the calculation units 26a-1-1 to 26p-1-1, the calculation units 26a-1-2 to 26p-1-2, and the calculation units 26a-1-3 to 26p-. For 1-3, computing units 26a-1-4 to 26p-1-4, computing units 26a-1-5 to 26p-1-5, and computing units 26a-1-6 to 26p-1-6, The transfer function H set only in accordance with the angle information is changed. In other words, "-dir1" is calculated for the calculation units 26a-1-1 to 26p-1-1, "-dir2" is set for the calculation units 26a-1-2 to 26p-1-2, and the calculation unit 26a-1- 3 to 26p-1-3 is "-dir3", the arithmetic units 26a-1-4 to 26p-1-4 are "-dir4", and the arithmetic units 26a-1-5 to 26p-1-5 are ""-dir5" and the calculation units 26a-1-6 to 26p-1-6 are always set to the transfer function H by "-dir6".
These computing units 26a-1-1 to 26p-1-1, computing units 26a-1-2 to 26p-1-2, computing units 26a-1-3 to 26p-1-3, computing unit 26a-1-4 ~ 26p-1-4, arithmetic units 26a-1-5 to 26p-1-5, and arithmetic units 26a-1-6 to 26p-1-6 are also given the same subscripts (ap), respectively. So as to be supplied to the adder 27.

Further, as the calculation unit 26 for inputting the recorded data of Player 2, two sets of calculation units 26 (ap) corresponding to Lch and Rch are provided. That is, arithmetic units 26a-L to 26p-L and arithmetic units 26a-R to 26p-R are provided.
The angle / direction changing unit 83a also changes the setting of the transfer function H according to only the angle information for these calculating units 26a-L to 26p-L and the calculating units 26a-R to 26p-R. For example, when Direction2 is defined as Lch and Direction6 is defined as Rch as shown in FIG. 15, "-dir2" is defined for the computation units 26a-L to 26p-L and "-dir2" is designated for the computation units 26a-R to 26p-R. The transfer function H according to “-dir6” is always set.
The outputs from the arithmetic units 26a-L to 26p-L and the arithmetic units 26a-R to 26p-R are also supplied to the adder 27 with the same subscripts (a to p). .

In the description of FIGS. 24 to 26 so far, as a premise of the sound field reproduction system of the present embodiment, the production side sells a medium 77 on which various information necessary for sound field reproduction is recorded. The sound field is reproduced on the user side based on the media 77.
However, various kinds of information necessary for reproducing the sound field can be provided not only to the user via the medium 77 but also via a network, for example.
In this case, the production side is configured to record and hold various information necessary for reproducing the sound field on a required recording medium, and to transmit the held various information to an external device via a network. An information processing apparatus is provided. On the other hand, the reproduction signal generating device 80 on the user side is configured to enable data communication via the network.
As described above, by providing various information for reproducing the sound field via the network, the various types of information can be provided in real time from the production side to the user side. It is also possible to reproduce the sound field in the measurement environment 1 in real time.

In the above description, the reproduction signal to be output from each reproduction speaker 18 is generated on the user side (reproduction signal generation device 80 side), but instead on the production side (recording device 70 side). 25, the reproduction signal to be output from each reproduction speaker 18 is recorded on the medium 77, and the user reproduces these from the medium 77 by providing the reproduction signal generation configuration shown in FIG. It can also be configured so that the sound field can be reproduced simply by reproducing the signal.
According to this, the configuration of the device provided on the user side can be simplified. However, on the other hand, on the production side, it is necessary to create and sell a plurality of types of media 77 corresponding to the number of reproduction speakers 18 and the pattern of the arrangement relationship assumed in the actual reproduction environment 20. It will be.
On the other hand, according to the sound field reproduction system of this example described above, only one type of media 77 can be created on the production side, and efficiency in this respect can be achieved.

24 and 25, the recording data and video data for each player are recorded on the medium 77, and the angle / Direction / transfer function correspondence information and the reproduction environment / transfer function correspondence information are recorded. Only the recorded data and video data for each player is recorded on the media 77, and the information necessary for reproducing the sound field, such as providing the angle / direction / transfer function correspondence information and the playback environment / transfer function correspondence information via the network. A part of them can be provided via a network.
In particular, regarding the reproduction environment / transfer function correspondence information, all information other than that set in the calculation unit 86 is unnecessary information. Therefore, for example, a required server device on the network holds the reproduction environment / transfer function correspondence information, and the user side first accesses the server device to reproduce the sound field, and the corresponding reproduction speaker. The transfer function E corresponding to the 18 arrangement number / pattern-related pattern is downloaded.
As a result, the amount of recorded information on the medium 77 can be reduced, and unnecessary information is not held in the reproduction signal generating apparatus 80, so that useless read operations can be omitted and the processing load on the controller 83 can be reduced. .

  In FIG. 25, the calculation unit 26, the adder 27, the adder 62, the calculation unit 86, and the adder 17 are illustrated as being configured by hardware, but the functions of these units are realized by software processing of the controller 83. It can also be configured to.

  25 illustrates the case where the reproduction signal generation device 80 is configured to include a reading unit for the medium 77, but the reproduction signal generation device 80 inputs each piece of information read from the medium 77 outside. Henceforth, it can also be comprised as AV amplifier which performs the same operation | movement based on each information.

  In the above description, the medium 77 is an optical disk recording medium, but may be another disk medium (a magnetic disk such as a hard disk or a magneto-optical disk). Alternatively, it may be other than a disk medium such as a recording medium using a semiconductor memory.

Further, in the above description, when reproducing the sound field on the second closed curved surface 14, after generating a reproduction signal obtained by performing an arithmetic process on the reproduced audio signal based on the transfer function H, the reproduced signal is further converted into the transfer function E. By performing arithmetic processing based on this, a reproduction signal corresponding to each arrangement position of the reproduction speaker 18 arranged on the second closed curved surface 14 is obtained, but instead, a transfer function H, a transfer function E, and A similar result can be obtained by generating a transfer function obtained by adding together and calculating the reproduction speech based on the synthesized transfer function.
Specifically, for example, a reproduction signal generation system for Player 1 shown in FIG. 25 will be described. Transfer function H-1 (ap) + transfer function EA (ap), transfer function H- 3 sets of operations that set the combined transfer function by 1 (ap) + transfer function EB (ap), transfer function H-1 (ap) + transfer function EC (ap) Each of these is supplied with the recorded audio signal of Player1. As a result, a reproduction signal (ap) to be output from the reproduction speaker 18A is obtained from the calculation unit that sets the transfer function H-1 (ap) + transfer function EA (ap). These are added by the adder 17A and supplied to the reproduction speaker 18A.
In addition, since the calculation unit that sets the transfer function H-1 (ap) + transfer function EB (ap) can obtain a reproduction signal (ap) to be output from the reproduction speaker 18B. These are added by the adder 17B and supplied to the reproduction speaker 18B.
Further, since the calculation unit that sets the transfer function H-1 (ap) + transfer function E-C (ap) can obtain a reproduction signal (ap) to be output from the reproduction speaker 18C. Are added by the adder 17C and supplied to the reproduction speaker 18C.
As for other players, by adopting the same configuration, it is possible to perform sound field reproduction similar to the configuration shown in FIG. 25 as a result.
In this case, the transfer function H for each angle / direction may be varied by setting the transfer function H to be added to the transfer function E in each of the above-described arithmetic units.
By the way, according to the above configuration, the ambience cannot be added after the reproduction signal is convoluted with the transfer function H (ap) as shown in FIG. In the case where ambience is added corresponding to the configuration in this case, separately from the above-described configuration of the reproduction signal generation system, calculation units 86A-a to 86A-p for inputting ambiences a to p, respectively, 86B-a to 86B-p and arithmetic units 86C-a to 86C-p are provided, and their outputs are similarly applied to adders 17A, 17B, and 17C with the same subscripts (A, B, and C), respectively. What is necessary is just to comprise so that it may input and may add. As a result, the transfer function E from the first closed curved surface 10 to the second closed curved surface 14 is obtained for each reproduction signal to be output from each reproduction speaker 18A, B, C generated by the reproduction signal generation system described above. The convolved ambience can be added, and the ambience can be reproduced similarly to the configuration shown in FIG.
Even when the reproduced audio signal is convoluted with the combined transfer function obtained by adding the transfer function H and the transfer function E in this manner, the reproduced audio signal is reproduced based on the transfer function H as a result. Since the same effect as that obtained when the signal is arithmetically processed by the transfer function E can be obtained, the reproduction signal obtained by arithmetically processing the reproduced audio signal based on the transfer function H can be obtained even when the reproduction signal is arithmetically processed by such a composite transfer function. It is assumed that the calculation is performed with the transfer function E.

Although the embodiment of the present invention has been described above, the present invention should not be limited to this.
For example, in the embodiment, the case where the present invention is applied to sound field reproduction in the case of reproducing sound in a film live venue, a home room, or the like has been exemplified, but also in the case of reproducing sound in a car audio system, for example. Can be applied. Alternatively, the present invention can also be suitably applied to virtual reality industrial equipment such as an amusement or game that gives a sense of reality or immersion.

  1 Measurement environment, 2 Measurement signal reproduction unit, 3, 21, 52, 67 Measurement speaker, 4, 13, 53, 68 Measurement microphone, 5, 15, 19, 25, 30, 40, 60, 80 Reproduction signal Generator, 6 sound reproducing unit, 7, 16, 26, 31, 41, 86 computing unit, 8, 18 reproduction speaker, 9, 17, 27, 32, 42, 62 adder, 10 first closed curved surface, 11 , 20 reproduction environment, 14 second closed curved surface, 28 controller, 29 ROM, 29a Direction correspondence information, 31 stereo effect processing unit, 35 sound source recording surface, 36 sound source, 37, 51, 64 recording microphone, 50 circular plane, 61a ~ 61p playback unit, 65 camera, 66 stage, 70 recording device, 71 angle / Direction / transfer function correspondence information generation unit, 72 playback environment / transfer function correspondence information generation unit, 73 Ambience data generation unit, 74 Line recording data generation unit of each player, 75 Angle information / Direction instruction information addition unit, 76 Recording unit, 77 Media, 81 Media reading unit, 82 Buffer memory, 83 Controller, 83a Angle / Direction change Part, 83b reproduction environment matching processing part, 84 memory part, 84a angle / Direction / transfer function correspondence information, 84b reproduction environment / transfer function correspondence information, 84c arrangement pattern information, 85 video reproduction system

Claims (14)

Recording process of recording sound from the sound source with a directional microphone from multiple directions so as to surround the required sound source,
A directional speaker arranged at a virtual sound image position outside the first closed curved surface is used to direct sound in a plurality of directions opposite to the plurality of directions in which the sound from the sound source is recorded in the recording step. The first pronunciation process to pronounce;
A first measurement step of measuring sounds in the plurality of directions sounded in the first sound production step at a plurality of positions on the first closed curved surface;
Based on the sound measured by the first measurement step, a first transfer function group corresponding to each of the plurality of positions on the first closed curved surface from the virtual sound image position is determined for each of the plurality of directions. A first transfer function generation step to generate,
By inputting the audio signal recorded in the recording step and performing arithmetic processing based on the first transfer function group in the corresponding direction to the input audio signal, the above on the first closed curved surface Obtaining reproduction audio signals for the plurality of directions as reproduction audio signals corresponding to each of a plurality of positions, and adding the reproduction audio signals for each of the plurality of positions on the first closed curved surface. And a first reproduction audio signal generating step for obtaining a first reproduction audio signal corresponding to each of the plurality of positions on the first closed curved surface. In the recording process,
The audio signal processing method according to claim 1 , wherein audio is recorded so that the sound source is surrounded by the directional microphone in a plane. In the recording process,
The audio signal processing method according to claim 1 , wherein audio is recorded so that the sound source is three-dimensionally surrounded by the directional microphone. Further, a first audio signal output step of outputting each of the first reproduction audio signals from a reproduction speaker arranged in a positional relationship geometrically equivalent to the plurality of positions on the first closed curved surface. The audio signal processing method according to claim 1. Further, a second sound generation step of sound generation from a sound source arranged in a geometrically equivalent positional relationship with the plurality of positions on the first closed curved surface,
A second measuring step for measuring the sound produced by the second sounding step at a plurality of positions on the second closed curved surface formed inside the first closed curved surface;
Based on the sound measured by the second measurement step, a second transfer function group corresponding to each of the plurality of positions on the second closed curved surface is generated from each of the plurality of sound sources. A second transfer function generation step, and
By applying arithmetic processing based on the second transfer function group to the first reproduction audio signal obtained by the first reproduction audio signal generation step, The audio signal processing method according to claim 1, further comprising a second reproduction audio signal generation step for obtaining a second reproduction audio signal corresponding to each of the plurality of positions. A second audio signal output step of outputting each of the second reproduction audio signals from a reproduction speaker arranged in a geometrically equivalent positional relationship with the plurality of positions on the second closed curved surface; The audio signal processing method according to claim 5 . Wherein in the first pronunciation step or said second sound process, audio signal processing method according to sound based time stretched pulse Could claim 1 or claim 5 or as the speech. Furthermore, it has an ambience recording step of recording sound generated outside the first closed curved surface by a directional microphone at a plurality of positions on the first closed curved surface,
In the first reproduction audio signal generation step,
The ambience recording step is performed for each of the first reproduction audio signals after performing the arithmetic processing based on the first transfer function group on the input audio signal to obtain the first reproduction audio signal. The audio signal processing method according to claim 1, wherein the corresponding audio signals among the audio signals for each of the plurality of positions recorded by the step are added. In the first measurement step,
While changing the angle of the first closed surface with respect to the directional speaker, measure the sound produced in the first sound production step at a plurality of positions on the first closed surface,
In the first transfer function generation step,
For each sound measured at each angle by the first measurement step, the first transfer function group is generated,
In the first reproduction audio signal generation step,
The audio signal processing method according to claim 1, wherein the input audio signal is subjected to arithmetic processing based on the first transfer function group obtained for each angle to obtain the first reproduction audio signal. In the first reproduction audio signal generation step,
The audio signal processing method according to claim 9 , wherein the one first transfer function group is selected according to viewpoint information of a video displayed in synchronization with the input audio signal. Sound field reproduction comprising a recording device for recording information on a recording medium, and an audio signal processing device for generating a reproduction audio signal for reproducing the sound field based on the information recorded on the recording medium A system,
The recording device comprises:
Recorded audio signal recorded in the recording process of recording the sound from the sound source with a directional microphone from multiple directions so as to surround the required sound source, and the directional pattern arranged at the virtual sound image position outside the first closed curved surface A plurality of voices produced by sounding a directional speaker in a plurality of directions opposite to the plurality of directions in which the sound from the sound source was recorded in the recording step at a plurality of positions on the first closed curved surface with the obtained based on the result of measurement for each direction, a recording means for recording a first transfer function groups for each of the plurality of directions relative to said recording medium,
The audio signal processing device includes:
Input means for inputting the first transfer function group and the recorded audio signal for each of the plurality of directions recorded on the recording medium;
In facilities Succoth arithmetic processing based on the first transfer function group in a direction corresponding respectively to the recording audio signal, as reproduced audio signal corresponding to each of the plurality of positions on the first closed surface Each of the plurality of reproduction sound signals for the plurality of directions is obtained, and the reproduction sound signals are added for each of the plurality of positions on the first closed curved surface, thereby A sound field reproduction system comprising: first reproduction audio signal generation means for obtaining a first reproduction audio signal corresponding to each position. The recording means in the recording device comprises:
A plurality of sounds on the second closed surface formed inside the first closed surface, with the sound produced from the sound source arranged in a geometrically equivalent positional relationship with the plurality of positions on the first closed surface Further recorded a second transfer function group corresponding to each of the plurality of positions on the second closed surface obtained from each of the plurality of sound sources, obtained based on the result measured at the position of
The audio signal processing device further includes:
By applying arithmetic processing based on the second transfer function group to the first reproduction audio signal obtained by the first reproduction audio signal generation means, The sound field reproduction system according to claim 11 , further comprising second reproduction sound signal generation means for obtaining a second reproduction sound signal corresponding to each of the plurality of positions. The recording means in the recording device comprises:
Further recording an ambience audio signal that records audio generated outside the first closed curved surface by a directional microphone at a plurality of positions on the first closed curved surface with respect to the recording medium,
The first reproduction audio signal generation means in the audio signal processing device is:
After performing the arithmetic processing based on the first transfer function group on the recorded audio signal to obtain the first reproduction audio signal, the ambience audio signal is obtained for each of the first reproduction audio signals. The sound field reproduction system according to claim 11 , wherein the corresponding sound signals are added. The recording means in the recording device comprises:
As the first transfer function group, the sound produced from the directional speaker is measured at a plurality of positions on the first closed curved surface while changing the angle of the first closed curved surface with respect to the directional speaker. The first transfer function group for each angle obtained based on the result was recorded,
The first reproduction audio signal generation means in the audio signal processing device is:
The sound field reproduction system according to claim 11 , wherein the recorded audio signal is subjected to arithmetic processing based on the first transfer function group for each angle to obtain the first reproduction audio signal.

Images