JP5611119B2 - Acoustic simulator, acoustic consulting apparatus, and processing method thereof - Google Patents

Description

  The present invention relates to a simulation technique for supporting appropriate installation of an audio device according to a use environment and a consulting technique using the simulation result.

  In recent years, video conferencing systems have become popular. If a video conference system is used, two-way communication using voice and video can be realized even between remote locations.

  In this system, sound is recorded through a microphone, transmitted to a remote side, and reproduced through a speaker. However, there is a possibility that various noises are mixed in the voice recorded by the microphone, and the mixing of the noise causes deterioration of the quality of the voice reproduced from the speaker.

  In general, ambient noise and acoustic echo including wind noise generated by air conditioners and projectors are likely to be a problem. Acoustic echo is generated when sound reproduced from a speaker installed in the same room as the microphone is mixed into the microphone.

  However, if the sound recorded by the microphone is transmitted far away without any processing (acoustic echoes are also sent), the far-side speaker returns with a delayed voice. It feels like coming. In this case, the ease of conversation is greatly impaired. For this reason, an acoustic signal processing system having a function of removing ambient noise and acoustic echo has been conventionally proposed (see, for example, Non-Patent Document 1).

  In addition to ambient noise and acoustic echo, the reverberation component of the speaker is also a kind of noise. For this reason, suppression of these reverberation components is also required. However, this reverberation component has a high correlation with the voice of the speaker. For this reason, when removing the reverberation component, the voice of the speaker may be erroneously suppressed. In particular, it is difficult to remove the initial reflected sound. However, in recent years, a technology has been developed to remove a rear reverberation component that delays arrival at the microphone by about several hundred ms from the sound directly transmitted from the speaker's mouth to the microphone (for example, see Non-Patent Document 2).

  To solve these problems, tuning and optimization are required before using the video conference system. For these operations, the acoustic characteristics recorded at the actual site are often used. For example, considering the acoustic environment as a kind of system, the sound between the input sound that entered the acoustic environment (the sound of the moment when it comes out of the speaker's mouth or speaker) and the output sound that comes out of the system (the sound that entered the microphone) The relationship is often measured as an impulse response and used for tuning and optimization. For example, TSP (Time Stretched Pulse) is used for measuring the impulse response.

  All of the tuning and optimization technologies that have been proposed so far are mainly changes in parameters used in acoustic signal processing. The recorded impulse response is used to simulate and evaluate speech speech in the corresponding environment. Used for.

  Incidentally, in the design of concert halls and other acoustic designs when building or building, a system has been developed that can simulate the acoustic environment from CAD data of the target building and know in advance how to hear the sound (for example, , See Patent Document 1).

Japanese Patent No. 2846162

Masato Togami et al., “Remote conference system with table-top sudden sound removal function using vertical microphone array”, IEICE Transactions D, Vol. J93-D No.10 pp. 2069-2084, 2010/10 . K. Kinoshita, M. Delcroix, T. Nakatani and M. Miyoshi, “Suppression of late reverberation effect on speech signal using long-term multiple-step linear prediction”, IEEE Transactions on Audio, Speech and Language processing, 17 (4) , pp.534-545, 2009

  However, the above-described acoustic signal processing technique is easily affected by the indoor environment (acoustic conditions). Specifically, it is easily affected by the reverberation time, the type and location of noise, the positional relationship between the speaker and the microphone, the reproduction volume of the speaker, and the like.

  For this reason, tuning and optimization of video conferencing systems, etc. require not only tuning parameters used in acoustic signal processing, but also optimizing the positional relationship between microphones and speakers and the number of microphones according to the operating environment. There is.

  In addition, existing acoustic simulators require that on-site CAD data be acquired in advance, and if CAD data does not exist, the acoustic characteristics could not be simulated in the first place.

  The present inventor, who has intensively studied these technical problems, is able to optimize the positional relationship between the microphone and the speaker according to the use environment even in an environment where CAD data does not exist or cannot be used, and the processing result. Invented acoustic consulting technology that uses.

  The acoustic simulator according to the present invention is assumed to be acoustic data actually recorded in a space for constructing an acoustic system, position information based on actual measurements of microphones and speakers used at the time of recording the acoustic data, and after system construction. Based on the position information of the sound source, the microphone, and the speaker, the acoustic characteristics at the installation position of the microphone are estimated.

  The acoustic consulting apparatus according to the present invention evaluates whether or not the acoustic characteristics estimated by the acoustic simulator satisfy a predetermined performance, and presents the evaluation result to the user.

ADVANTAGE OF THE INVENTION According to this invention, even when the CAD data regarding the space which construct | assembles an acoustic system does not exist, or when it cannot utilize, it can support the construction of the acoustic environment suitable for space.
Problems, configurations, and effects other than those described above will be clarified by the following description of embodiments.

The figure explaining the installation environment of the video conference system which concerns on an example. The figure which shows the hardware constitutions of the acoustic consulting apparatus (acoustic simulator) which concerns on an example. The flowchart which shows the process sequence performed with the acoustic simulator which concerns on an example. The figure which shows the example of a microphone information table. The figure which shows the example of a speaker information table. The figure which shows the example of a table regarding a virtual speaker position. The figure which shows the structural example of an acoustic characteristic measurement part. The figure which shows the structural example of an ambient noise measurement part. The figure which shows the structural example of an acoustic simulation part. The figure explaining the direct sound component and reverberation component of an impulse response. The flowchart which shows the process sequence performed with the acoustic consulting apparatus which concerns on an example. The figure which shows the example of a desired performance table. The figure which shows the structural example of a performance evaluation part. The flowchart explaining the process sequence performed when the number and arrangement | positioning of a microphone and arrangement | positioning of a speaker are optimized automatically.

  Hereinafter, embodiments of the present invention will be described with reference to the drawings. The embodiment of the present invention is not limited to the embodiments described later, and various modifications are possible within the scope of the technical idea.

  Hereinafter, a mechanism of an audio consulting apparatus suitable for use in constructing a video conference system in a space where CAD data does not exist will be described. FIG. 1 shows an installation environment example of a video conference system to which an acoustic consulting apparatus is applied. Note that the “video conference system” in this specification is used to include a video conference system and a web conference system.

(Components and arrangement of the video conference system)
In this embodiment, the video conference system is assumed to be a general-purpose video conference system including a microphone and a speaker. However, it may be a video conference system optimized for a specific application.

  The video conference system installation environment 101 is not particularly limited as long as it is a space (environment) for constructing a video conference system. Here, a conference room is assumed. In the case of this embodiment, it is assumed that a conference microphone 104 is installed on the desk 102 arranged in the conference room. In addition, the conference speaker 103 is arranged in the same conference room. The conference speaker 103 and the conference microphone 104 may be always connected to a general-purpose video conference system.

  The video conference system according to FIG. 1 assumes a case where there are two speakers. In FIG. 1, assumed speaker positions are represented by assumed speaker positions 105-1 and 105-2. However, the number of speakers may be one or more than three in terms of system. Further, the conference speaker 103 and the conference microphone 104 are not limited to one system, and may be a plurality of units.

  FIG. 1 shows a conference speaker 103, a conference microphone 104, and assumed speaker positions 105-1 and 105-2 that give acoustic conditions when using the video conference system, and four units used for actual measurement of acoustic characteristics. The measurement microphone array 106 and two measurement speaker arrays 107 are depicted.

  In the case of this embodiment, the measurement microphone array 106 and the measurement speaker array 107 are arranged by the measurement user when measuring the acoustic characteristics of the video conference system installation environment 101. In the case of FIG. 1, the measurement microphone arrays 106 are arranged at the four corners of the desk 102. The measurement speaker array 107 is arranged behind the assumed speaker positions 105-1 and 105-2.

  Here, the measurement speaker array 107 is used to emit a reference sound when measuring the acoustic characteristics. In the case of this embodiment, the measurement speaker array 107 is an aggregate of a plurality of speakers, but may be configured by a single speaker.

  In the case of FIG. 1, a plurality of measurement microphone arrays 106 and a plurality of measurement speaker arrays 107 are arranged, but only one of them may be arranged. The measurement microphone 106 and the measurement speaker array 107 are removed from the conference room after the acoustic characteristics are measured by the acoustic consulting apparatus and the optimum conditions are output. However, a part may be shared with the conference speaker 103 and the conference microphone 104.

(Hardware configuration of acoustic consulting equipment)
FIG. 2 shows a hardware configuration of the acoustic consulting apparatus according to the embodiment. The acoustic simulation apparatus is realized as a part of the function of the acoustic consulting apparatus. Therefore, the hardware configuration of the acoustic consulting apparatus is the same as that of the acoustic simulation apparatus. Below, it demonstrates as a hardware constitutions of an acoustic consulting apparatus.

  The acoustic consulting apparatus according to the embodiment is configured by connecting the measurement microphone array 106 and the measurement speaker array 107 to a computer.

  The acoustic signal captured by the measurement microphone array 106 is converted from an analog signal to a digital signal by a multi-channel AD (Analog to Digital) converter 202. The converted digital signal is given to the central processing unit 203.

  The central processing unit 203 executes various programs. In the case of this embodiment, the central processing unit 203 executes processing for simulating acoustic characteristics and processing for evaluating the processing results when the conference microphone 104 and the conference speaker 103 are arranged at an arbitrary position in the conference room. To do. In this specification, a program that realizes the processing function is referred to as an “acoustic signal processing program”. The acoustic signal processing program is stored in the nonvolatile memory 204 and is read out to the central processing unit 203 as necessary. Incidentally, a work memory for executing the program is secured on the volatile memory 205.

  As described above, the acoustic consulting apparatus according to the embodiment actually measures the acoustic characteristics of the video conference system installation environment 101 with the measurement microphone array 106 and the measurement speaker array 107 and uses them as basic data for acoustic simulation. The acoustic characteristics here are impulse response characteristics and ambient noise.

  When measuring the acoustic characteristics, the central processing unit 203 transmits a reference signal for impulse response measurement as a digital signal to a multi-channel DA (Digital to Analog) converter 206. The reference signal is converted into an analog signal by the multi-channel DA converter 206 and output to the measurement speaker array 107. The measurement speaker array 107 radiates sound corresponding to the input reference signal to the video conference system setting environment 101.

  The central processing unit 203 is provided with a mouse 208 and a keyboard 209 as user interfaces. The user inputs information to the central processing unit 203 using these interfaces. The result of the acoustic simulation is displayed on the display 210, and the user can visually confirm the simulation result and the evaluation result.

(Processing as an acoustic simulator)
First, the basic function (acoustic simulation function) of the acoustic consulting apparatus according to this embodiment will be described. Here, the acoustic simulation function is a function for simulating acoustic characteristics recorded when the conference speaker 103 and the conference microphone 104 are virtually set at arbitrary positions in the video conference system installation environment 101.

  However, in this embodiment, it is assumed that there is no CAD data for the video conference system installation environment 101. For this reason, the acoustic consulting apparatus operating as an acoustic simulator measures the acoustic characteristics of the video conference system installation environment 101 using the measurement microphone array 106 and the measurement speaker array 107, and uses the measurement result at the virtual position. Calculate acoustic characteristics. Hereinafter, the provider of the acoustic simulation function is referred to as an acoustic simulator.

  FIG. 3 shows an outline of the processing procedure of the acoustic simulator. In processing 301 to processing 303, the acoustic simulator performs registration processing of the position information of the conference microphone 104 and the conference speaker 103 installed in the video conference system installation environment 101 and the assumed speaker positions 105-1 and 105-2. Run. Note that the execution order of the processing 301 to the processing 303 is an example, and may be executed in any order.

  In the process 301, the acoustic simulator executes a process for registering information related to the conference microphone 104. The information here is input through the mouse 208, the keyboard 209, and other input devices so that the central processing unit 203 can process the information. Since CAD data does not exist, registration (setting) work is performed manually. The same applies to registration of position information of other audio equipment.

  The information of the conference microphone 104 includes the installation location in the video conference system installation environment 101, the microphone directivity, the microphone orientation, and the like.

  FIG. 4 shows an example of registration of information of the conference microphone 104. As will be described later, the registration items shown in FIG. 4 are common to the information of the measurement microphone. However, the conference information and the measurement information are managed in separate tables.

  Each row in FIG. 4 corresponds to information on each microphone. In the case of FIG. 4, the use of three microphones is assumed. Each row is given a microphone ID that uniquely identifies the microphone. Each row stores the three-dimensional position (x, y, z) and orientation of the microphone. Any unit system can be used as long as the units are consistent. It is assumed that the coordinate system is absolute, and the same coordinate system is used for each video conference system installation environment 101.

  If the conference microphone 104 is not set, a coordinate value to be installed may be input. For example, an actual measurement value is input as a coordinate value for designating the location of the conference microphone 104. When the information regarding the installation position can be referred to, the information may be manually input. In the case of this embodiment, the conference microphone 104 is already installed and is used as a reference point when actually measuring the installation positions of the measurement microphone array 106 and the measurement speaker array 107.

  In addition, information on the directivity of the microphone is given to each row. From the information on the directivity, the sound pressure level for each azimuth angle with respect to the front is uniquely determined. The directivity characteristic can be generally known from a microphone catalog or the like.

  In process 302, the acoustic simulator executes a process for registering information related to the conference speaker 103. The information here is also input through the mouse 208, the keyboard 209, and other input devices so that the central processing unit 203 can perform processing.

  The information of the conference speaker 103 includes the installation location in the video conference system installation environment 101, the radiation characteristics of the speaker, the direction of the speaker, and the like.

  FIG. 5 shows a registration example of information of the conference speaker 103. As will be described later, the registration items shown in FIG. 5 can also be used for registering information of the measurement speaker array. However, the conference information and the measurement information are managed in separate tables.

  Each row in FIG. 5 corresponds to information of each speaker. In the case of FIG. 5, the use of three speakers is assumed. Each row is given a speaker ID that uniquely identifies the speaker. Each row stores a three-dimensional position (x, y, z) and orientation of the speaker. The unit system is arbitrary, but the same coordinate system as the microphone is used.

  If the conference speaker 103 is not installed, a coordinate value to be installed may be input. For example, an actual measurement value is input as the coordinate value for designating the location of the conference speaker 103. When the information regarding the installation position can be referred to, the information may be manually input. When the conference speaker 103 is already installed, the set position may be used as a reference point for actually measuring the installation positions of the measurement microphone array 106 and the measurement speaker array 107.

  In addition, information on the radiation characteristics of the speaker is given to each row. From the information on the radiation characteristics, the sound pressure level for each azimuth angle with respect to the front is uniquely determined. The radiation characteristics can be generally known from a catalog of speakers or the like.

  In process 303, the acoustic simulator executes a process of registering information related to the assumed speaker position. The assumed speaker position is information that designates a range that is assumed as the seating position of the participant of the video conference. The information here is also input through the mouse 208, the keyboard 209, and other input devices so that the central processing unit 203 can perform processing.

  FIG. 6 shows a registration example of the assumed speaker position. Each line in FIG. 6 corresponds to information on the assumed speaker position. If there are a plurality of assumed speaker positions, a plurality of speaker positions are set. FIG. 6 shows a case where there are three assumed speaker positions. Each row is given an assumed speaker position ID that uniquely identifies the assumed speaker position. Each row stores a three-dimensional position (x, y, z) that gives the center position of the assumed speaker position and a radius R that gives a range for the center position. The coordinate system is absolute and uses the same coordinate system as the microphone.

  Next, in processing 304 to processing 305, the acoustic simulator sets position information and the like of a measuring device (acoustic device) for measuring acoustic characteristics of the video conference system installation environment 101. The execution order of the process 304 and the process 305 is an example, and any of them may be executed first.

  In process 304, the acoustic simulator executes a process for registering information regarding the measurement microphone 106. The information here is input through the mouse 208, the keyboard 209, and other input devices so that the central processing unit 203 can process the information.

  The information of the measurement microphone 106 includes an installation location in the video conference system installation environment 101, a microphone directivity, a microphone orientation, and the like. As described above, the information related to the measurement microphone 106 is recorded in a table different from the information related to the conference microphone 104. Note that the coordinate system is absolute and uses the same coordinate system as the microphone. Here, the coordinate value of the measurement microphone 106 may be input as relative position information with respect to a reference point (for example, the conference microphone 104) set in the video conference system installation environment 101. When this input method is adopted, the central processing unit 203 executes processing for conversion to absolute coordinates.

  In process 305, the acoustic simulator executes a process for registering information related to the measurement speaker array 107. The information here is input through the mouse 208, the keyboard 209, and other input devices so that the central processing unit 203 can process the information.

  The information of the measurement speaker array 107 includes the installation location in the video conference system installation environment 101, the radiation characteristics of the speaker, the direction of the speaker, and the like. As described above, the information related to the measurement speaker array 107 is recorded in a table different from the information related to the conference speaker 103. Note that the coordinate system is absolute and uses the same coordinate system as the microphone. Similarly, the coordinate values of the measurement speaker array 107 may be input as relative position information with respect to a reference point (for example, the conference microphone 104) set in the video conference system installation environment 101. When this input method is adopted, the central processing unit 203 executes processing for conversion to absolute coordinates.

  In the process 306, the acoustic simulator uses the measurement microphone array 106 and the measurement speaker array 107 installed in the video conference system installation environment 101 to measure acoustic characteristics specific to the video conference system setting environment 101. In process 306, the acoustic simulator performs measurement of transfer characteristics (impulse response) and measurement of ambient noise between the measurement speaker array 107 and the measurement microphone array 106.

  FIG. 7 shows a functional block configuration of a program that realizes a processing function corresponding to the processing 306. In the following description, the program is referred to as an acoustic characteristic measurement unit 701. The acoustic characteristic measurement unit 701 includes an impulse response measurement unit 702 and an ambient noise measurement unit 703.

  The impulse response measurement unit 702 measures the impulse response in the video conference system setting environment 101 using, for example, the TSP method (see, for example, Patent Document 1). In addition, sound including all frequency components such as white noise is radiated from the measurement speaker array 107 and recorded by the measurement microphone array 106, and the correlation coefficient between the signal recorded by the microphone and the original signal of the radiated sound is calculated. The impulse response may be measured by examining it.

  As shown in FIG. 7, when measuring the impulse response, a multi-channel AD converter 202 connected to the measurement microphone array 106 and a multi-channel DA converter 206 connected to the measurement speaker array 107 are used.

  The multi-channel DA converter 206 receives a white signal or TSP signal (acoustic signal S3) used for impulse response measurement from the impulse response measuring unit 702, and converts the acoustic signal S3 from a digital signal to an analog signal. The multi-channel AD converter 202 is synchronously controlled with the multi-channel DA converter 206, and converts an audio signal during impulse response measurement from an analog signal to a digital signal (acoustic signals S1, S2). The converted digital signal is given to the impulse response measurement unit 702 and the ambient noise measurement unit 703.

  The impulse response measurement unit 702 applies a correlation coefficient estimation process or a TSP (Time Stretched Pulse) inverse transform process to a given signal to obtain an impulse response S4. Since these processes are already known, detailed description thereof is omitted.

  On the other hand, the ambient noise measurement unit 703 measures the ambient noise in the video conference system installation environment 101 from a given signal. Recording of ambient noise controls the devices in the video conference system installation environment 101 so that noise as close as possible to noise during actual video conference occurs. For example, when an air conditioner or a projector is provided in the video conference system installation environment 101, ambient noise is recorded in a state where these devices are operated. Of course, no sound is output from the measurement speaker array 107 when ambient noise is recorded. Similarly, when recording ambient noise, be careful not to accidentally record speaker sound. However, in the case of assuming that the ambient noise is a paper rubbing sound or a table tapping sound, the recording environment may be devised so that these sounds are generated during recording.

  FIG. 8 shows a detailed block configuration of the ambient noise measurement unit 703. The ambient noise measurement unit 703 includes a sound source separation unit 802 that separates the acoustic signal S2 collected by the measurement microphone array 106 into signals S11, S12,..., S1N for each sound source, and the volume of each sound source from the signal for each sound source. And sound source localization units 803-1, 803-2,..., 803-N for estimating a spatial location. Here, the sound source localization units 803-1, 803-2,..., 803-N output sound volume and sound source position information S5 (S21, S22,... S2N) of each sound source.

  The sound source separation unit 802 uses independent component analysis, minimum dispersion beamformer, non-negative matrix decomposition and other general sound source separation processing techniques, and uses a plurality of channels of microphone input signals as signals S11, S12,. To separate.

  The sound source localization units 803-1, 803-2,..., 803-N obtain the position of each sound source using an SRP-PHAT (Steered Response Power-Phase Transform) method based on the phase difference. In addition, when the measurement microphone arrays 106 are arranged in a distributed manner, a method of estimating the sound source position from the amplitude ratio between the microphones may be used.

  In the process 307, the acoustic simulator executes a virtual information registration process regarding the conference microphone 104 and the conference speaker 103 for the acoustic simulation. Based on the information registered in this process, the acoustic simulator executes an acoustic simulation. The user can register virtual values for the information registered for the conference microphone 104 and the conference speaker 103. That is, virtual values relating to the installation position, orientation, performance, and the like can be registered. For example, the position and orientation information registered in the processes 301 and 302 may be used as they are, and only the directivity characteristics of the conference microphone 104 may be virtually changed.

  In the case of this embodiment, the user performs registration (setting) of these pieces of information using, for example, a GUI (Graphical User Interface). Information registration may be performed by directly inputting a numerical value or the like, or a method of selecting from a predefined list may be employed.

  In process 308, the acoustic simulator executes an acoustic simulation based on the acoustic characteristics actually measured for the video conference system setting environment 101 and the information about the conference microphone 104 and the conference speaker 103 that are virtually set, and the simulation is performed. The result is output and the process is terminated.

  FIG. 9 shows a functional block configuration of a program that realizes a processing function corresponding to the processing 308. In the following description, the program is referred to as an acoustic simulation unit 901. The acoustic simulation unit 901 has a function of estimating the impulse response of the utterance of the conference participant, a function of estimating the residual echo amount collected by the conference speaker 104, and noise at the microphone position virtually set in the simulation. It has a function to simulate.

  The acoustic simulation unit 901 includes a direct sound / reverberation sound division unit 902, an assumed speaker position impulse response estimation unit 903, a residual echo estimation unit 904, and an assumed microphone position noise simulation unit 905.

  The direct sound / reverberation division unit 902 divides the impulse response S4 measured by the impulse response measurement unit 702 into a direct sound component and a reverberation sound component. FIG. 10 shows an example of the impulse response S4. The upper part of the figure shows the waveform of the impulse response obtained when the distance between the measurement speaker 107 and the measurement microphone 106 is 1 m, and the lower part of the figure shows the waveform of the impulse response obtained when the distance is 3 m. It is. Here, the horizontal axis is time, and the vertical axis is signal intensity.

  As shown in the figure surrounded by a broken line, the waveform appearing near the head of the impulse response corresponds to the direct sound component, and the waveform appearing thereafter corresponds to the reverberation component. As can be seen by comparing the two waveforms, the direct sound component is clearly affected by the distance. It can be seen that the peak value of the direct sound component is larger at 1 m than at 3 m. On the other hand, it can be seen that the reverberation component does not change greatly.

  In the case of this example, the ratio of direct sound at a distance of 1 m and 3 m was about 9.5 dB, and the ratio of reverberant sound was about 2 dB. When the distance changes 3 times from 1 m to 3 m, the volume is reduced by about 9.5 dB. Therefore, it can be considered that the volume of the direct sound component changes in inverse proportion to the square of the distance. On the other hand, the volume of reverberation is considered to be determined almost independently of changes in distance.

  First, the direct sound / reverberation division unit 902 obtains the direct sound start point smax of the impulse response S4 using the following equation.

The end point of the direct sound is given by s _max + w. Here, w is the window width, and is set to a fixed value.

Next, the direct sound / reverberation sound division unit 902 obtains the direct sound component h _direct of the impulse response using the start point S _max using the following relational expression.

On the other hand, the direct sound / reverberation division unit 902 obtains the reverberation component h _reverb using the following equation.

The assumed speaker position impulse response estimation unit 903 uses the direct sound component h _direct and the reverberation component h _reverb of the impulse response, and receives a speech from the assumed speaker position with a virtually arranged conference microphone. Estimate the impulse response in the case. The impulse response estimation unit 903 is provided with information on the assumed speaker position and information on the assumed conference microphone. In FIG. 9, this information is indicated by S41. The impulse response h _synth is given by:

Here, α is a direct sound component attenuation rate and is given by the following equation.

Here, r _pre is a distance between the measurement speaker array 107 and the measurement microphone array 106 used when the impulse response is measured. r _post is a distance between the assumed speaker position and the conference microphone 104. The assumed speaker position has a size, not a single point. Therefore, setting the r _post giving the greatest r _post within the range of the set assumed speaker position.

β _pre is a coefficient determined depending on the directivity of the measurement microphone array 106 used for measuring the impulse response. In this embodiment, the direction in which the measurement microphone array 106 faces is the reference direction, and the directivity characteristic of the measurement microphone array 106 corresponding to the relative direction of the measurement speaker array 107 with respect to the direction is β _pre . .

β _post is a coefficient determined depending on the directivity characteristics of the virtually arranged conference microphone. In the case of this embodiment, the direction in which the conference microphone is directed is the reference direction, and the directivity characteristic of the measurement microphone array 106 corresponding to the relative direction of the position of the interlocutor relative to the direction is β _post .

γ _pre is a coefficient determined depending on the radiation characteristics of the speaker array for measurement 107 used for measuring the impulse response. In this example, the direction in which the measurement speaker array 107 faces is a reference direction, and the radiation characteristic of the measurement speaker array 107 corresponding to the relative direction of the measurement microphone array 106 with respect to the direction is γ _pre .

γ _post is a coefficient determined depending on the radiation characteristics of the assumed speaker position virtually arranged. In this example, the direction in which the assumed speaker is facing is set as the reference direction, and the radiation characteristic of the assumed speaker corresponding to the relative direction of the conference microphone with respect to the direction is set as γ _post .

  In general, it is considered that an assumed speaker is facing a direction in which a display installed in a conference room can be seen. Therefore, in the case of this embodiment, the assumed speaker position is set to the facing position of the display. Further, it is desirable that the radiation characteristics of the assumed speaker are measured in advance with a dummy head or the like and stored in a database.

The assumed speaker position impulse response estimation unit 903 outputs the impulse response h _synth generated for each assumed speaker position and ends the process. When there are a plurality of measured impulse responses, the impulse response estimation unit 903 selects an impulse response that minimizes the difference between r _pre and r _post .

The residual echo estimation unit 904 estimates the residual echo at the position of the virtually arranged conference microphone. As this preprocessing, the residual echo estimation unit 904 generates an impulse response from the virtually arranged conference speaker to the virtually arranged conference microphone based on the direct sound component and the reverberant component of the impulse response S4. . The impulse response is generated by the residual echo estimator 904 by the same processing procedure as the impulse response estimator 903 for the assumed speaker position. Let the generated impulse response be h _echo . Note that the residual echo estimation unit 904 is provided with information on the assumed speaker position, the assumed conference microphone, and the conference speaker. In FIG. 9, this information is indicated by S42.

Next, the residual echo estimation unit 904 calculates an impulse response h _residual of the residual echo from the following equation.

Here, λ and t _spec are parameters determined based on the specifications of the echo canceller to be used. For example, in the case of an echo canceller having a performance of echo cancellation time T seconds and 20 dB, λ corresponds to 0.1 and t _spec corresponds to T seconds. Such information is indicated by S43 in FIG. The residual echo estimation unit 904 outputs h _echo and h _residual and ends the processing.

The noise simulation unit 905 at the assumed microphone position estimates the noise level P _noise at the assumed microphone position using the sound volume and position information S21 to S2N of the sound source separated by the sound source separation.

Here, N is the number of sound sources. P _observed (i) is the volume of the i-th sound source separated by sound source separation. r _pre (i) is the distance from the i-th sound source position to the acoustic characteristic measurement microphone, and r _post (i) is the distance from the i-th sound source position to the virtually arranged conference microphone. In FIG. 9, information on the assumed conference microphone is indicated by S44. The noise simulation unit 905 outputs the estimated noise level P _noise and ends the process.

If necessary, the acoustic simulator displays the impulse responses h _synth , h _echo, h _residual and noise level P _noise calculated as a result of the simulation on the display 210 with characters and figures. By confirming the contents of this screen display, the user determines in advance what acoustic characteristics can be obtained when the conference microphone 104 and the conference speaker 103 are used at the virtual position targeted for simulation. can do.

  As described above, the acoustic simulator according to the present embodiment is clearly different from the prior art that adjusts the parameters of the audio signal processing in that the simulation is executed by virtual adjustment of the installation position.

(Processing 1 as acoustic consulting equipment)
Next, the processing operation as the acoustic consulting apparatus according to this embodiment will be described. Here, a case will be described in which whether or not the use conditions of the conference microphone and the speaker virtually input by the user satisfy the desired performance is determined every time the input is performed, and the determination result is notified to the user.

  The acoustic consulting apparatus evaluates the processing result of the acoustic simulation described above (that is, the estimated acoustic signal) and determines whether or not the conference microphone 104 and the conference speaker 103 are suitable for the video conference system setting environment 101. The evaluation result of is output. Of course, the acoustic consulting apparatus according to the embodiment is based on the premise that the CAD data related to the video conference system setting environment 101 cannot be used.

  FIG. 11 shows an outline of the processing procedure of the acoustic consulting apparatus. In FIG. 11, the same reference numerals are given to the portions corresponding to those in FIG. 3. The difference between FIG. 11 and FIG. 3 is processing 309, processing 310, and processing 311.

  In process 309, evaluation performance setting for evaluating the result of the acoustic simulation is executed. The user inputs the desired performance through the operation of the mouse 208, the keyboard 209, and other input devices. FIG. 12 shows an example of desired performance. In FIG. 12, the reverberation ratio amount of the speaker utterance collected by the conference microphone 104, the environmental noise ratio amount, and the residual echo ratio amount after the acoustic echo canceller are defined. Both are defined in the format of SNR (Signal To Noise Ratio).

  In the case of FIG. 11, the process 309 is arranged between the acoustic characteristic measurement process (process 306) and the microphone and speaker virtual information setting process (process 307), but the simulation result determination process (process) It may be arranged at any time point before executing 310).

  In the case of the acoustic consulting apparatus shown in FIG. 11, the virtual information of the conference microphone 104 and the conference speaker 103 registered in the process 307 merely gives an initial condition for evaluating the simulation result. Therefore, in the case of this embodiment, the information registered in the processing 301 and the processing 302 may be read as it is and registered as virtual information.

  In process 310, the acoustic consulting apparatus determines whether or not the simulation result of the acoustic environment executed for the virtual conference microphone and conference speaker satisfies a desired performance set in advance by the user.

  If it is determined that the performance is satisfied, the acoustic consulting apparatus outputs information virtually set for the conference microphone 103 and the conference speaker 104 as a condition that satisfies the desired performance, and ends the processing. To do. For example, the position and orientation information of the conference microphone 104 and the conference speaker 103 that can obtain desired performance is output.

  On the other hand, when it is determined that the performance is not satisfied, the acoustic consulting apparatus proceeds to processing 311. In the process 311, the acoustic consulting apparatus executes a process of accepting a change to the information of the conference microphone 104 and the conference speaker 103 that are virtually registered for simulation. The user interface used in processing 307 is used to input changes to the registration information. The registration information change can be input manually by the user or automatically. The automatic setting will be described later. In any case, when the completion of the change of the setting information is input from the user, the acoustic consulting apparatus returns to the processing 308 and executes an acoustic simulation based on the changed information.

  FIG. 13 shows a functional block configuration of a program used for determining whether or not an acoustic simulation result satisfies a desired performance. In the following description, the program is referred to as a performance evaluation unit 1301. The performance evaluation unit 1301 includes the following three evaluation units and one comparison unit.

The direct sound / reverberation sound ratio evaluation unit 1302 evaluates the ratio of the direct sound component h _direct and the reverberation sound component h _reverb from the impulse response h _synth of the assumed speaker position. First, the acoustic consulting apparatus separates the input impulse response h _synth into a direct sound component h _direct (t) and a reverberant sound component h _reverb (t) using Equations 2 and 3. When these components are obtained, the direct sound / reverberant ratio evaluation unit 1302 calculates the ratio _Preverb of the direct sound component and the reverberant part based on the following equation.

The direct sound / reverberation sound ratio evaluation unit 1302 outputs the minimum value of the calculated ratio _Preverb .
The speaker utterance / residual echo ratio evaluation unit 1303 estimates the ratio P _echo based on Equation 9 for each assumed speaker position.

Here, ρ is obtained by Expression 10.

Here, h _{1, direct} represents an impulse response when the distance between the virtually set conference microphone position and the assumed speaker position is 1 m. A is the assumed speaker volume at a distance of 1 m. μ is determined by Equation 11.

Here, h _sp represents an impulse response from the position of the conference speaker virtually set to the assumed speaker position. B is the sound pressure level of the speaker output signal at the assumed speaker position.

The speaker utterance / residual echo ratio evaluation unit 1303 _calculates and outputs the minimum value of P _echo .
Speaker speech / noise ratio evaluating unit 1304 obtains the P _n defined by Equation 12 for each assumed speaker position, determines and outputs the minimum value of P _n.

The desired performance comparing unit 1305 compares the desired performance S51 set by the user with _Preverb , P _echo, and P _n calculated by the respective units in the previous stage, and determines whether each value falls within the desired performance. judge. The comparison result for each value is output as the determination result S52.

  The acoustic consulting apparatus displays the determination result on the display 210 using characters and graphics. The user can know the determination result as to whether or not the use of the conference microphone 104 or the conference speaker 103 that satisfies the virtually specified condition satisfies the desired performance by confirming the content of the screen display. . If the desired performance is not satisfied, a condition for obtaining the desired performance can be searched by repeatedly specifying a new candidate.

(Processing 2 as an acoustic consulting device)
Here, an example of the processing operation of the acoustic consulting apparatus according to this embodiment will be described. Here, a case will be described in which the acoustic consulting apparatus is equipped with a function for automatically correcting the conference microphone and speaker so that the virtual conditions satisfy the desired performance.

  FIG. 14 shows an outline of the processing procedure of the acoustic consulting apparatus. In FIG. 14, the same reference numerals are given to corresponding parts to those in FIG. 11, and the processing content up to processing 308 is the same as in FIG. 11. Therefore, in the following, the description will be started from the time point after the acoustic simulation of the process 308 is executed.

  In processing 311, the acoustic consulting apparatus checks an error between the desired performance set by the user in advance and the simulation result. This processing is executed in the desired performance comparison unit 1305 shown in FIG. After this process 311, the acoustic consulting apparatus proceeds to process 312.

  In processing 312, the acoustic consulting apparatus determines whether or not the difference between the error in the previous simulation execution time and the error in the current simulation execution time is equal to or smaller than a predetermined threshold (that is, whether or not the convergence condition is satisfied). To do. If a negative result is obtained in this process 312, the acoustic consulting apparatus proceeds to process 313.

  In the process 313, the acoustic consulting apparatus changes the virtual information of the conference microphone and the speaker so that the evaluation function C defined in Equation 13 transitions in the minimum gradient direction, for example.

Here, a, b, and c represent weights of the performance evaluation scale, respectively.
In this specification, the change value of the cost function C when the position of the microphone and the position of the speaker are shifted by a minute direction is ΔC. In addition, the minute directions when moving in the minimum gradient direction are denoted by ΔM and ΔS, respectively.

  ΔM is given as a three-dimensional vector and represents the amount of change in each of the coordinate values x, y, z that specify the position of the conference microphone. Similarly, ΔS is given as a three-dimensional vector, and represents the amount of change in each of the coordinate values x, y, z specifying the position of the conference speaker.

The matrix [MS] _new indicates the virtual conference microphone placement and speaker placement after moving in the minimum gradient direction, as shown in the following equation, and the matrix [MS] _old is in the minimum gradient direction. The arrangement of the virtual conference microphone and the arrangement of the speakers before moving is shown.

  After the virtual information is automatically changed in this way, the acoustic consulting apparatus returns to the execution of the acoustic simulation in the process 308.

  If a positive result is obtained in the process 312 (when the error is smaller than a predetermined threshold and the convergence condition is satisfied), the acoustic consulting apparatus proceeds to the process 310. That is, the acoustic consulting apparatus determines whether the simulation result satisfies a desired performance. This determination process itself is the same as in the case of FIG. If the simulation result satisfies the desired performance, the acoustic consulting apparatus ends the process at that time.

  On the other hand, if a negative result is obtained, the acoustic consulting apparatus increases the number of conference microphones by one in process 314, and then returns to the execution of the acoustic simulation in process 308.

  As described above, in the case of this embodiment, the acoustic consulting apparatus automatically sets the position of the conference microphone and the position of the conference microphone (the number of conference microphones as necessary) that are optimal for the video conference. Can do. Of course, the determination result is displayed on the display 210 with characters and figures. For this reason, even when the user does not have the CAD data of the video conference system installation environment 101, the user can automatically obtain information regarding the optimum number and position of the conference microphones and speakers.

  In the above-described embodiment, the information is output when a condition that satisfies the desired performance is found, and the execution and evaluation of the simulation based on the virtual information change process and the changed information are stopped. , Repeats the execution and evaluation of simulation based on the change of virtual information and the information after the change within the variable range set in advance by the user, and displays the spatial arrangement and other conditions that satisfy the desired performance within the variable range It may be displayed above. In this case, an arrangement reflecting the user's desire can be selectively introduced even within a range satisfying the desired performance, and the usability can be improved.

(Other examples)
In addition, this invention is not limited to the form example mentioned above, Various modifications are included. For example, the above-described embodiments have been described in detail for easy understanding of the present invention, and are not necessarily limited to those having all the configurations described. Moreover, it is possible to replace a part of a certain form example with the structure of another form example, and it is also possible to add the structure of another form example to the structure of a certain form example. Moreover, it is also possible to add, delete, or replace another structure with respect to a part of structure of each form example.

  Moreover, you may implement | achieve some or all of each structure, a function, a process part, a process means, etc. which were mentioned above as an integrated circuit or other hardware, for example. Each of the above-described configurations, functions, and the like may be realized by the processor interpreting and executing a program that realizes each function. That is, it may be realized as software. Information such as programs, tables, and files for realizing each function can be stored in a memory, a hard disk, a storage device such as an SSD (Solid State Drive), or a storage medium such as an IC card, an SD card, or a DVD.

  Control lines and information lines indicate what is considered necessary for the description, and do not represent all control lines and information lines necessary for the product. In practice, it can be considered that almost all components are connected to each other.

  DESCRIPTION OF SYMBOLS 101 ... Video conference system installation environment, 102 ... Desk, 103 ... Conference speaker, 104 ... Conference microphone, 105-1 ... Assumed speaker position, 105-2 ... Assumed speaker position, 106 ... Measurement microphone array, 107 ... Speaker array for measurement, 202 ... Multi-channel AD converter, 203 ... Central processing unit, 204 ... Non-volatile memory, 205 ... Volatile memory, 206 ... Multi-channel DA converter, 208 ... Mouse, 209 ... Keyboard, 210 ... Display, 701 ... Acoustic characteristic measurement unit, 702 ... Impulse response measurement unit, 703 ... Ambient noise measurement unit, 802 ... Sound source separation unit, 803-1, 803-2, 803-N ... Sound source localization unit, 901 ... Sound simulation unit 902 ... Direct sound / reverberation division unit 903 ... Impulse response estimation unit for assumed speaker position, 90 ... residual echo estimation unit, 905 ... noise simulation unit at assumed microphone position, 1301 ... performance evaluation unit, 1302 ... direct sound / reverberation sound ratio evaluation unit, 1303 ... speaker utterance / residual echo ratio evaluation unit, 1304 ... speaker utterance / Noise ratio evaluation unit, 1305 ... Desired performance comparison unit

Claims (11)

A first storage device for storing acoustic data actually recorded in a space for constructing an acoustic system;
A second storage device for storing first information about the performance of the first microphone and the first speaker used at the time of recording the acoustic data and the actually measured position in the acoustic system;
A third storage device for storing second information regarding the position of the sound source assumed when the acoustic system is constructed;
A first setting receiving unit that receives a setting of third information related to the position and performance of the second microphone and the second speaker used when the acoustic system is constructed;
A fourth storage device for storing the third information;
An acoustic simulator comprising: a simulation unit configured to estimate an acoustic characteristic when the second microphone is used in the acoustic system based on the acoustic data and the first, second, and third information. The acoustic simulator according to claim 1,
An acoustic characteristic measurement unit that performs measurement of the acoustic data by the first microphone and the first speaker;
A second setting accepting unit for accepting settings of the first and second information;
An acoustic simulator comprising: a change receiving unit that receives a change in the setting of the third information. In the acoustic simulator according to claim 2,
The first setting receiving unit receives a virtual change with respect to the number of second microphones. In the acoustic simulator according to claim 2,
The acoustic simulator according to claim 1, wherein the first setting reception unit receives a virtual change with respect to a position of the second microphone. The acoustic simulator according to claim 1,
The acoustic simulator is a video conference system. A first storage device for storing acoustic data actually recorded in a space for constructing an acoustic system;
A second storage device for storing first information about the performance of the first microphone and the first speaker used at the time of recording the acoustic data and the actually measured position in the acoustic system;
A third storage device for storing second information regarding the position of the sound source assumed when the acoustic system is constructed;
A fourth storage device for storing third information on the position and performance of the second microphone and the second speaker used when the acoustic system is constructed;
A simulation unit for estimating acoustic characteristics when the second microphone is used in the acoustic system based on the acoustic data and the first, second, and third information;
A determination unit that determines whether or not the estimated acoustic characteristics satisfy a desired performance;
An acoustic consulting apparatus comprising: a presentation unit that outputs a determination result of the determination unit to a user interface. The acoustic consulting apparatus according to claim 6,
The determination unit includes a setting change unit that automatically changes at least a part of the third information when it is determined that the estimated acoustic characteristics do not satisfy a desired performance.
The said simulation part uses the said 3rd information after a change, and newly estimates the acoustic characteristic at the time of using a said 2nd microphone with the said acoustic system. The acoustic consulting apparatus characterized by the above-mentioned. The acoustic consulting apparatus according to claim 7,
The setting changer changes the number of second microphones. An acoustic consulting apparatus, wherein: The acoustic consulting apparatus according to claim 7,
The setting change unit changes the position of the second microphone. Processing to store the acoustic data actually recorded in the space for constructing the acoustic system in a storage device;
A process of accepting, through an input device, the setting of the first information regarding the performance of the first microphone and the first speaker used at the time of recording the acoustic data and the actually measured position in the acoustic system;
A process of accepting the setting of the second information related to the position of the sound source assumed at the time of constructing the sound system through the input device;
A process of accepting the setting of the third information related to the position and performance of the second microphone and the second speaker used at the time of constructing the acoustic system through the input device;
An acoustic simulation method comprising: processing for estimating an acoustic characteristic when the second microphone is used in the acoustic system based on the acoustic data and the first, second, and third information. Processing to store the acoustic data actually recorded in the space for constructing the acoustic system in a storage device;
A process of accepting, through an input device, the setting of the first information regarding the performance of the first microphone and the first speaker used at the time of recording the acoustic data and the actually measured position in the acoustic system;
A process of accepting the setting of the second information related to the position of the sound source assumed at the time of constructing the sound system through the input device;
Based on the third information on the position and performance of the second microphone and the second speaker used at the time of constructing the acoustic system, the acoustic data, and the first and second information, the second microphone is A process of estimating acoustic characteristics when used in the acoustic system;
A process of determining whether or not the estimated acoustic characteristics satisfy a desired performance;
And a process of outputting a determination result of the determination unit to a user interface.

Images