JP5515728B2 - Terminal device, processing method, and processing program - Google Patents

Description

  The present invention uses a voice input unit that receives voice input from the surroundings and a voice output unit that outputs voice based on a voice signal received from a terminal device at another base connected via a network. The present invention relates to a terminal device, a processing method, and a processing program that hold a conference with a terminal device at a base.

  When a conference is held between a plurality of bases, the video conference system transmits and receives audio signals such as voices between the bases. The terminal device at each site accepts input of speech such as the speech of a user who is a participant in the conference through a microphone. The terminal device transmits the sound collected by the microphone as a sound signal to the terminal device at another site via the network. The terminal device outputs an audio signal transmitted from another site as audio from the speaker.

  The microphone also accepts an acoustic echo input in which the sound output from the speaker wraps around. The terminal device may transmit an audio signal including the user's utterance and acoustic echo to the terminal device at another base. In recent years, in order to remove the influence of acoustic echoes, a proposal has been made to calculate the transmission characteristics of acoustic echoes that circulate from a speaker to a microphone and subtract the acoustic echo components from the audio signal (Patent Document 1).

JP 2008-261923 A

  However, in the technique described in Patent Document 1 described above, no consideration is given to reducing the acoustic echo itself that is a source of calculation of transfer characteristics. Therefore, the problem that it is difficult to remove the component of the acoustic echo from the audio signal when the acoustic echo is large is an example.

  In order to solve the above-described problem, the present invention adjusts the output mode of the audio output unit so that the acoustic echo can be accurately removed, thereby reducing the acoustic echo, a terminal device, a processing method, and a processing program The purpose is to provide.

In order to solve the above-described problems and achieve the object, the terminal device according to claim 1 is a voice input unit that receives voice input and voice received from a terminal device at another base connected via a network. A terminal device comprising: an audio output unit that outputs audio based on a signal; and a detection unit that detects an arrangement of a user , wherein the generation unit generates a test audio signal, and the generation unit generates the test audio signal Test sound output means for converting a test sound signal into a test sound and outputting it by the sound output section, and acoustic echo information generated by inputting the test sound output by the test sound output means to the sound input section A measurement unit for measuring the output, and a change for changing an output mode of the audio output unit based on the acoustic echo information measured by the measurement unit Comprising a control means, wherein the change control means, based on the arrangement of the user detected by the detection means performs change control of the output mode, the test sound output means by said detecting means, When it is detected that the user is placed in a predetermined range around the terminal device, the test sound is output.

  The terminal device according to claim 2 is characterized in that, in the above invention, the change control means performs change control of a directivity angle of a sound output by the sound output unit.

According to a third aspect of the present invention, in the above invention, the terminal device further includes an acquisition unit that acquires arrangement information related to the arrangement of the terminal device, and the test voice output unit is configured to perform the test when the arrangement information is changed. It is characterized by outputting sound.

The processing method according to claim 4 includes: a voice input unit that receives voice input; a voice output unit that outputs voice based on a voice signal received from a terminal device at another base connected via a network; a detection knowledge means for detecting the placement of the user, a processing method by the terminal device provided with, converting a generation step of generating a test sound signal, the test sound signal generated by said generating step to test sound A test voice output step to be output by the voice output unit, a measurement step of measuring acoustic echo information generated by the test voice output by the test voice output step being input to the voice input unit, and the measurement A change control step of changing an output mode of the sound output unit based on the acoustic echo information measured by the step, and the change The control step performs change control of the output mode based on the arrangement of the users detected by the detection unit, and the test voice output step sets the predetermined range around the terminal device by the detection unit. When it is detected that the user is placed, the test sound is output.

The processing program according to claim 5 includes: a voice input unit that receives voice input; a voice output unit that outputs voice based on a voice signal received from a terminal device at another base connected via a network; a detection knowledge unit and a processing program for the terminal apparatus having a for detecting the placement of the user, a generation step of generating a test sound signal, the test sound signal generated by said generating step to test sound A test voice output step that is converted and output by the voice output unit; a measurement step that measures acoustic echo information generated by the test voice output by the test voice output step being input to the voice input unit; Based on the acoustic echo information measured by the measurement step, a change control process for changing the output mode of the audio output unit The change control step performs change control of the output mode based on the arrangement of the users detected by the detection means, and the test sound output step is executed by the detection means. The test voice is output when it is detected that the user is placed in a predetermined range around the terminal device.

According to the first aspect of the invention, the output mode of the audio output unit can be changed based on the acoustic echo information measured by the test audio signal. Therefore, the acoustic echo can be reduced so that the acoustic echo can be accurately removed. Then, the reduced acoustic echo information can be filtered by the adaptive filter, and the acoustic echo can be accurately executed. In addition, it is possible to control the change of the output mode of the audio output unit by detecting the user's arrangement. Therefore, the change control of the output mode of the audio output unit can be made appropriate in accordance with the arrangement of the users. Furthermore, when the user uses the terminal device, a test voice can be output. Therefore, it is possible to optimize the timing for executing the output mode change control of the audio output unit.

  According to the second aspect of the invention, the acoustic echo information can be easily reduced by changing and controlling the angle at which the voice is output by the voice output unit.

According to the third aspect of the present invention, it is possible to output a test voice when there is a change in the arrangement information of the terminal device. Therefore, it is possible to optimize the timing for executing the output mode change control of the audio output unit.

According to the fourth aspect of the present invention, the output mode of the voice output unit can be changed based on the acoustic echo information measured by the test voice signal. Therefore, the acoustic echo can be reduced so that the acoustic echo can be accurately removed. Then, the reduced acoustic echo information can be filtered by the adaptive filter, and the acoustic echo can be accurately executed. In addition, it is possible to control the change of the output mode of the audio output unit by detecting the user's arrangement. Therefore, the change control of the output mode of the audio output unit can be made appropriate in accordance with the arrangement of the users. Furthermore, when the user uses the terminal device, a test voice can be output. Therefore, it is possible to optimize the timing for executing the output mode change control of the audio output unit.

According to the fifth aspect of the present invention, the output mode of the sound output unit can be changed by the computer based on the acoustic echo information measured by the test sound signal. Therefore, the acoustic echo can be reduced so that the acoustic echo can be accurately removed. The reduced acoustic echo information can be filtered by the adaptive filter by the computer, and the acoustic echo can be accurately executed. In addition, it is possible to perform change control of the output mode of the audio output unit by detecting the arrangement of the user by the computer. Therefore, the change control of the output mode of the audio output unit can be made appropriate in accordance with the arrangement of the users. Further, the computer can output a test voice when the user uses the terminal device. Therefore, it is possible to optimize the timing for executing the output mode change control of the audio output unit.

  As described above, according to the terminal device, processing method, and processing program according to the present invention, the acoustic echo can be reduced by adjusting the output mode of the audio output unit so that the acoustic echo can be accurately removed. There is an effect that can be.

It is explanatory drawing which shows an example of the video conference system of embodiment of this invention. It is explanatory drawing which shows an example of a functional structure of the video conference terminal of embodiment of this invention. It is explanatory drawing which shows an example of a functional structure of the audio | voice I / F of embodiment of this invention. It is explanatory drawing which shows an example of the change control of the direction of the speaker of embodiment of this invention. It is explanatory drawing which shows an example of the measurement result of the acoustic echo information of embodiment of this invention. It is a flowchart which shows the content of the process of the video conference system of embodiment of this invention. It is explanatory drawing which shows an example of the angle control of the speaker with respect to the some user in the modification of embodiment of this invention. It is explanatory drawing which shows an example of the change control of the ON / OFF state of a peaker in the modification of embodiment of this invention. It is explanatory drawing which shows an example of the change control of the output state of a peaker in the modification of embodiment of this invention. It is a flowchart which shows the content of the process of the video conference terminal in the modification of embodiment of this invention.

  Exemplary embodiments of a terminal device, a processing method, and a processing program according to the present invention are explained in detail below with reference to the accompanying drawings.

(Embodiment)
(overall structure)
A case where the terminal device according to the embodiment of the present invention is applied to a video conference terminal used in a video conference system that performs a video conference at a plurality of sites will be described with reference to FIG. FIG. 1 is an explanatory diagram showing an example of a video conference system according to an embodiment of the present invention. In FIG. 1, a video conference system 100 is configured by connecting video conference terminals 110a and 110b installed in conference rooms A and B via a network NW.

  In the video conference system 100, video conference terminals 110a and 110b installed in geographically separated conference rooms A and B are connected via a network NW such as the Internet, or are connected to remote conference rooms A and B in a building. The installed video conference terminals 110a and 110b are connected via a network NW such as a LAN (local area network).

  In addition, although FIG. 1 demonstrates the case where a video conference is performed in two meeting rooms A and B, it is good also as performing a video conference in three or more bases. In addition, although the video conference terminals 110a and 110b are described as being connected to each other via a network, a configuration in which the video conference terminals 110a and 110b are connected to each other via a management server installed at an arbitrary position on the network may be employed. The network NW may be a public telephone line network.

  The video conference system 100 causes the video conference terminal 110 to transmit and receive video and audio of the conference in the conference rooms A and B. Specifically, the video conference terminal 110a acquires video and audio of a user who participates in the video conference in the conference room A using the camera 212a and the microphone 214a. The video conference terminal 110a packetizes the acquired video and audio, and transmits them as video signals and audio signals to the video conference terminal 110b in the conference room B via the network NW.

  The video conference terminal 110a receives the video signal and the audio signal transmitted from the video conference terminal 110b in the conference room B via the network NW. The video conference terminal 110a reproduces video and audio from the received signal through the display 211a and the speakers 213a (SP1 to SP4). The number of speakers 213 installed in the conference rooms A and B is not limited to four and can be increased or decreased as necessary.

  When the sound is reproduced by the speakers 213a (SP1 to SP4), an acoustic echo is generated due to reverberation in the conference room A or sound wraparound. The video conference terminal 110a transmits a voice signal including a voice and a sound echo from a user participating in the video conference in the conference room A collected by the microphone 214a to the video conference terminal 110b of the communication destination. Note that the number of microphones 214 installed in the conference rooms A and B is not limited to two and can be increased or decreased as necessary.

  Here, the video conference terminal 110a generates a test audio signal different from the audio signal received from the video conference terminal 110b. The video conference terminal 110a outputs a test audio signal as a test audio from the speaker 213a.

  The video conference terminal 110a collects the echo and wraparound of the test sound output from the speaker 213a with the microphone 214a. The video conference terminal 110a measures acoustic echo information caused by the test voice based on the sound collection by the microphone 214a.

  The video conference terminal 110a performs change control of the output mode of the speakers 213a (SP1 to SP4). Specifically, the video conference terminal 110a changes the direction of each speaker 213a (SP1-SP4). The video conference terminal 110a changes the direction of each speaker 213a (SP1 to SP4) while outputting the test voice, and measures voice echo information. The video conference terminal 110a adjusts the direction of each speaker 213a (SP1 to SP4) that is the directivity angle of the output sound so that the acoustic echo due to the test sound is minimized.

  The video conference terminal 110a sets an adaptive filter for filtering the acoustic echo. That is, the video conference terminal 110a sets the adaptive filter based on the acoustic echo input by the speaker 213a (SP1 to SP4) whose direction is adjusted so that the acoustic echo is minimized. The video conference terminal 110a filters an audio signal including voice and acoustic echo from a user participating in the video conference in the conference room A collected by the microphone 214a by an adaptive filter and transmits the filtered audio signal to the video conference terminal 110b of the communication destination. To do.

  By using the speakers 213a (SP1 to SP4) adjusted so that the acoustic echo is reduced by the test voice, the acoustic echo during the meeting is also reduced. Therefore, the video conference terminal 110a can reliably remove the acoustic echo from the audio signal to be transmitted to the video conference terminal 110b.

  In the embodiment of the present invention, the configuration of removing the acoustic echo by adjusting the direction of the speaker 213a (SP1 to SP4) has been described by taking the video conference terminal 110a as an example, but the present invention is not limited to this. That is, other devices such as the video conference terminal 110b may have the same configuration.

(Functional configuration)
A functional configuration of the video conference terminal 110 according to the embodiment of the present invention will be described with reference to FIG. FIG. 2 is an explanatory diagram illustrating an example of a functional configuration of the video conference terminal according to the embodiment of this invention.

  In FIG. 2, the video conference terminal 110 inputs and outputs video to and from a CPU (Central Processing Unit) 201, a RAM (Random Access Memory) 202, a ROM (Read Only Memory) 203, a display 211 and a camera 212. Video I / F 204 to be controlled, audio I / F 205 that controls input / output of audio to / from speaker 213 and microphone 214, operation unit 206 that receives input of various information, and communication I that controls communication with external devices / F207 and a storage medium 208 for storing various types of information. Each component of the video conference terminal 110 is connected by a bus 200.

  The CPU 201 controls the entire video conference terminal 110. The CPU 201 executes various programs read from the ROM 203 using the RAM 202 as a work area.

  The video I / F 204 displays various information on the display 211 under the control of the CPU 201. The video I / F 204 is, for example, a video of the local site that is the conference room A or the conference room B captured by the camera 212 or a video received from the video conference terminal 110 of another site that is the conference room B or the conference room A. An image obtained by decoding the signal, a processing screen related to a video conference with another base, and the like are displayed on the display 211.

  The video I / F 204 captures a video of the user at the local site by the camera 212 under the control of the CPU 201. The video I / F 204 outputs the video captured by the camera 212 to the storage medium 208 according to the control of the CPU 201.

  The sound I / F 205 causes the speaker 213 to output various sounds according to the control of the CPU 201. Audio | voice I / F205 outputs the audio | voice which decoded the audio | voice signal received from the video conference terminal 110 of another base, the guidance audio | voice regarding a video conference with another base, etc. to the speaker 213.

  The voice I / F 205 collects the voice of the user at the local site by the microphone 214 under the control of the CPU 201. When sound is output from the speaker 213, acoustic echoes due to reverberation or wraparound in the conference rooms A and B occur. The voice I / F 205 also collects acoustic echoes of its own base by the microphone 214 under the control of the CPU 201. The sound I / F 205 outputs the sound collected by the microphone 214 to the storage medium 208 according to the control of the CPU 201.

  The audio I / F 205 generates a test audio signal different from the audio received from another base (another conference room) according to the control of the CPU 201. The audio I / F 205 adjusts the output mode of the speaker 213 using the generated test audio signal so that the acoustic echo is minimized.

  Details of the audio I / F 205 according to the embodiment of this invention will be described with reference to FIG. FIG. 3 is an explanatory diagram illustrating an example of a functional configuration of the audio I / F according to the embodiment of this invention.

  In FIG. 3, the audio I / F 205 is input from a test signal generation unit 301 that generates a test audio signal, an output mode control unit 302 that can control the driving unit 305 to adjust the output mode of the speaker 213, and a microphone 214. A residual echo measurement unit 303 that measures acoustic echo information regarding the acoustic echo to be performed, an adaptive filter setting unit 304 that sets an adaptive filter used to remove the acoustic echo, and a D / A that converts a digital audio signal into analog audio A converter 310 and an A / D converter 311 that converts analog sound into a digital sound signal are provided.

  The test signal generator 301 generates a test audio signal according to the control of the CPU 201. The test sound signal is a signal from which a test sound different from the sound received from another base is output. The audio I / F 205 converts the test audio signal into the test audio by the D / A converter 310 under the control of the CPU 201 and outputs the test audio signal from the speakers 213 (SP1 to SP4).

  Specifically, the CPU 201 determines that the user exists in a predetermined range around the video conference terminal 110 from the video of the local site captured by the camera 212 or the installation location of the video conference terminal 110 has changed. For example, a test audio signal is generated by the audio I / F 205. In other words, the CPU 201 generates a test audio signal at the timing when the video conference is started or when the change occurs in the surrounding environment.

  The output mode control unit 302 drives the drive unit 305 according to the control of the CPU 201 to execute the direction change control of the speakers 213 (SP1 to SP4). Specifically, the CPU 201 performs change control of a predetermined angle from the direction of the user facing each speaker 213 (SP1 to SP4) by the camera 212, and outputs a test sound from each speaker 213 (SP1 to SP4). .

  Here, the change control of the direction of the speakers 213 (SP1 to SP4) according to the embodiment of the present invention will be described with reference to FIG. FIG. 4 is an explanatory diagram illustrating an example of change control of the direction of the speaker according to the embodiment of this invention.

  In FIG. 4, when the CPU 201 detects the direction of the user M facing the speaker 213 (SP1 to SP4) by the camera 212, the CPU 201 sets the direction of the user M to θ1. Specifically, the CPU 201 extracts a human image from images captured by the camera 212. CPU201 detects the direction of the user M with respect to the speaker 213 (SP1-SP4) based on the direction of the camera 212 at the time of imaging, and the extracted person. The CPU 201 controls the audio I / F 205 to set θ2, θ3, θ4, and θ5 at predetermined angular intervals from θ1. In the example of FIG. 4, θ1 is 0 °, and the range is 60 ° to −60 ° at 30 ° intervals.

  The CPU 201 controls the audio I / F 205 to direct the speakers 213 (SP1 to SP4) toward the respective users and to output test audio from the speakers 213 (SP1 to SP4), respectively. As described above, by setting the angle at a predetermined angle from the direction of the user M, it is possible to prevent the speaker 213 (SP1 to SP4) from being in an unsuitable direction for the user M to listen to the voice.

  If there is no user facing each speaker 213 (SP1 to SP4), for example, the direction initially set by each video conference terminal 110 or the direction in which a user such as a chair can be placed is set to θ1. It is good as well.

  Returning to FIG. 3, the residual echo measurement unit 303 measures acoustic echo information regarding the acoustic echo input from the microphone 214. The residual echo measurement unit 303 extracts the test sound signal when the test sound signal generated by the test signal generation unit 301 is output as the test sound by the speaker 213. The CPU 201 converts the sound collected by the microphone 214 into an audio signal by the A / D converter 311. The residual echo measurement unit 303 detects the acoustic echo information by comparing the extracted test sound signal with the sound signal collected by the microphone 214.

  The measurement result of the acoustic echo information according to the embodiment of the present invention will be described with reference to FIG. FIG. 5 is an explanatory diagram illustrating an example of a measurement result of acoustic echo information according to the embodiment of the present invention.

  In FIG. 5, a measurement result table 500 is stored in the storage medium 208, and shows the magnitude of the acoustic echo component when the direction change control shown in FIG. 4 is executed for each speaker 213 (SP1 to SP4). Yes. That is, it is the result of measuring the acoustic echo by the test voice for all combinations in which the direction of each speaker 213 (SP1 to SP4) is any one of θ1 to 5, respectively.

  In the example of FIG. 5, the acoustic echo is in the minimum state 501 when the speaker 213 (SP1) is θ1, the speaker 213 (SP2) is θ2, the speaker 213 (SP3) is θ1, and the speaker 213 (SP4) is θ1. It is shown that.

  Returning to FIG. 3, the CPU 201 drives the drive unit 305 by controlling the audio I / F 205 based on the measurement result by the residual echo measurement unit 303 according to the output of the test audio. Specifically, the CPU 201 changes the output mode control unit 302 so that the speaker 213 (SP1) is θ1, the speaker 213 (SP2) is θ2, the speaker 213 (SP3) is θ1, and the speaker 213 (SP4) is θ1. Execute control.

  The adaptive filter setting unit 304 compares the audio signal output from the speaker 213 with the audio signal input from the microphone 214 under the control of the CPU 201 and sets an adaptive filter that filters the acoustic echo component. That is, since the adaptive filter is set according to the output mode of the speaker 213 set so that the acoustic echo due to the test voice is minimized, the acoustic echo can be accurately filtered.

  Returning to FIG. 2, the operation unit 206 accepts input of various information from a user or the like. The operation unit 206 includes a touch panel, operation buttons, and the like. The operation unit 206 accepts input of information regarding a video conference and outputs an input signal to the CPU 201.

  The communication I / F 207 is connected to a network NW such as the Internet through a communication line, and is connected to other video conference terminals 110 and other external devices via this network NW. The communication I / F 207 controls an interface inside the network NW and the video conference terminal 110 and controls data input / output with respect to an external device. As the communication I / F 207, for example, a modem or a LAN adapter can be employed.

  The communication I / F 207 transmits the video and audio of its own site stored in the storage medium 208 to the video conference terminal 110 at another site via the network NW according to the control of the CPU 201. The communication I / F 207 transmits video and audio as video signals and audio signals via the network NW at a predetermined timing in accordance with the control of the CPU 201.

  The storage medium 208 is an HD (hard disk) or an FD (flexible disk) as an example of a removable recording medium. The storage medium 208 has respective drive devices, and various data are recorded under the control of the CPU 201. Further, data is read from the storage medium 208 according to the control of each drive device.

  If each component is described in association with each function, the functions of the generation unit, test audio output unit, measurement unit, and change control unit of the present invention are realized by the CPU 201 and the audio I / F 205 shown in FIG. To do. Specifically, the test signal generator 301 shown in FIG. 3 functions as the generation means of the present invention, the residual echo measurement section 303 functions as the measurement means according to the present invention, and the output mode control section 302 functions as the change control means according to the present invention. Realize. The CPU 201, the camera 212, and the video I / F 204 realize the function of the detection unit of the present invention. Further, the speaker 213 implements the functions of the audio output unit of the present invention and the microphones of the audio input unit of the present invention.

(Contents of processing of the video conference system 100)
The contents of processing of the video conference system 100 according to the embodiment of the present invention will be described with reference to FIG. FIG. 6 is a flowchart showing the contents of processing of the video conference system according to the embodiment of the present invention.

  In the flowchart of FIG. 6, first, the CPU 201 determines whether or not a video conference has started (step S601). To start a video conference, for example, a connection request is made to another video conference terminal 110 via the communication I / F 207 based on the operation of the operation unit 206 by the user. This is determined by receiving a response from another video conference terminal 110 via the communication I / F 207.

  In step S601, the CPU 201 sets the speakers 213 (SP1 to SP4) in an initial state (step S602) after waiting for the video conference to be started and starting the video conference (step S601: Yes). The initial state is the state of θ1 shown in FIG.

  The CPU 201 controls the audio I / F 205 to cause the test signal generator 301 to generate a test audio signal (step S603), and outputs the test audio from the speaker 213 set in step S602.

  The CPU 201 controls the audio I / F 205 and measures acoustic echo information using the residual echo measurement unit 303 input from the microphone 214 (step S604). The CPU 201 stores the measured acoustic echo information in the storage medium 208 as the measurement result table 500 (step S605).

  The CPU 201 controls the audio I / F 205 to change the direction of the speaker 213 by driving the drive unit 305 (step S606), and the combinations of all directions within a predetermined angle range for each speaker 213 (SP1 to SP4). It is determined whether or not the processing has been completed (step S607).

  If all combinations have not been completed in step S606 (step S607: No), the CPU 201 returns to step S603, controls the voice I / F 205, and generates a test audio signal by the test signal generator 301. And repeat the process.

  In step S606, when all the combinations are completed (step S607: Yes), the CPU 201 determines the setting of the minimum state of the acoustic echo from the measurement result table 500 stored in the storage medium 208 (step S608). .

  CPU201 sets the direction of speaker 213 (SP1-SP4) based on the setting determined in Step S607 (Step S609), and ends a series of processings.

  The processing in each component of the present invention will be described in association with each processing or each function in the embodiment of the present invention. By the processing of the CPU 201, the audio I / F 205, and the test signal generation unit 301 in step S603, the present processing is performed. The process of the production | generation process and test audio | voice output process in the processing method of invention is performed. The processing of the measurement process in the processing method of the present invention is executed by the processing of the CPU 201, the audio I / F 205, and the residual echo measurement unit 303 in step S604. The processing of the change control step in the processing method of the present invention is executed by the processing of the CPU 201, the audio I / F 205, and the driving unit 305 in steps S605 to S609.

  As described above, according to the video conference system, the video conference terminal, and the processing method of the embodiment of the present invention, the test audio signal is generated, and the speaker orientation is set so that the acoustic echo due to the test audio is minimized. be able to. Therefore, since the filtering process by the adaptive filter can be performed in a state where the acoustic echo is minimum, the acoustic echo can be accurately removed.

  Further, according to the embodiment of the present invention, the speaker direction change control is performed within a predetermined range from the direction in which the user exists. Therefore, the speaker direction can be controlled within an appropriate usage range for the user.

  In particular, according to the embodiment of the present invention, even when there are a plurality of microphones and speakers, or when there are a plurality of users, acoustic echo removal is performed after minimizing acoustic echoes. Will be performed. Accordingly, accurate acoustic echo removal can be performed.

(Other variations)
In the embodiment of the present invention, in particular, in step S601 shown in FIG. 6, the test audio signal is generated when the video conference is started. However, the present invention is not limited to this. Specifically, the CPU 201 captures an image of the conference room in which the video conference is being held by the camera 212 functioning as the detection unit and the acquisition unit of the present invention at the start of the video conference or at a predetermined cycle. If the captured image is different from the previously captured image, the CPU 201 may generate a test audio signal as a timing at which a change occurs in the surrounding environment such as the arrangement of the user and the installation location of the video conference terminal. Good.

  Here, the content of the process of the video conference terminal 110 in the modification of the embodiment of the present invention will be described with reference to FIG. FIG. 10 is a flowchart showing the contents of processing of the video conference terminal according to the modification of the embodiment of the present invention. In the flowchart of FIG. 10, processes similar to those in FIG. 6 are denoted by the same step numbers and description thereof is omitted.

  In the flowchart of FIG. 1, when a video conference is started in step S601, the CPU 201 images the surrounding environment in the conference room with the camera 212 via the video I / F 204 (step S1001), and records the captured image data. It is stored in the medium 208.

  The CPU 201 compares the image data previously stored in the storage medium 208 with the image data captured in step S1001, and determines whether or not the installation location where the video conference terminal 110 is installed has changed. (Step S1002).

  In step S1002, when there is no change in the installation location (step S1002: No), the series of processing ends as it is. That is, when the environment is the same as that of the previous video conference, it is possible to save the trouble of adjusting the speaker again. If the installation location is changed in step S1002 (step S1002: Yes), the process proceeds to step S602 and subsequent steps shown in FIG.

  In the flowchart of FIG. 10, the installation location change is detected using image data captured after the start of the video conference. However, the present invention is not limited to this. In other words, by comparing the image data captured at a predetermined cycle after the start of the video conference, it is possible to detect entry / exit of users during the conference, change of conference location, etc., and respond quickly to changes in the surrounding environment It is good.

  Thus, by adjusting the timing for generating the test audio signal, the setting of the speaker 213 can be accurately changed at a timing at which the state of the acoustic echo may change. Therefore, acoustic echo filtering can be reliably performed. The change in the surrounding environment is not limited to detection by the camera 212. That is, a user's login state or position detecting means such as GPS (Global Positioning System) may be used. In this way, it is possible to provide versatility to the surrounding environment detection means.

  In the embodiment of the present invention, the test voice is not particularly limited, but is not limited to one type. That is, acoustic echo information may be measured using a plurality of types of test sounds. In this way, acoustic echo information can be accurately measured. The test sound is not limited to an audible sound, and may be a non-audible sound. By making the test sound non-audible, even if the test sound is output while the user is using such as during a video conference, the user does not feel uncomfortable.

  In the embodiment of the present invention, the test audio signal has been described as generating a test audio signal different from the audio of another site, but the present invention is not limited to this. Specifically, the test audio signal may be generated based on part or all of the audio signal received from another site. That is, an audio signal received from another site may be used as it is as a test audio signal, or a part of an audio signal received from another site may be used as a test audio signal. In this way, the processing load for generating the test audio signal can be reduced.

  In the embodiment of the present invention, the direction of the speaker 213 is set to the minimum state 500 as shown in the measurement result table 500, but the present invention is not limited to this. That is, the acoustic echo size may be set to a predetermined value or less, such as a level at which the acoustic echo can be removed. In this way, it is not necessary to try all combinations of orientations of the speakers 213, so that quick setting and reduction of processing load can be achieved.

  When there are a plurality of minimum states 500, the direction close to θ1 facing the user may be set with priority. In this way, the function of the part that allows the user to listen to the sound as the original role of the speaker 213 is not impaired.

  In the embodiment of the present invention, the direction of one user is set to θ1 with respect to the speaker 213, but the present invention is not limited to this. Specifically, the angle control of the speaker 213 may be performed on a plurality of users within a range of angles including the plurality of users.

  Here, the case where the angle control of the speaker 213 is performed by a plurality of users in a modification of the embodiment of the present invention will be described with reference to FIG. FIG. 7 is an explanatory diagram showing an example of speaker angle control for a plurality of users according to a modification of the embodiment of the present invention.

  In FIG. 7, two users M1 and M2 exist in front of the speaker 213. The video conference terminal 110 images the users M1 and M2 with the camera 212. The video conference terminal 110 detects the directions of the users M1 and M2 with respect to the speaker 213 from the captured image data and the orientation of the camera 212 at the time of imaging. The video conference terminal 110 sets the direction of the speaker by generating a test sound at a predetermined interval with respect to the angle range θ including the directions of the users M1 and M2. In this way, the direction of the speaker can be accurately set for a plurality of users.

  In the embodiment of the present invention, the angle change of the speaker 213 is particularly controlled to detect the minimum state of the acoustic echo. However, the present invention is not limited to this. Specifically, the minimum state of the acoustic echo may be detected based on the ON / OFF state of the speaker 213 instead of the angle of the speaker 213.

  Here, the change control of the ON / OFF state of the speaker in a modification of the embodiment of the present invention will be described with reference to FIG. FIG. 8 is an explanatory view showing an example of change control of the ON / OFF state of the peaker in a modification of the embodiment of the present invention.

  In FIG. 8, a measurement result table 800 is stored in the storage medium 208 and indicates the magnitude of the acoustic echo component when each speaker 213 (SP1 to SP4) is turned on or off. The audio I / F 205 turns each speaker 213 (SP1 to SP4) ON or OFF according to the control of the CPU 201. That is, it is the result of measuring the acoustic echo by the test voice when all the combinations in which the state of each speaker 213 (SP1 to SP4) is either ON or OFF, except for the OFF state. .

  In the example of FIG. 8, the acoustic echo is in the minimum state 801 when the speaker 213 (SP1) is OFF, the speaker 213 (SP2) is OFF, the speaker 213 (SP3) is OFF, and the speaker 213 (SP4) is ON. It is shown that.

  Thus, by measuring the acoustic echo information according to the ON / OFF state of the speaker 213 (SP1 to SP4), the minimum state 801 can be reduced with a simpler mechanism than when the angle of the speaker 213 (SP1 to SP4) is controlled. Can be set.

  Moreover, it is good also as detecting the minimum state of an acoustic echo based on an output value, making the sum total output constant about the output of the speaker 213 (SP1-SP4) instead of the angle of a speaker 213, or an ON / OFF state.

  Here, the change control of the output state of the speaker in a modification of the embodiment of the present invention will be described with reference to FIG. FIG. 9 is an explanatory diagram showing an example of change control of the output state of the peaker in a modification of the embodiment of the present invention.

  In FIG. 9, a measurement result table 900 is stored in the storage medium 208, and indicates the magnitude of the acoustic echo component for each speaker 213 (SP1 to SP4) by a combination of output states where the sum of output values is 100. . The audio I / F 205 can change the output value of each speaker 213 (SP1 to SP4) by an amplifier (not shown) or the like under the control of the CPU 201. That is, it is a result of measuring acoustic echoes due to a combination and a test voice in which a certain output is maintained when the speakers 213 (SP1 to SP4) are summed.

  In the example of FIG. 9, the acoustic echo is in the minimum state 901 when the speaker 213 (SP1) is 25, the speaker 213 (SP2) is 25, the speaker 213 (SP3) is 25, and the speaker 213 (SP4) is 25. It is shown that.

  In the above description, the embodiment and some of the modifications have been described as separate examples, but the present invention is not limited to this. That is, as a combination of the above, the methods according to the embodiment and some modifications may be used in appropriate combination.

  Note that the methods described in the embodiments and modifications of the present invention can be realized by executing a program prepared in advance on a computer such as a personal computer or a workstation. The communication program is recorded on a computer-readable recording medium such as a hard disk, a flexible disk, a CD-ROM, an MO, and a DVD, and is executed by being read from the recording medium by the computer. The program may be a transmission medium that can be distributed via a network such as the Internet.

100 video conference system 110 (110a, 110b) video conference terminal 200 bus 201 CPU
202 RAM
203 ROM
204 Video I / F
205 Voice I / F
206 Operation unit 207 Communication I / F
DESCRIPTION OF SYMBOLS 208 Storage medium 211 Display 212 Camera 213 Speaker 214 Microphone 301 Test signal generation part 302 Output mode control part 303 Residual echo measurement part 304 Adaptive filter measurement part 305 Drive part

Claims (5)

A voice input unit that receives a voice input; a voice output unit that outputs a voice based on a voice signal received from a terminal device connected to another base via a network; and a detection unit that detects the arrangement of the user. A terminal device comprising:
Generating means for generating a test audio signal;
Test audio output means for converting the test audio signal generated by the generating means into test audio and outputting the test audio signal by the audio output unit;
Measuring means for measuring acoustic echo information generated by the test sound output by the test sound output means being input to the sound input unit;
Based on the acoustic echo information measured by the measuring means, change control means for changing the output mode of the audio output unit;
With
The change control means performs change control of the output mode based on the arrangement of the users detected by the detection means,
The test voice output means outputs the test voice when the detection means detects that the user is placed in a predetermined range around the terminal apparatus.   The terminal apparatus according to claim 1, wherein the change control unit performs change control of a directivity angle of a sound output by the sound output unit. An acquisition means for acquiring arrangement information related to the arrangement of the terminal device;
The terminal device according to claim 1, wherein the test sound output unit outputs the test sound when the arrangement information is changed. Detection knowledge means for detecting an audio input unit for receiving input of voice, a voice output unit for outputting sound based on the audio signal received from the terminal device connected different hub via a network, the placement of the user A processing method by a terminal device comprising:
A generation process for generating a test audio signal;
A test voice output step of converting the test voice signal generated by the generation step into a test voice and outputting the test voice signal by the voice output unit;
A measurement step of measuring acoustic echo information generated by inputting the test voice output by the test voice output step to the voice input unit;
Based on the acoustic echo information measured by the measurement step, a change control step for changing the output mode of the audio output unit;
Including
The change control step performs change control of the output mode based on the arrangement of the users detected by the detection means,
The test voice output step outputs the test voice when the detecting means detects that the user is placed in a predetermined range around the terminal device. Detection knowledge means for detecting an audio input unit for receiving input of voice, a voice output unit for outputting sound based on the audio signal received from the terminal device connected different hub via a network, the placement of the user A processing program for a terminal device comprising:
A generation process for generating a test audio signal;
A test voice output step of converting the test voice signal generated by the generation step into a test voice and outputting the test voice signal by the voice output unit;
A measurement step of measuring acoustic echo information generated by inputting the test voice output by the test voice output step to the voice input unit;
Based on the acoustic echo information measured by the measurement step, a change control step for changing the output mode of the audio output unit;
To the computer,
The change control step performs change control of the output mode based on the arrangement of the users detected by the detection means,
The test voice output step outputs the test voice when the detecting means detects that the user is placed in a predetermined range around the terminal device. Processing program.