JP7095356B2 - Communication terminal and conference system - Google Patents

Description

The present invention relates to a communication terminal and a conference system.

In recent years, a conference system for holding a conference between remote locations via a communication network such as the Internet has become widespread.

In this conference system, communication terminals are installed in a plurality of conference rooms that are remote from each other. Each communication terminal captures and collects images of conference rooms such as those involved in the conference and audio such as remarks. Then, each communication terminal converts the captured and collected images and sounds into digital data and transmits the images and sounds to other communication terminals set in the other conference rooms. Further, when the communication terminal receives an image or sound from another communication terminal, the communication terminal displays the image on the display and outputs the sound to the speaker. As a result, a technique for conducting a conference between remote locations in a state close to an actual conference is already known.

Further, a technique is already known in which a communication terminal transmits the collected voice to a voice recognition engine, and the voice recognition engine automatically converts the contents of remarks of the parties and the like into text and automatically generates minutes.

An example of the above technique is the conference system of Patent Document 1. In the conference system of Patent Document 1, each of a plurality of conference terminals (communication terminals) transmits the voice of the conference room (own base) in which the conference terminal is installed (own base) to the recording server (voice recognition engine), and the recording server performs each conference. The minutes are created by synthesizing the voice received from the terminal in chronological order.

However, the sound processing such as noise removal processing that is easily recognized by the voice recognition engine (machine) and the (natural) sound processing that is easy for the person at the opposite base of the remote conference to hear are different. For this reason, in the communication terminal, it is necessary to perform two different types of sound processing, one for creating minutes and the other for a conference, so that two voice processing chips are required, which increases the cost. was there.

The present invention has been made in view of the above background, and even when each communication terminal has only one sound processing unit, the minutes can be created, the audio output can be easily heard by the participants, and so on. It is an object of the present invention to provide a communication terminal and a conference system that make it possible to achieve both.

The invention according to claim 1 made to solve the above-mentioned problems can be connected to a conference server that relays at least voice between a plurality of communication terminals and a minutes server that creates minutes using the voice. In the communication terminal, if the detection unit that detects the connection with another communication terminal and the detection unit do not detect the connection, the connection destination is set to the conference server, and the detection unit detects the connection. In this case, the connection destination setting unit for setting the connection destination to the minutes server, the microphone, the first voice input to the microphone and the speech of the participant at the own base, and the other communication terminal are provided. When the connection destination is set to the minutes server for the second voice output from the speaker and input to the microphone by the participants of the other bases, the minutes are created. When the sound processing is executed and the connection destination is set to the conference server, the sound processing unit that executes the sound processing for the conference and the sound processing unit for creating the minutes or the conference When the connection destination is set to the minutes server, the first voice and the second voice after the sound processing of the above is executed are transmitted to the minutes server, and the connection destination is the connection destination. When set in the conference server, it is characterized by including a transmission unit for transmitting to the conference server.

As described above, according to the invention of claim 1, even if the communication terminal of each base has only one sound processing unit, the minutes can be created and the audio output can be easily heard by the participants. , Can be compatible.

It is a figure which shows one Embodiment of the conference system which incorporated the conference terminal as the communication terminal of this invention. It is a functional block diagram of the conference terminal shown in FIG. It is an external view of the conference terminal shown in FIG. It is a flowchart which shows the processing procedure of the conference terminal shown in FIG. It is a flowchart which shows the processing procedure of the conference terminal shown in FIG.

Hereinafter, an embodiment of the present invention will be described with reference to the drawings.

First, the configuration of the conference system will be described with reference to FIG. As shown in the figure, the conference system 1 includes a conference reservation server 11, a conference server 12, a minutes server 13, and a plurality of conference terminals 141 to 14n that communicate with these servers 11 to 13 via the Internet N. It is equipped with 15.

The conference reservation server 11 communicates with the conference terminals 141 to 14n and 15, and conference information (meeting date and time, conference participants, roles, conference terminal to be used, etc.) is input in advance from the conference terminals 141 to 14n and 15. To. Each of the conference terminals 141 to 14n and 15 makes an inquiry to the conference reservation server 11 at the time of startup, and when a corresponding conference is found, controls the conference based on the information set in advance.

The conference server 12 is a server that relays audio, video, and the like between a plurality of conference terminals 141 to 14n. Further, the conference server 12 also monitors the status of whether or not each conference terminal 141 to 14n is connected to the conference server 12, and controls the call of the conference server 12 at the start of the conference.

The minutes server 13 is composed of a voice recognition engine or the like, and creates minutes by converting the voice received from the conference terminal 15, for example, into text.

By arranging each of the plurality of conference terminals 141 to 14n in the conference room of each base and communicating with the conference server 12, the video and audio of each conference room can be shared. In the example shown in FIG. 1, the conference terminals 141 to 14n are arranged at each base and connected to the conference server 12, and the conference terminal 15 is connected to the conference terminal 141 which is one of the plurality of conference terminals 141 to 14n. It is connected to the minutes server 13.

Next, the configurations of the conference terminals 141 to 14n and 15 will be described with reference to FIGS. 2 and 3.

The conference terminals 141 to 14n and 15 of the present embodiment are each composed of a so-called interactive whiteboard. As shown in FIGS. 2 and 2, the conference terminals 141 to 14n and 15 include a camera 21, a touch panel display 22, a microphone 23, a speaker 24, a CPU 25, a storage device 26, a memory 27, and a LANI as a transmission unit. It includes a / F unit 28, an operation unit 29, and a connecting unit 30 as a detection unit.

The camera 21 has a function of acquiring surrounding images as moving images and transmitting them to the CPU 25. As shown in FIG. 3, the camera 21 is fixedly installed at a specific position of the conference terminals 141 to 14n and 15. By providing the cameras 21 in the conference terminals 141 to 14n and 15 in this way, the images of the conference participants at each base taken by the cameras 21 can be shared by the plurality of conference terminals 141 to 14n and 15. In the example shown in FIG. 3, only one camera 21 is provided, but two or more cameras may be provided so that the camera 21 to be used can be switched according to the speaker position.

The touch panel display 22 has a function of displaying an image received from the CPU 25 on the screen. The touch panel display 22 has a function of displaying the position touched by the user with a finger or a pen on the screen and transmitting the coordinates of the touched position to the CPU 25. By providing the touch panel display 22 on the conference terminals 141 to 14n and 15 in this way, the data written by the user with a pen or a finger can be shared by the plurality of conference terminals 141 to 14n and 15. As the touch panel display 22, for example, a display-integrated capacitive touch panel can be mentioned. Further, examples of the display include an LCD and an electronic paper display.

The microphone 23 has a function of acquiring the voices of the conference participants and transmitting them to the CPU 25. The microphone 23 is fixedly installed at a specific position of the conference terminals 141 to 14n and 15. In this embodiment, a plurality of microphones 23 are arranged side by side. As a result, the conference terminals 141 to 14n and 15 can detect the direction of the speaker from the voices from the plurality of microphones 23 and perform a process of removing noise from other directions. By providing the microphones 23 in the conference terminals 141 to 14n and 15 in this way, the voices of the conference participants at each base obtained by the microphones 23 can be shared by the plurality of conference terminals 141 to 14n. In the example shown in FIG. 3, a plurality of microphones 23 are provided, but only one microphone 23 may be provided.

The CPU 25 controls the conference terminals 141 to 14n and 15 as a whole. The CPU 25 has a CODEC 25A and a DSP 25B. The CODEC 25A encodes the video, drawing, and audio acquired from the camera 21, the touch panel display 22, and the microphone 23, and outputs the video, drawing, and audio to the LANI / F unit 28, which will be described later. Further, the CODEC 25A decodes the video, drawing, and audio received from the LANI / F unit 28 and outputs them to the touch panel display 22 and the speaker 24. As an example of this CODEC25A, VP8, VP9, H.I. 264 / AVC, H. 264 / SVC, H. There are 265.

The DSP 25B as an audio processing unit executes video processing of video acquired from the camera 21 and acoustic processing of audio acquired from the microphone 23.

The speaker 24 has a function of outputting the sound received from the CPU 25.

The storage device 26 stores programs such as device control and video conference control performed by the CPU 25. An example is volatile memory such as DDR memory.

The LANI / F unit 28 connects to the conference server 12 via the Internet N or the like, and transmits / receives images / sounds. Examples of the LANI / F unit 28 include a wired LAN corresponding to 10Base-T and 100Base-T and connecting to Ethernet, and a wireless LAN corresponding to 802.11a / b / g / n / ac.

The operation unit 29 has a keyboard, buttons, and the like, and the user can control the devices of the conference terminals 141 to 14n and 15.

The connecting portion 30 is composed of a connector or the like, and can be connected (connected) to other conference terminals 141 to 14n and 15. The connecting unit 30 detects the connection of the other conference terminals 141 to 14n and 15, and outputs the detection result to the CPU 25.

The conference terminals 141 to 14n and 15 having the above configuration transmit images and sounds acquired from the camera 21 and the microphone 23 to the conference server 12. The conference server 12 transmits the received images / sounds to the other conference terminals 141 to 14n and 15.

For example, when a conference is held on the conference terminals 141, 142, 143, the data transmitted by the conference terminal 141 is transmitted to the other conference terminals 142, 143 via the conference server 12, and the conference terminals 144 to 14n that do not participate in the conference. Not transmitted to 15. Similarly, the data of the conference terminals 142 and 143 are transmitted to the conference terminals 141 to 143 participating via the conference server 12, and the data is transmitted to the conference terminals 144 to 14n and 15 not participating in the conference. Not done.

By performing the above control, it is possible to hold a conference between a plurality of conference terminals (multi-site).

Next, the operation of the conference system having the above-described configuration will be described below with reference to the flowcharts of FIGS. 4 and 5. First, the CPU 25 of the conference terminals 141 to 14n (hereinafter, simply abbreviated as conference terminals 141 to 14n) executes the process shown in FIG. 4 when the power is turned on. First, the conference terminals 141 to 14n perform sound processing executed by the DSP 25B and initialization settings of the microphone 23 and the speaker 24 (step S1).

After that, the conference terminals 141 to 14n determine whether or not the connecting unit 30 has detected the connection (step S2). When the conference terminals 141 to 14n determine that the connection unit 30 has detected the connection (Y in step S2), the conference terminals 141 to 14n set themselves as the terminal 1 (main) (step S3), and then proceed to step S4. On the other hand, when the conference terminals 141 to 14n determine that the connection unit 30 has not detected the connection (N in step S2), the conference terminals 141 to 14n immediately proceed to step S4.

In step S4, the conference terminals 141 to 14n are set to output the speaker 24 at a normal volume. Further, in step S4, the conference terminals 141 to 14n turn on the echo canceller that removes the second voice that is the utterance of the participant of the other base, which is output from the speaker 24 and input to the microphone 23. Further, in step S4, the conference terminals 141 to 14n set a noise suppressor for the conference to remove peripheral noise from the voice input to the microphone 23.

After that, when the user operates the operation unit 29 to start the conference (Y in step S5), the conference terminals 141 to 14n connect to the conference server 12 (step S6). As a result, the conference terminals 141 to 14n perform the voice processing set in step S4 on the voice input by the microphone 23, and then transmit the voice to the conference server 12. Further, the conference terminals 141 to 14n receive images and sounds acquired by other conference terminals 141 to 14n via the conference server 12 and output them to the touch panel display 22 and the speaker 24.

In step S4, the echo canceller is turned on. Therefore, in step S6, the conference terminals 141 to 14n execute an echo canceller on the voice input to the microphone 23. As a result, the conference terminals 141 to 14n remove the second voice, which is the utterance of the participant of the other base, which is output from the speaker 24 and input to the microphone 23, and are directly input to the microphone 23 from the participant of the own base. Only the first voice, which is the utterance, is transmitted to the conference server 12.

In step S4, the noise suppressor is also set to be stronger for the conference. Therefore, in step S6, the conference terminals 141 to 14n remove the utterances of the participants at other bases output from the speaker 24 and input to the microphone 23 as noise, further enhancing the effect of removing the second voice. be able to. Then, since the other conference terminals 141 to 14n receive the voice on which the echo canceller or the noise suppressor with a high effect is executed and output it from the speaker 24, it becomes easy to hear the utterances of the participants at the other bases.

After that, when the conference is completed (Y in step S7), the conference terminals 141 to 14n determine whether or not the connection unit 30 has detected the disconnection (step S8).

When it is determined that the connecting unit 30 has detected the disconnection (Y in step S8), the conference terminals 141 to 14n return to step S1. On the other hand, if it is determined that the connection unit 30 has not detected the disconnection (N in step S8), the conference terminals 141 to 14n return to step S4.

On the other hand, the CPU 25 of the conference terminal 15 (hereinafter, simply abbreviated as the conference terminal) executes the process shown in FIG. 5 when the power is turned on. First, the conference terminal 15 performs acoustic processing executed by the DSP 25B and initialization settings of the microphone 23 and the speaker 24 (step S10).

After that, the conference terminal 15 determines whether or not the connection unit 30 has detected the connection (step S11). When the conference terminal 15 determines that the connection unit 30 has detected the connection (Y in step S11), the conference terminal 15 sets itself as the terminal 2 (sub) (step S12), and then proceeds to step S13.

In step S13, the conference terminal 15 is set to mute so that no sound is output from the speaker 24. In step S13, the conference terminal 15 turns off the echo canceller. Further, in step S13, the conference terminal 15 sets a noise suppressor that removes ambient noise from the voice input to the microphone 23 for creating minutes.

After that, when the user operates the operation unit 29 to start the conference (Y in step S14), the conference terminal 15 functions as a connection setting unit and connects to the minutes server 13 (step S15). As a result, the conference terminal 15 performs the voice processing set in step S13 on the voice input by the microphone 23, and then transmits the voice to the minutes server 13.

In step 13, the echo canceller is turned off. Therefore, in step S15, the conference terminal 15 does not execute the echo canceller on the voice input to the microphone 23. As a result, the conference terminal 15 does not remove the second voice, which is the utterance of the participant of the other base, which is output from the speaker 24 of the connected conference terminal 141 and input to the microphone 23, and the participant of the own base does not remove the second voice. The first voice and the second voice, which are utterances directly input to the microphone 23, are transmitted to the minutes server 13.

In step S13, the noise suppressor is also set to be weak for creating minutes. Therefore, in step S15, the conference terminal 15 does not remove the utterances of the participants at other bases output from the speaker 24 and input to the microphone 23 as noise. As a result, the voice acquired by the plurality of conference terminals 141 to 14n is output from one conference terminal 15 to the minutes server 13.

After that, when the conference is completed (Y in step S16), the conference terminal 15 determines whether or not the connection unit 30 has detected the disconnection (step S17).

When it is determined that the connecting unit 30 has detected the disconnection (Y in step S17), the conference terminal 15 returns to step S1. On the other hand, if it is determined that the connection unit 30 has not detected the disconnection (N in step S17), the conference terminal 15 returns to step S13 and retains the setting of the acoustic processing.

On the other hand, when the conference terminal 15 determines that the connection unit 30 has not detected the connection (N in step S11), the conference terminal 15 proceeds to step S18. In step S18, the conference terminal 15 sets the speaker 24 to a normal volume, turns on the echo canceller, and sets the noise suppressor for the conference, as in step S4 shown in FIG.

After that, when the conference starts (Y in step S19), the conference terminal 15 connects to the conference server 12 (step S20). As a result, when the conference terminal 15 is not connected to the other conference terminals 141 to 14n, the first speaker of the own base input by the image or the microphone 23 is the same as the other conference terminals 141 to 14n. Only the voice is transmitted to the conference server 12.

After that, when the conference is completed (Y in step S21), the conference terminal 15 determines whether or not the connection unit 30 has detected the disconnection (step S22).

When it is determined that the connecting unit 30 has detected the disconnection (Y in step S22), the conference terminal 15 returns to step S1. On the other hand, when it is determined that the connection unit 30 has not detected the disconnection (N in step S22), the conference terminal 15 returns to step S18 and holds the setting of the acoustic processing.

According to the above-described embodiment, the conference terminal 15 is output from the first voice input to the microphone 23, which is the utterance of the participant at the own base, and the speaker 24 provided by the other connected conference terminal 141. Then, the sound processing for creating the minutes is executed for the second voice input to the microphone 23, which is the utterance of the participant of the other base. Then, the conference terminal 15 transmits the first voice and the second voice after the sound processing for creating the minutes is executed to the minutes server 13. As a result, the conference terminals 141 to 14n arranged at each base do not need to transmit the voice of the speaker at the base to the minutes server 13, and do not need to execute the acoustic processing for creating the minutes. Therefore, by simply connecting the conference terminal 15 to one of the conference terminals 141 to 14n of each base, even if the conference terminals 141 to 14n of each base have only one DSP 25B, the minutes can be created. And the audio output that is easy for the participants to hear can be achieved at the same time.

Further, according to the above-described embodiment, the conference terminal 15 sets the connection destination to the conference server 12 when the connection is not detected, and sets the connection destination to the minutes server 13 when the connection is detected. ing. Further, the conference terminal 15 executes acoustic processing for creating minutes when the destination is set to the minutes server 13, and acoustic processing for conferences when the destination is set to the conference server 12. To execute. As a result, when the conference terminal 15 is not connected to the other conference terminals 141 to 14n, the first speaker of the own base input by the image or the microphone 23 is the same as the other conference terminals 141 to 14n. Only the voice is transmitted to the conference server 12.

Further, according to the above-described embodiment, the sound processing for the conference includes an echo canceller that removes the second sound from the sound input to the microphone 23, and the sound processing for creating the minutes includes an echo canceller. do not have. As a result, it is possible to more accurately create minutes and output audio that is easy for participants to hear.

Further, according to the above-described embodiment, the acoustic processing for the conference and the acoustic processing for creating the minutes include a noise suppressor for removing ambient noise, and the noise suppressor for creating the minutes and the noise suppressor for the conference. , Are set to have different removal effects. As a result, it is possible to more accurately create minutes and output audio that is easy for participants to hear.

According to the above-described embodiment, the conference terminal 15 is capable of switching between the acoustic processing for creating minutes and the acoustic processing for conference, but the present invention is not limited to this. The conference terminal 15 may be such that only sound processing for creating minutes can be performed.

Further, according to the above-described embodiment, the conference terminals 141 to 14n perform only the acoustic processing for the conference, but the present invention is not limited to this. Similar to the conference terminal 15, the conference terminals 141 to 14n can be switched between the sound processing for creating minutes and the sound processing for conferences, and any of these conference terminals 141 to 14n is a minutes server. It may be possible to make a terminal connected to 13.

The present invention is not limited to the above embodiment. That is, it can be variously modified and carried out within a range that does not deviate from the gist of the present invention.

12 Conference server 13 Minutes server 15 Conference terminal (communication terminal)
23 Microphone 24 Speaker 25 CPU (connection setting unit)
25B DSP (acoustic processing unit)
28 Transmission unit 30 Connection unit (detection unit)
141 Conference terminals (other communication terminals located at the same location)
142-14n conference terminal (communication terminal located at another base)

Japanese Unexamined Patent Publication No. 2013-183182

Claims (4)

In a communication terminal that can connect to at least a conference server that relays voice between a plurality of communication terminals and a minutes server that creates minutes using the voice.
A detector that detects connections with other communication terminals located at your base, and
When the connection is not detected by the detection unit, the connection destination is set to the conference server, and when the detection unit detects the connection, the connection destination is set to the minutes server. ,
With Mike
With speakers
The first voice input to the microphone of the participant of the own base and the other base output from the speaker or the speaker of another communication terminal arranged at the own base and input to the microphone. When the connection destination is set to the minutes server for the second voice spoken by the participants, the sound processing for creating the minutes is executed, and the connection destination is the conference server. When set to, the sound processing unit that executes the sound processing for the conference and
The connection destination of the first voice and the second voice after the sound processing for creating the minutes or the sound processing for the conference is executed by the sound processing unit is set to the minutes server. A communication terminal comprising: a transmission unit that transmits to the minutes server, and if the connection destination is set to the conference server, transmission to the conference server. The conference acoustic processing includes an echo canceller that removes the second voice from the voice input to the microphone.
The communication terminal according to claim 1, wherein the acoustic processing for creating minutes does not include the echo canceller. The acoustic processing for the conference and the acoustic processing for creating the minutes include a noise suppressor that removes ambient noise.
The communication terminal according to claim 1 or 2 , wherein the noise suppressor for creating minutes is set to have a weaker removal effect than the noise suppressor for meetings. With the conference server
With the minutes server
The communication terminal according to claim 1 connected to the minutes server, and
With other communication terminals located at the same base as the communication terminal connected to the conference server,
A conference system including a communication terminal arranged at another base for transmitting / receiving voice to / from the other communication terminal via the conference server.

Images