JP2019176386A - Communication terminals and conference system - Google Patents

Abstract

To provide communication terminals and a conference system, capable of achieving both of creation of a record of proceedings and output of voice to be easily understood by participants even when each communication terminal comprises only one sound processing unit.SOLUTION: A conference terminal 15, when detecting coupling with a conference terminal 141, changes over a connection destination to a record-of-proceedings server 13. The conference terminal 15 can connect with the record-of-proceedings server 13 for creating a record of proceedings using received voice. The conference terminal 15 includes a DSP 25B that executes sound processing for creating the record of proceedings on first voice, utterance input to a microphone 23 by a participant at a site thereof, and second voice, utterance output from a speaker 24 of the other conference terminal 141 and input to the microphone 23 by a participant at another site. The conference terminal 15 transmits to the record-of-proceedings server 13 the first voice and the second voice on which the sound processing has been executed by the DSP 25B.SELECTED DRAWING: Figure 1

Description

  The present invention relates to a communication terminal and a conference system.

  In recent years, conference systems that conduct conferences between remote locations via a communication network such as the Internet have become widespread.

  In this conference system, communication terminals are installed in a plurality of conference rooms that are remote from each other. Each of the communication terminals captures and collects an image of a conference room such as a party of the conference and a voice such as a speech. Then, each communication terminal converts captured and collected images and sound into digital data, and transmits the digital data to another communication terminal set in another conference room. In addition, when the communication terminal receives an image or sound from another communication terminal, the communication terminal displays an image on the display and outputs the sound on the speaker. Thus, a technique for performing a conference between remote locations in a state close to an actual conference is already known.

  In addition, a technology is already known in which a communication terminal transmits collected speech to a speech recognition engine, and the speech recognition engine automatically transcribes the content of speech from parties and the like to automatically generate minutes.

  An example of the above technique is a conference system disclosed in Patent Document 1. In the conference system of Patent Document 1, each of a plurality of conference terminals (communication terminals) transmits the audio of the conference room (local site) in which the conference terminals are installed to a recording server (voice recognition engine). Minutes are created by synthesizing the voice received from the terminal in time series.

  However, acoustic processing such as noise removal processing that is easily recognized by the speech recognition engine (machine) is different from (natural) acoustic processing that is easily heard by the person at the opposite site of the remote conference. For this reason, in the communication terminal, since it is necessary to perform two kinds of different acoustic processing for the minutes creation and for the conference, two speech processing chips are required, resulting in an increase in cost. was there.

  The present invention has been made in view of the above background, and even if each communication terminal has only one acoustic processing unit, the creation of minutes, audio output that is easy to hear for participants, An object of the present invention is to provide a communication terminal and a conference system that make it possible to satisfy both of these requirements.

  In order to solve the above-mentioned problem, the invention according to claim 1 is connectable to a conference server that relays at least voice between a plurality of communication terminals and a minutes server that creates minutes using the voice. In the communication terminal, when the connection is not detected by the detection unit detecting the connection with another communication terminal and the detection unit, a connection destination is set in the conference server, and the connection is detected by the detection unit. The connection destination setting unit for setting the connection destination in the minutes server, the microphone, the first voice that is the speech of the participant at the local site input to the microphone, and the other communication terminal When the connection destination is set in the minutes server for the second voice that is the utterance of the participant at the other site that is output from the speaker and input to the microphone, Acoustic processing And when the connection destination is set in the conference server, an audio processing unit that executes audio processing for conference, and audio processing for creating the minutes or audio processing for the conference by the audio processing unit When the connection destination is set in the minutes server, the first sound and the second sound after the execution is executed are transmitted to the minutes server, and the connection destination is sent to the conference server. A transmission unit configured to transmit to the conference server when set.

  As described above, according to the first aspect of the present invention, even if the communication terminal at each site has only one acoustic processing unit, the minutes are created and the audio output is easy for the participants to hear. , Can be compatible.

It is a figure which shows one Embodiment of the conference system incorporating the conference terminal as a communication terminal of this invention. It is a functional block diagram of the conference terminal shown in FIG. It is an external view of the conference terminal shown in FIG. It is a flowchart which shows the process sequence of the conference terminal shown in FIG. It is a flowchart which shows the process sequence of the conference terminal shown in FIG.

  Hereinafter, an embodiment of the present invention will be described with reference to the drawings.

  First, the configuration of the conference system will be described with reference to FIG. As shown in the figure, the conference system 1 includes a conference reservation server 11, a conference server 12, a minutes server 13, and a plurality of conference terminals 141 to 14n communicating with these servers 11 to 13 via the Internet N. 15.

  The conference reservation server 11 communicates with the conference terminals 141 to 14n and 15, and conference information (conference date and time, conference participants, roles, conference terminals to be used, etc.) is input from the conference terminals 141 to 14n and 15 in advance. The Each of the conference terminals 141 to 14n and 15 makes an inquiry to the conference reservation server 11 at the time of activation, and when a corresponding conference is found, controls the conference based on information set in advance.

  The conference server 12 is a server that relays audio, video, and the like between the plurality of conference terminals 141 to 14n. In addition, the conference server 12 also performs status monitoring as to whether or not each conference terminal 141 to 14n is connected to the conference server 12, and performs call control of the conference server 12 at the start of the conference.

  The minutes server 13 is composed of a voice recognition engine or the like, and creates minutes by, for example, converting the voice received from the conference terminal 15 into text.

  Each of the plurality of conference terminals 141 to 14n is arranged in a conference room at each base, and can communicate with the conference server 12 to share video, audio, and the like of each conference room. In the example shown in FIG. 1, the conference terminals 141 to 14n are arranged at the respective bases, connected to the conference server 12, and the conference terminal 15 is connected to the conference terminal 141 which is one of the plurality of conference terminals 141 to 14n. The minutes server 13 is connected.

  Next, the configuration of the conference terminals 141 to 14n and 15 will be described with reference to FIGS.

  Each of the conference terminals 141 to 14n and 15 of the present embodiment is configured by a so-called interactive whiteboard. As shown in FIG. 2 and the like, the conference terminals 141 to 14n and 15 include a camera 21, a touch panel display 22, a microphone 23, a speaker 24, a CPU 25, a storage device 26, a memory 27, and a LANI as a transmission unit. / F part 28, operation part 29, and connecting part 30 as a detection part are provided.

  The camera 21 has a function of acquiring surrounding video as a moving image and transmitting it to the CPU 25. As shown in FIG. 3, the camera 21 is fixedly installed at a specific position of the conference terminals 141 to 14 n and 15. By providing the cameras 21 in the conference terminals 141 to 14n and 15 as described above, the video of the conference participants at the respective locations taken by the camera 21 can be shared by the plurality of conference terminals 141 to 14n and 15. In the example shown in FIG. 3, only one camera 21 is provided, but two or more cameras 21 may be provided so that the camera 21 to be used can be switched according to the speaker position.

  The touch panel display 22 has a function of displaying the video received from the CPU 25 on the screen. The touch panel display 22 has a function of displaying a position touched by a user with a finger or a pen on the screen and transmitting coordinates of the touched position to the CPU 25. By providing the touch panel display 22 on the conference terminals 141 to 14n and 15 as described above, the data written by the user with a pen or a finger can be shared by the plurality of conference terminals 141 to 14n and 15. As the touch panel display 22, for example, a display-integrated capacitive touch panel can be cited. Examples of the display include an LCD and an electronic paper display.

  The microphone 23 has a function of acquiring the voice of the conference participant and transmitting it to the CPU 25. The microphone 23 is fixedly installed at a specific position of the conference terminals 141 to 14n and 15. In the present embodiment, a plurality of microphones 23 are arranged side by side. Thereby, the conference terminals 141 to 14n and 15 can detect the direction of the speaker from the voices from the plurality of microphones 23 and perform processing to remove noise from other directions. By providing the microphones 23 in the conference terminals 141 to 14n and 15 as described above, the voices of the conference participants at the respective locations obtained by the microphone 23 can be shared by the plurality of conference terminals 141 to 14n. In the example shown in FIG. 3, a plurality of microphones 23 are provided, but only one microphone 23 may be provided.

  The CPU 25 controls the conference terminals 141 to 14n and 15 as a whole. The CPU 25 has a CODEC 25A and a DSP 25B. The CODEC 25A encodes video, drawing, and audio acquired from the camera 21, touch panel display 22, and microphone 23, and outputs them to the LAN I / F unit 28 described later. The CODEC 25A decodes the video, drawing, and audio received from the LAN I / F unit 28 and outputs them to the touch panel display 22 and the speaker 24. Examples of the CODEC 25A include VP8, VP9, H.264, and the like. H.264 / AVC, H.H. H.264 / SVC, H.H. 265.

  The DSP 25 </ b> B as an acoustic processing unit executes video processing of video acquired from the camera 21 and audio processing of audio acquired from the microphone 23.

  The speaker 24 has a function of outputting sound received from the CPU 25.

  The storage device 26 stores programs such as device control and video conference control performed by the CPU 25. An example is a volatile memory such as a DDR memory.

  The LAN I / F unit 28 is connected to the conference server 12 via the Internet N or the like, and transmits and receives images and sounds. Examples of the LAN I / F unit 28 include a wired LAN connected to Ethernet corresponding to 10Base-T and 100Base-T, and a wireless LAN compatible with 802.11a / b / g / n / ac.

  The operation unit 29 includes a keyboard, buttons, and the like, and the user can control the devices of the conference terminals 141 to 14n and 15.

  The connection part 30 is comprised from a connector etc., and can connect (connect) with the other conference terminals 141-14n, 15. The connection unit 30 detects the connection of the other conference terminals 141 to 14n and 15, and outputs the detection result to the CPU 25.

  The conference terminals 141 to 14n and 15 configured as described above transmit the images and sounds acquired from the camera 21 and the microphone 23 to the conference server 12. The conference server 12 transmits the received image / sound to the other conference terminals 141 to 14n and 15.

  For example, when a conference is performed at the conference terminals 141, 142, and 143, the data transmitted by the conference terminal 141 is transmitted to the other conference terminals 142 and 143 via the conference server 12, and the conference terminals 144 to 14 n that are not participating, 15 is not transmitted. Similarly, the data of the conference terminals 142 and 143 is transmitted to the conference terminals 141 to 143 participating through the conference server 12, and the data is transmitted to the conference terminals 144 to 14n and 15 not participating in the conference. Not.

  By performing the control as described above, a conference can be performed between a plurality of conference terminals (multiple locations).

  Next, the operation of the conference system configured as described above will be described below with reference to the flowcharts of FIGS. First, the CPU 25 of the conference terminals 141 to 14n (hereinafter simply referred to as conference terminals 141 to 14n) executes the process shown in FIG. 4 when the power is turned on. First, the conference terminals 141 to 14n perform acoustic processing executed by the DSP 25B and initialization settings of the microphone 23 and the speaker 24 (step S1).

  Thereafter, the conference terminals 141 to 14n determine whether or not the connection unit 30 has detected connection (step S2). When the conference terminals 141 to 14n determine that the connection unit 30 has detected connection (Y in step S2), the conference terminals 141 to 14n set themselves as the terminal 1 (main) (step S3), and then proceed to step S4. On the other hand, when the conference terminals 141 to 14n determine that the connection unit 30 has not detected connection (N in step S2), the process immediately proceeds to step S4.

  In step S4, the conference terminals 141 to 14n set the speaker 24 to output at a normal volume. In step S <b> 4, the conference terminals 141 to 14 n turn on the echo canceller that removes the second sound that is the speech of the participant at the other base that is output from the speaker 24 and input to the microphone 23. In step S4, the conference terminals 141 to 14n set a noise suppressor for removing ambient noise from the voice input to the microphone 23 for the conference.

  Thereafter, when the user operates the operation unit 29 to start the conference (Y in step S5), the conference terminals 141 to 14n connect to the conference server 12 (step S6). Accordingly, the conference terminals 141 to 14n perform the audio processing set in step S4 on the audio input by the microphone 23 and then transmit the audio to the conference server 12. In addition, the conference terminals 141 to 14 n receive the images and sounds acquired by the other conference terminals 141 to 14 n via the conference server 12 and output them to the touch panel display 22 and the speaker 24.

  In step S4, the echo canceller is turned on. For this reason, in step S <b> 6, the conference terminals 141 to 14 n execute an echo canceller on the voice input to the microphone 23. As a result, the conference terminals 141 to 14n remove the second voice that is the speech of the participant at the other site that is output from the speaker 24 and input to the microphone 23, and is directly input to the microphone 23 from the participant at the local site. Only the first voice that is the utterance is transmitted to the conference server 12.

  In step S4, the noise suppressor is also set stronger for the conference. For this reason, in step S6, the conference terminals 141 to 14n remove the utterances of the participants at the other bases that are output from the speaker 24 and input to the microphone 23 as noise, thereby further enhancing the second voice removal effect. be able to. And since the other conference terminals 141-14n receive the audio | voice with which the echo canceller and the highly effective noise suppressor were performed, and output from the speaker 24, it becomes easy to hear the speech of the participant of another base.

  Thereafter, when the conference is ended (Y in step S7), the conference terminals 141 to 14n determine whether or not the connection unit 30 has detected disconnection (step S8).

  If it determines with the connection part 30 having detected the connection cancellation | release (it is Y at step S8), the conference terminals 141-14n will return to step S1. On the other hand, if it determines with the connection part 30 not detecting the connection cancellation | release (N by step S8), the conference terminals 141-14n will return to step S4.

  On the other hand, the CPU 25 of the conference terminal 15 (hereinafter simply referred to as a conference terminal) executes the process shown in FIG. 5 when the power is turned on. First, the conference terminal 15 performs acoustic processing executed by the DSP 25B and initialization settings for the microphone 23 and the speaker 24 (step S10).

  Thereafter, the conference terminal 15 determines whether or not the connection unit 30 has detected connection (step S11). When the conference terminal 15 determines that the connection unit 30 has detected connection (Y in step S11), the conference terminal 15 sets itself as the terminal 2 (sub) (step S12), and then proceeds to step S13.

  In step S <b> 13, the conference terminal 15 sets mute so that no sound is output from the speaker 24. In step S13, the conference terminal 15 turns off the echo canceller. In step S <b> 13, the conference terminal 15 sets a noise suppressor for removing the ambient noise from the voice input to the microphone 23 for creating the minutes.

  Thereafter, when the user starts the conference by operating the operation unit 29 (Y in step S14), the conference terminal 15 functions as a connection setting unit and connects to the minutes server 13 (step S15). Thereby, the conference terminal 15 performs the audio processing set in step S <b> 13 on the audio input by the microphone 23, and then transmits it to the minutes server 13.

  In step 13, the echo canceller is turned off. For this reason, the conference terminal 15 does not execute an echo canceller on the voice input to the microphone 23 in step S15. As a result, the conference terminal 15 does not remove the second voice, which is the utterance of the participant at the other site that is output from the speaker 24 of the connected conference terminal 141 and is input to the microphone 23, and the The first voice and the second voice, which are utterances directly input to the microphone 23, are transmitted to the minutes server 13.

  In step S13, the noise suppressor is also set to be weak for creating minutes. For this reason, in step S15, the conference terminal 15 does not remove the utterances of the participants at the other bases output from the speaker 24 and input to the microphone 23 as noise. Thereby, the voice acquired by the plurality of conference terminals 141 to 14n is output to the minutes server 13 from one conference terminal 15.

  Thereafter, when the conference ends (Y in step S16), the conference terminal 15 determines whether or not the connection unit 30 has detected release of the connection (step S17).

  If it determines with the connection part 30 having detected connection cancellation | release (it is Y at step S17), the conference terminal 15 will return to step S1. On the other hand, if it determines with the connection part 30 not detecting the connection cancellation | release (N in step S17), the conference terminal 15 will return to step S13 and hold | maintain the setting of an acoustic process.

  In contrast, if the conference terminal 15 determines that the connection unit 30 has not detected connection (N in step S11), the process proceeds to step S18. In step S18, as in step S4 shown in FIG. 4, the conference terminal 15 sets the speaker 24 to the normal volume, turns on the echo canceller, and sets the noise suppressor for the conference.

  Thereafter, when the conference starts (Y in step S19), the conference terminal 15 connects to the conference server 12 (step S20). As a result, when the conference terminal 15 is not connected to the other conference terminals 141 to 14n, the first of the speaker at the local site input by the image or the microphone 23 is input in the same manner as the other conference terminals 141 to 14n. Only audio is transmitted to the conference server 12.

  Thereafter, when the conference ends (Y in step S21), the conference terminal 15 determines whether or not the connection unit 30 has detected the release of the connection (step S22).

  If it determines with the connection part 30 having detected the connection cancellation | release (it is Y at step S22), the conference terminal 15 will return to step S1. On the other hand, if it determines with the connection part 30 not detecting the connection cancellation | release (N in step S22), the conference terminal 15 will return to step S18 and will hold | maintain the setting of an acoustic process.

  According to the above-described embodiment, the conference terminal 15 is output from the first voice that is the speech of the participant at the local site input to the microphone 23 and the speaker 24 included in the other conference terminal 141 connected. The acoustic processing for creating the minutes is executed on the second voice that is the utterance of the participant at the other base input to the microphone 23. Then, the conference terminal 15 transmits the first voice and the second voice after the acoustic processing for creating the minutes is executed to the minutes server 13. Thereby, the conference terminals 141 to 14n arranged at each site do not need to transmit the voice of the speaker at the site to the minutes server 13, and do not need to execute the acoustic processing for creating the minutes. Therefore, even if the conference terminal 15 is connected to one of the conference terminals 141 to 14n at each site and the conference terminals 141 to 14n at each site have only one DSP 25B, the minutes are created. And audio output that is easy to hear for participants.

  Further, according to the above-described embodiment, the conference terminal 15 sets the connection destination to the conference server 12 when the connection is not detected, and sets the connection destination to the minutes server 13 when the connection is detected. ing. Further, the conference terminal 15 executes acoustic processing for creating minutes when the transmission destination is set in the minutes server 13, and performs audio processing for conference when the transmission destination is set in the conference server 12. Execute. As a result, when the conference terminal 15 is not connected to the other conference terminals 141 to 14n, the first of the speaker at the local site input by the image or the microphone 23 is input in the same manner as the other conference terminals 141 to 14n. Only audio is transmitted to the conference server 12.

  Further, according to the above-described embodiment, the conference acoustic processing includes the echo canceller that removes the second speech from the speech input to the microphone 23, and the minutes creation acoustic processing includes the echo canceller. Absent. Thereby, it is possible to achieve both the creation of the minutes and the sound output that is easy for the participant to hear, with higher accuracy.

  Further, according to the above-described embodiment, the audio processing for conference and the audio processing for creating minutes include a noise suppressor that removes ambient noise, and a noise suppressor for creating minutes, a noise suppressor for conference, and Are set to have different removal effects. Thereby, it is possible to achieve both the creation of the minutes and the sound output that is easy for the participant to hear, with higher accuracy.

  According to the above-described embodiment, the conference terminal 15 can be switched between the acoustic processing for creating minutes and the acoustic processing for conference. However, the present invention is not limited to this. The conference terminal 15 may be one that can only perform acoustic processing for creating minutes.

  Moreover, according to embodiment mentioned above, although the conference terminals 141-14n performed only the acoustic process for meetings, it is not restricted to this. Similarly to the conference terminal 15, the conference terminals 141 to 14n can be switched between the acoustic processing for creating the minutes and the acoustic processing for the conference, and any of these conference terminals 141 to 14n can be switched to the minutes server. 13 may be a terminal that can be connected to the terminal 13.

  The present invention is not limited to the above embodiment. That is, various modifications can be made without departing from the scope of the present invention.

12 Conference server 13 Minutes server 15 Conference terminal (communication terminal)
23 Microphone 24 Speaker 25 CPU (Connection Setting Unit)
25B DSP (Sound Processing Unit)
28 Transmitting section 30 Connecting section (detecting section)
141 Conference terminal (other communication terminals located at the same location)
142-14n Conference terminal (communication terminal located at another base)

JP2013-183182A

Claims (4)

In a communication terminal that can be connected to a conference server that relays at least audio between a plurality of communication terminals and a minutes server that creates the minutes using the audio,
A detection unit for detecting connection with another communication terminal;
A connection destination setting unit that sets a connection destination to the conference server when the connection is not detected by the detection unit, and sets a connection destination to the minutes server when the connection is detected by the detection unit; ,
With a microphone,
The first speech that is the speech of the participant at the local site input to the microphone and the second speech that is the speech of the participant at the other site that is output from the speaker included in the other communication terminal and input to the microphone. If the connection destination is set in the minutes server, the sound processing for creating the minutes is executed, and if the connection destination is set in the conference server, An acoustic processing unit for performing the acoustic processing of
The connection destination is set in the minutes server for the first sound and the second sound after the sound processing unit performs the sound processing for creating the minutes or the sound processing for the meeting. And a transmission unit that transmits to the meeting server, and transmits to the conference server when the connection destination is set to the conference server. The audio processing for the conference includes an echo canceller that removes the second audio from the audio input to the microphone,
The communication terminal according to claim 1, wherein the acoustic processing for creating the minutes does not include the echo canceller. The audio processing for the conference and the audio processing for creating the minutes include a noise suppressor that removes ambient noise,
The communication terminal according to claim 1, wherein the noise suppressor for creating the minutes and the noise suppressor for the conference are set so as to have different removal effects. The conference server;
The minutes server;
The communication terminal according to claim 1, which is connected to the minutes server;
Other communication terminals arranged at the same base as the communication terminal connected to the conference server,
A conference system comprising: a communication terminal disposed at another site that transmits and receives audio to and from the other communication terminal via the conference server.

Images