JP5229014B2 - COMMUNICATION CONTROL DEVICE, COMMUNICATION CONTROL SYSTEM, COMMUNICATION CONTROL METHOD, AND COMMUNICATION CONTROL DEVICE PROGRAM - Google Patents

Description

  The present invention relates to a communication control device, a communication control system, a communication control method, and a program for the communication control device, and more specifically, a communication control device, a communication control system, and a communication control for controlling data transmission / reception in a video conference. The present invention relates to a method and a program of a communication control apparatus.

  2. Description of the Related Art Conventionally, there is a communication control device in which a plurality of communication devices at remote locations are connected to the same video conference system, as if a conference is held at the same location. In such a communication control device, when a plurality of users speak at the same time, the users may refrain from each other and stop speaking, and it may be difficult to measure timing in re-speaking. Therefore, a communication control device that makes it easy for a user to hear a speech even when a plurality of users speak at the same time is known (see, for example, Patent Document 1).

  According to the communication control device described in Patent Literature 1, call information transmitted from a plurality of communication devices is received and recorded. In the recorded call information, if there are simultaneous utterances that overlap in time, the priority of the timing for outputting the call information is set. Then, the call information is shifted in time based on the set priority order and output to the communication device connected to the video conference system. Thereby, even if it is a case where a some user speaks simultaneously, the user can hear each remark easily and can aim at comfortable communication.

JP 2002-232576 A

  However, in the communication control device of the invention described in Patent Document 1, when simultaneous utterances in which the user's utterances overlap in time are performed, the call information of each user is delayed in time according to the priority. However, since they are played sequentially, there is a problem that the conversation is played with a delay and a smooth conversation is not possible.

  Therefore, the present invention has been made to solve the above-described problems, and includes a communication control device, a communication control system, a communication control method, and a program for the communication control device that can promote a smooth conversation when simultaneously speaking. The purpose is to provide.

  To achieve the above object, a communication control apparatus according to a first aspect of the present invention is a communication control apparatus that controls data communication between a plurality of terminal apparatuses via a network, and is transmitted from the terminal apparatus. The first utterance detecting means for detecting the utterance of the first user of the terminal device based on the received data and the terminal device different from the terminal device based on the data transmitted from the terminal device different from the terminal device. A second utterance detecting means for detecting the utterance of the second user of the terminal device, and the second utterance detecting means during the utterance of the first user whose utterance is detected by the first utterance detecting means. Simultaneous speech recognizing means for recognizing simultaneous speech when two user's utterances are detected, and after the simultaneous speech recognizing means recognizes simultaneous speech, the first user and the second user are both Separate from the simultaneous stop detection means for detecting a simultaneous stop that is in a stopped state, the terminal device in which the simultaneous stop is detected when the simultaneous stop is detected by the simultaneous stop detection means, and the terminal device A priority terminal determination unit that determines one of the terminal devices as a priority terminal that is a priority terminal device for instructing the user to be notified of the utterance, and the priority terminal determination unit. When the priority terminal is determined, there is provided first notification instruction means for instructing the priority terminal to notify the user to utter.

  According to a second aspect of the present invention, in addition to the configuration of the first aspect of the invention, the communication control device measures time when the first utterance detecting unit detects the utterance of the first user. Whether the time measured by the measuring means has reached a first predetermined time when the second user's utterance is detected by the measurement starting means for causing the means to start measuring time and the second utterance detecting means Measuring time judging means for judging whether or not, wherein the simultaneous utterance recognizing means includes the second utterance detecting means during the utterance of the first user whose utterance is detected by the first utterance detecting means. When a user's utterance is detected and it is determined by the measurement time determination means that the first predetermined time has not been reached, it is recognized as simultaneous speech.

  According to a third aspect of the present invention, there is provided the communication control device according to the second aspect of the invention, wherein the simultaneous stop detecting unit is configured to wait until the time measured by the measuring unit reaches a second predetermined time. In addition, the state in which both the first user and the second user are stopped is detected as the simultaneous stop.

  According to a fourth aspect of the present invention, in addition to the configuration of the first aspect of the present invention, the priority terminal determining unit is configured to detect the first user's utterance by the first utterance detecting unit. The terminal device in which an utterance is detected is determined as the priority terminal.

  In addition to the configuration of the invention according to any one of claims 1 to 3, the communication control device according to the invention according to claim 5 is a priority order of the terminal device for instructing the user to utter the speech. And priority order storage means for storing the identification information for identifying the terminal device in association with each other, wherein the priority terminal determination means is the terminal device in which the simultaneous stop is detected by the simultaneous stop detection means. The terminal device specified by the identification information corresponding to the highest priority stored in the priority storage means is determined as the priority terminal.

  A communication control system according to a sixth aspect of the present invention includes the communication control device according to any one of the first to fifth aspects, and the terminal device connected to the communication control device via a network. In the communication control system, the terminal device prompts the user to speak when there is a notification instruction by a notification unit that notifies the user and the first notification instruction unit of the communication control device. First notification control means for informing the notification means to the effect.

  According to a seventh aspect of the present invention, in addition to the configuration of the sixth aspect of the invention, the communication control device is configured such that when the priority terminal is determined by the priority terminal determination means, the priority control unit Second notification instruction means for instructing the terminal device other than the terminal to notify the user not to speak is provided, and the terminal device includes the second notification instruction means of the communication control device. And a second notification control means for notifying the notification means to urge the user not to speak when the notification instruction is issued.

  In addition to the configuration of the invention according to claim 6 or 7, the communication control system of the invention according to claim 8 is a display screen for displaying content to be notified to the user, or content to be notified to the user. It is a speaker which outputs by voice.

  The communication control method of the invention according to claim 9 is a communication control method for controlling data communication between a plurality of terminal devices via a network, based on data transmitted from the terminal device, Based on the first utterance detection step of detecting the utterance of the first user of the terminal device and the data transmitted from the terminal device different from the terminal device, the first utterance of the terminal device different from the terminal device. A second utterance detection step for detecting the utterances of two users, and the utterance of the second user in the second utterance detection step during the utterance of the first user whose utterances are detected in the first utterance detection step And the simultaneous speech recognition step for recognizing simultaneous speech when the speech is detected, and after the simultaneous speech recognition is recognized in the simultaneous speech recognition step, both the speech of the first user and the second user are stopped. A simultaneous stop detection step for detecting a simultaneous stop that is in a state, and when the simultaneous stop is detected in the simultaneous stop detection step, the terminal device from which the simultaneous stop is detected and the terminal device are different A priority terminal determination step of determining any one of the terminal devices as a priority terminal that is the terminal device to be prioritized in order to instruct the user to be notified of the utterance, and the priority terminal determination step. A first notification instruction step of instructing the priority terminal to notify the user that the user is uttered when the priority terminal is determined.

  According to a tenth aspect of the present invention, a program for a communication control apparatus causes a computer to function as various processing means of the communication control apparatus according to any one of the first to fifth aspects.

  In the communication control device according to the first aspect, the first utterance detecting means detects the utterance of the first user, and the second utterance detecting means detects the utterance of the second user. And after the 1st user and the 2nd user's utterance are recognized as a simultaneous utterance by the simultaneous utterance recognition means, in the state where both the 1st user and the 2nd user's utterance stopped by the simultaneous stop detection means. When a certain simultaneous stop is detected, the priority terminal determination unit determines any one as a priority terminal for instructing the user to notify the user of speech. Then, the determined priority terminal is instructed by the first notification instruction means to prompt the user to speak. Thereby, during the data communication, when the utterances of a plurality of users occur in the near state at the same time, and when the utterances of the plurality of users stop in the close state at the same time, an instruction for prompting the user to speak is given to the priority device. The user of the priority terminal does not need to consider whether or not to continue his / her speech. As a result, even if simultaneous speech and simultaneous stop occur, the user can thereafter have a smooth conversation.

  In the communication control apparatus according to claim 2, in addition to the effect of the invention according to claim 1, when the second user's utterance is detected by the second utterance detection means after the measurement is started by the measurement start means. Whether or not the first predetermined time has been reached is determined by the measurement time determining means. The simultaneous speech recognition means recognizes the simultaneous speech when the speech of the second user is detected during the speech of the first user and the measurement time determination means determines that the first predetermined time has not been reached. To do. Thereby, it is recognized as simultaneous utterance only when a plurality of utterances are started in the first predetermined time period. As a result, even if there are utterances from a plurality of users during a time period longer than necessary, simultaneous utterances are not recognized.

  In the communication control apparatus according to claim 3, in addition to the effect of the invention according to claim 2, the simultaneous stop detection unit is configured to receive the first user before the time measured by the measurement unit reaches the second predetermined time. And the state where both the 2nd user's utterances stopped is detected as a simultaneous stop. Thereby, simultaneous speech is detected, and it is recognized that simultaneous speech is stopped only when all speeches are stopped in the second predetermined time period. As a result, simultaneous speech is detected and simultaneous stop is not recognized after an unnecessarily long time has elapsed.

  In the communication control device according to claim 4, in addition to the effects of the invention according to any one of claims 1 to 3, the priority terminal determination unit is a terminal device in which the utterance of the first user is detected by the first utterance detection unit As a priority terminal. Thereby, even if it is simultaneous utterance, it becomes possible for the user who uttered earlier to give priority. As a result, a smooth conversation can be urged to the user.

  In the communication control apparatus according to the fifth aspect, in addition to the effect of the invention according to any one of the first to third aspects, the priority terminal is determined based on the priority order stored in the priority order storage means. Thereby, when simultaneous speech and simultaneous stop occur, since the priority is determined in advance, it is possible to easily determine the priority terminal, and thus it is possible to prompt the user to have a smooth conversation.

  A communication control system according to a sixth aspect includes the communication control device according to any one of the first to fifth aspects, and the terminal device connected to the communication control device via a network. The terminal device includes first notification control means for informing the notification means that the user is prompted to speak when the notification instruction is given by the first notification instruction means of the communication control device. Thereby, when simultaneous speech and simultaneous stop occur, the user of the priority terminal can know whether or not to continue his / her speech with the notification means. As a result, a smooth conversation can be realized.

  In the communication control system according to claim 7, in addition to the effect of the invention according to claim 6, the communication control device notifies the terminal device other than the priority terminal that the user is not allowed to speak. 2nd notification instruction means for instructing. The terminal device further includes second notification control means for notifying the notification means that the user is not allowed to speak when the notification instruction is given by the second notification instruction means of the communication control device. Yes. Thereby, when simultaneous speech and simultaneous stop occur, the user of the terminal device other than the priority terminal can know from the notification means that he / she should not speak. As a result, a smooth conversation can be realized.

  In the communication control system according to the eighth aspect, in addition to the effect of the invention according to the sixth or seventh aspect, the notifying means outputs a display screen for displaying the content to be notified to the user or the content to be notified to the user by voice. It is a speaker. Thereby, when simultaneous speech and simultaneous stop occur, the user of the terminal device knows whether or not he / she can continue his / her speech by the content displayed on the screen or the sound output from the speaker. be able to. As a result, a smooth conversation can be realized.

  In the communication control method according to the ninth aspect, the first user's speech is detected in the first speech detection step, and the second user's speech is detected in the second speech detection step. And after the 1st user and the 2nd user's utterance are recognized as simultaneous utterance in the simultaneous utterance recognition step, in the state where both the 1st user and the 2nd user's utterance stopped in the simultaneous stop detection step. When a certain simultaneous stop is detected, at the priority terminal determination step, one of them is determined as a priority terminal for instructing the user to notify the user of speech. Then, the determined priority terminal is instructed to prompt the user to speak in the first notification instruction step. As a result, when a plurality of users 'utterances occur at the same time during data communication, and when a plurality of users' utterances stop at the same time, an instruction to prompt the user to speak is given to the priority terminal. The user of the priority terminal does not need to consider whether or not to continue his / her speech. As a result, even if simultaneous speech and simultaneous stop occur, the user can thereafter have a smooth conversation.

  The program of the communication control apparatus according to claim 10 causes the computer to function as various processing means of the communication control apparatus according to any one of claims 1 to 5. Therefore, the effect of the invention according to any one of claims 1 to 5 can be achieved by causing a computer to execute the program of the communication control device.

It is a figure which shows an example of the connection form of the communication control system. 2 is a block diagram showing an electrical configuration of a communication control apparatus 100. FIG. 3 is a schematic diagram showing a configuration of a priority table 1100. FIG. 3 is a block diagram showing an electrical configuration of a terminal device 200. FIG. It is a figure which shows one specific example of the video conference screen 271 displayed on the monitor 270 of the terminal device 200. 4 is a flowchart of main processing executed by the communication control apparatus 100. 3 is a flowchart of an utterance control process subroutine executed by the communication control apparatus 100. It is a timing chart figure which shows an example of audio | voice data. It is a timing chart figure which shows an example of audio | voice data. It is a timing chart figure which shows an example of audio | voice data. 4 is a flowchart of main processing executed by the terminal device 200. It is a figure which shows one specific example of the video conference screen 271 displayed on the monitor 270 of the terminal device 200. It is a figure which shows one specific example of the video conference screen 271 displayed on the monitor 270 of the terminal device 200.

  Hereinafter, a communication control system 1 according to an embodiment of the present invention will be described with reference to the drawings. First, the outline of the communication control system 1 will be described with reference to FIG.

  As shown in FIG. 1, a plurality of terminal devices 200 are connected to the communication control device 100 via a network 2. Then, the communication control device 100 controls data communication between the terminal devices 200. In the present embodiment, it is assumed that the communication control apparatus 100 performs a video conference by controlling transmission / reception of audio data and image data. In FIG. 1, two terminal devices 200 are illustrated, but actually, it is sufficient that a video conference can be performed via the network 2, and two or more terminal devices may be used. For example, a plurality of terminal devices 200 may exist in the same site of a certain company, or may exist in different offices or in different regions or countries.

  The communication control device 100 is, for example, a well-known personal computer and is a general-purpose device. And the video conference between the terminal devices 200 is implement | achieved by relaying the image data, audio | voice data, etc. which were transmitted from the several terminal device 200. FIG. The communication control device 100 creates image data for display to be shared in a video conference by combining individual image data transmitted from each terminal device 200. And it has the function to transmit audio | voice data and the synthesized image data with respect to the some terminal device 200 connected to the video conference.

  The terminal device 200 is, for example, a known personal computer and is a general-purpose device. The terminal device 200 includes a camera 250 (see FIG. 4) for inputting an image for use in a video conference from the outside, and a microphone 240 (see FIG. 4) for inputting the audio of a video conference participant from the outside. 4). In addition, a monitor 270 (see FIG. 4) for outputting image data received from the communication control apparatus 100 to the outside and a speaker 260 (see FIG. 4) for outputting audio data to the outside are provided.

  Next, the electrical configuration of the communication control apparatus 100 will be described with reference to the block diagram of FIG. As illustrated in FIG. 2, the communication control apparatus 100 includes a CPU 101 that controls the communication control apparatus 100. A ROM 102, a RAM 103, a hard disk drive (HDD) 104, a counter 105, a display control unit 106, an input control unit 107, a timing device 108, and a communication control unit 109 are connected to the CPU 101 via a bus 111. .

  The ROM 102 stores a program for starting the BIOS executed by the CPU 101 and setting values. Various data are temporarily stored in the RAM 103. The HDD 104 stores various programs executed by the communication control apparatus 100. Also, the HDD 104 is provided with a priority table storage area 1401 in which a priority table 1100 (see FIG. 3) is stored. Details of the priority table 1100 will be described later. The counter 105 functions as a timer and measures time.

  The display control unit 106 is connected to a monitor 120 for displaying image data, and controls the display of the monitor 120. The input control unit 107 is connected to a keyboard 130 and a mouse 140 for a user to input an operation, and controls these inputs. In the time measuring device 108, time is measured as an internal clock. The communication control unit 109 controls data transmission / reception with the terminal device 200 via the network 2.

  Here, the priority table 1100 will be described with reference to FIG. In the priority order table 1100, terminal IDs and priority orders are stored in association with each other. Information for identifying the terminal device 200 is stored in the terminal ID. In the present embodiment, as an example, the terminal ID is a Mac address of the terminal device 200. The terminal ID may be unique information that can identify the terminal device 200, and may be the IP address of the terminal device 200 or the like.

  In the priority order, it is determined that speech data transmitted from a plurality of terminal devices 200 is simultaneously received (hereinafter referred to as “simultaneous speech”), and reception of the speech data is simultaneously stopped (hereinafter referred to as “simultaneous stop”). If it is determined that the terminal device 200 is prioritized in order to transmit the information for prompting the utterance, it is stored. Details of the simultaneous speech and simultaneous stop will be described later.

  Next, the electrical configuration of the terminal device 200 will be described with reference to the block diagram of FIG. As illustrated in FIG. 4, the terminal device 200 includes a CPU 201 that controls the terminal device 200. A ROM 202, a RAM 203, a hard disk drive (HDD) 204, a counter 205, a timing device 206, an input / output control unit 207, and a communication control unit 208 are connected to the CPU 201 via a bus 211.

  The ROM 202 stores a program for starting the BIOS executed by the CPU 201 and setting values. Various data are temporarily stored in the RAM 203. The HDD 204 stores various programs executed by the terminal device 200. The counter 205 functions as a timer and measures time. The time measuring device 206 measures time as an internal clock.

  The input / output control unit 207 includes a keyboard 220 for a user to input an operation, a mouse 230, a microphone 240 for acquiring voice data based on the user's speech, and a camera 250 for acquiring image data for use in a video conference. Is connected. The input / output control unit 207 also includes a speaker 260 that outputs audio data transmitted from the communication control device 100 and a monitor that displays image data transmitted from the communication control device 100 when a video conference is being performed. 270 is connected. The communication control unit 208 controls data transmission / reception with an external device such as the communication control apparatus 100 via the network 2.

  Next, the video conference screen 271 displayed on the monitor 270 of the terminal device 200 will be described with reference to FIG.

  When the video conference is executed, a video conference screen 271 is displayed on the monitor 270 of each terminal device 200 as shown in FIG. In the video conference screen 271, an individual area 272 for displaying image data transmitted from each terminal device 200 to the communication control device 100 is provided for each terminal device 200. The image data displayed in the individual area 272 is image data captured by the camera 250 of each terminal device 200. In the present embodiment, an individual area 272 corresponding to the terminal device 200 identified by the terminal ID “terminal A” is provided on the left side of the video conference screen 271. An individual area 272 corresponding to the terminal device 200 identified by the terminal ID “terminal B” is provided on the right side of the video conference screen 271.

  Next, with reference to FIG. 6, main processing executed by the communication control apparatus 100 will be described. Here, the main process of FIG. 6 is executed when the power supply of the communication control apparatus 100 is turned on. Note that when the communication control device 100 is powered off, the main process is terminated. Further, it is assumed that other processes executed by the communication control apparatus 100 are executed by other processes, and detailed description thereof is omitted.

  When the main process is executed, it is determined whether or not the user's utterance has been detected (S11). This determination process is determined based on whether or not the voice data transmitted from the terminal device 200 includes voice data based on the user's utterance. That is, it is determined whether or not the received audio data is above a certain level and the frequency is in a predetermined frequency region. Here, as an example, the predetermined frequency region is 80 Hz to 1500 Hz, which is a frequency region used in Japanese.

  When the user's utterance is not detected (S11: NO), that is, when it is equal to or higher than a certain level and voice data in a predetermined frequency region is not received, the process returns to S11 to detect voice data based on the user's utterance. Until it is done, the determination process of S11 is performed.

  When the user's utterance is detected (S11: YES), that is, when the voice data of a predetermined frequency range and above is received, the timer is initialized to “0” and the count is started (S12). ). This count start process is performed using the counter 105 which is a timer counter. The counter 105 is counted up based on a signal from a timing device 108 connected to the CPU 101.

  Next, it is determined whether or not the conversation of the speaker has been stopped (S13). That is, it is determined whether or not reception of the audio data detected in S11 is interrupted.

  If it is determined that the conversation of the speaker has been stopped (S13: YES), that is, if it is determined that the reception of the audio data detected in S11 has been interrupted, the process returns to S11 and the audio data from the user's utterance is again obtained. To detect.

  If it is determined that the conversation of the speaker is not stopped (S13: NO), that is, if it is determined that the reception of the voice data detected in S11 is not interrupted, whether or not the simultaneous conversation is detected. Is determined (S14). That is, the determination is made based on whether or not the voice data based on the user's utterance has been received from the terminal device 200 other than the terminal device 200 that has transmitted the voice data detected in S11.

  When it is determined that the simultaneous conversation is not detected (S14: NO), that is, the voice data based on the user's utterance is not received from the terminal device 200 other than the terminal device 200 that has transmitted the voice data detected in S11. If it is determined, the process returns to S13. Thereby, the processes of S13 and S14 are repeated until the conversation of the speaker is stopped or the simultaneous conversation is detected.

  When it is determined that the simultaneous conversation is detected (S14: YES), that is, it is determined that the voice data based on the user's utterance has been received from the terminal device 200 other than the terminal device 200 that has transmitted the voice data detected in S11. In this case, it is determined whether or not the measurement time is shorter than the predetermined time “T1” (S15). This measurement time is a measurement value of the counter 105 that has started counting in S12. Further, the “T1” time only needs to be determined that the audio data transmitted from the plurality of terminal devices 200 has been received at the same time, and is set to “500 milliseconds” in the present embodiment as an example.

  When it is determined that the measured time is shorter than the “T1” time (S15: YES), that is, when the measured value of the counter 105 is shorter than 500 milliseconds, it is determined whether or not all conversations are stopped (S16). ). That is, it is determined by whether or not the user's utterance of the terminal device 200 performing the simultaneous conversation is stopped and reception of all audio data is stopped.

  When it is determined that the measurement time is equal to or longer than “T1” time (S15: NO), that is, when the measurement value of the counter 105 is 500 milliseconds or more, the simultaneous conversation is not recognized, and the process returns to S11. The voice data from the utterance is detected again.

  When it is determined that all conversations are not stopped (S16: NO), that is, when the user's utterance of the terminal device 200 performing the simultaneous conversation is not stopped and reception of the voice data is continuously performed. The determination process of S16 is performed until all conversations are stopped.

  When it is determined that all conversations have been stopped (S16: YES), that is, when the user's utterance of the terminal device 200 performing simultaneous conversation is stopped and reception of all voice data is stopped, the measurement time It is determined whether or not the time is shorter than the predetermined time “T2” (S17). The “T2” time only needs to be determined that the voice data transmitted from the plurality of terminal devices 200 are simultaneously stopped by the plurality of terminal devices 200. As an example, “T2” time is “2 seconds”.

  If it is not determined that the measurement time is shorter than the “T2” time (S17: NO), that is, if the measurement time is equal to or longer than the “T2” time, the process returns to S11. If it is determined that the measurement time is shorter than the “T2” time (S17: YES), an utterance control process (see FIG. 7) is performed (S18), and the process returns to S11.

  Next, the utterance control process will be described with reference to FIG. This speech control process is a subroutine of the process executed in S18 of the main process in FIG.

  When the speech control process is executed, the priority table 1100 stored in the priority table storage area 1401 of the HDD 104 is acquired (S21). Next, a priority terminal is determined based on the priority order table 1100 (S22). The priority terminal is a terminal device 200 for transmitting information for prompting utterance in the terminal device 200 that has transmitted voice data when simultaneous speech and simultaneous stop are detected. Here, as an example, in the priority order table 1100 as shown in FIG. 3, the terminal device 200 identified by “terminal A” whose terminal ID is “1” is determined as the priority terminal. . If the user of the terminal device 200 identified by the terminal ID with the priority “1” is not performing simultaneous speech and simultaneous stop, the terminal device 200 with the priority “2” or higher is prioritized. Candidate for terminal.

  Next, a signal to urge the user to speak is transmitted to the terminal device 200, which is a priority terminal, as a speech control signal (S23). Here, specifically, a value of “1” is set in the speech control signal and transmitted to the priority terminal. Next, a signal to urge the user not to speak is transmitted as a speech control signal to the terminal device 200 other than the priority terminal (S24). Here, specifically, a value of “0” is set in the utterance control signal and transmitted to other terminal devices 200 other than the priority terminal. Then, the speech control process ends, and the process returns to the main process (see FIG. 6).

  Next, the operation of the main process described above will be described with reference to the timing chart of FIG. 8 as an example. FIG. 8 shows temporal transitions of voice data (“A1”, “A2”, and “B1”) by user's utterances received from each terminal device 200 connected to the communication control device 100.

  Here, as shown in FIG. 3, the priority order stored in the priority order table 1100 is that the priority order value of the terminal device 200 that has transmitted the voice data (A1) and the voice data (A2) is “1”. The priority value of the terminal device 200 that has transmitted the audio data (B1) is “2”.

  First, reception of audio data (A1) is started at the timing of time tA1. And reception of audio | voice data (B1) is started at the timing of time tB1. Time determination processing is performed at this timing, and it is determined whether or not the time from the time tA1 to the time tB1 is shorter than the “T1” time which is a predetermined time. Here, since the time from the time tA1 to the time tB1 is shorter than the “T1” time, it is recognized as simultaneous speech.

  Next, reception of the audio data (A1) is stopped at the timing of time t1. And reception of audio | voice data (B1) stops at the timing of the time t2. Time determination processing is performed at this timing, and it is determined whether or not the time from time tA1 to time t2 is shorter than the “T2” time which is a predetermined time. Here, since the time from the time tA1 to the time t2 is shorter than the “T2” time, it is recognized as a simultaneous stop. Then, based on the priority table 1100, the terminal device 200 corresponding to the priority “1” is determined as the priority terminal, and a signal to urge the priority terminal to speak is transmitted as the speech control signal. For example, in the priority order table 1100 illustrated in FIG. 3, the terminal device 200 identified by the terminal ID “terminal A” is the priority terminal.

  Then, voice data is transmitted from the terminal device 200 identified by the terminal ID “terminal A”, and reception of the voice data (A2) is started at the timing of time tA2.

  In the above-described main process, it is determined whether or not the measurement time is shorter than the “T1” time which is a predetermined time (see FIG. 6: S15). The timing is the timing at time tB1 in the timing chart described above. . Further, in the main process described above, it is determined whether or not the measurement time is shorter than the “T2” time which is a predetermined time (see FIG. 6: S17). The timing is the time t2 in the timing chart described above. .

  Next, the operation of the main process described above will be described with reference to the timing chart of FIG. 9 as an example. FIG. 9 shows a temporal transition of voice data (“A3”, “A4”, and “B2”) by user's utterances received from each terminal device 200 connected to the communication control device 100.

  Here, as shown in FIG. 3, the priority order stored in the priority order table 1100 is that the priority order value of the terminal device 200 that has transmitted the voice data (A3) and the voice data (A4) is “1”. The priority value of the terminal device 200 that has transmitted the audio data (B2) is “2”.

  First, reception of audio data (A3) is started at the timing of time tA3. Then, reception of audio data (B2) is started at the timing of time tB2. Time determination processing is performed at this timing, and it is determined whether or not the time from the time tA3 to the time tB2 is shorter than the “T1” time which is a predetermined time. Here, since the time from the time tA3 to the time tB2 is shorter than the “T1” time, it is recognized as simultaneous speech.

  Next, reception of the audio data (B2) is stopped at the timing of time t3. And reception of audio | voice data (A3) stops at the timing of the time t4. Time determination processing is performed at this timing, and it is determined whether or not the time from time tA3 to time t4 is shorter than the “T2” time which is a predetermined time. Here, since the time from the time tA3 to the time t4 is shorter than the “T2” time, it is recognized as a simultaneous stop. Then, based on the priority table 1100, the terminal device 200 corresponding to the priority “1” is determined as the priority terminal, and a signal to urge the priority terminal to speak is transmitted as the speech control signal. For example, in the priority order table 1100 illustrated in FIG. 3, the terminal device 200 identified by the terminal ID “terminal A” is the priority terminal.

  Then, the voice data is transmitted from the terminal device 200 identified by the terminal ID “terminal A”, and reception of the voice data (A4) is started at the timing of time tA4.

  In the above-described main process, it is determined whether or not the measurement time is shorter than the “T1” time which is a predetermined time (see FIG. 6: S15). The timing is the time tB2 in the timing chart described above. . In the main process described above, it is determined whether or not the measurement time is shorter than the “T2” time which is a predetermined time (see FIG. 6: S17). The timing is the time t4 in the timing chart described above. .

  Next, the operation of the main process described above will be described with reference to the timing chart of FIG. 10 as an example. FIG. 10 shows a temporal transition of voice data (“A5” and “B3”) received from each terminal apparatus 200 connected to the communication control apparatus 100 and uttered by the user.

  Here, as shown in FIG. 3, as the priority order stored in the priority order table 1100, the priority order value of the terminal device 200 that has transmitted the voice data (A5) is “1”, and the voice data (B3). The priority value of the terminal device 200 that has transmitted “2” is “2”.

  First, reception of audio data (A5) is started at the timing of time tA5. And reception of audio | voice data (B3) is started at the timing of time tB3. Time determination processing is performed at this timing, and it is determined whether or not the time from time tA5 to time tB3 is shorter than a predetermined time “T1”. Here, since the time from the time tA5 to the time tB3 is shorter than the “T1” time, it is recognized as simultaneous speech.

  Next, reception of the audio data (A5) is stopped at the timing of time t5. And reception of audio | voice data (B3) stops at the timing of the time t6. Time determination processing is performed at this timing, and it is determined whether or not the time from time tA5 to time t6 is shorter than the “T2” time which is a predetermined time. Here, since the time from the time tA5 to the time t6 is “T2” time or more, it is not recognized as a simultaneous stop.

  In the main process described above, it is determined whether or not the measurement time is shorter than the “T1” time, which is a predetermined time (see FIG. 6: S15). The timing is the time tB3 in the timing chart described above. . In the above-described main process, it is determined whether or not the measurement time is shorter than the “T2” time which is a predetermined time (see FIG. 6: S17). The timing is the time t6 in the timing chart described above. .

  Next, a main process executed by the terminal device 200 will be described with reference to FIGS. Here, the main process of FIG. 11 is executed when the power of the terminal device 200 is turned on. Note that when the power of the terminal device 200 is turned off, the main process is terminated. Further, it is assumed that other processes executed by the terminal device 200 are executed by other processes, and detailed description thereof is omitted.

  When the main process is executed, it is determined whether or not an utterance control signal has been received (S31). This utterance control signal is a signal transmitted from the communication control apparatus 100 in S23 or S24 of the utterance control process (see FIG. 7) of the communication control apparatus 100. If it is determined that the speech control signal has not been received (S31: NO), the process returns to S31 and the determination process is repeated. That is, it becomes a state waiting for reception of the speech control signal.

  If it is determined that an utterance control signal has been received (S31: YES), it is determined whether or not the received utterance control signal is a signal that prompts the user to speak (S32). That is, it is determined whether or not the value of the speech control signal is “1”.

  When it is determined that the received utterance control signal is a signal for prompting the user to speak (S32: YES), that is, when the value of the utterance control signal is “1”, the user is uttered. Information for prompting is displayed in the individual area 272 corresponding to the terminal device of the video conference screen 271 (S33). Here, as an example, as shown in FIG. 12, “Please continue to speak” is displayed in the individual area 272 corresponding to the terminal device 200 identified by “terminal A” which is the terminal ID having a high priority. . Then, the process returns to S31.

  When it is determined that the received utterance control signal is not a signal for prompting the user to speak (S32: NO), when the value of the utterance control signal is not "1" but "0", Then, information indicating that the user is not allowed to speak is displayed in the individual area 272 corresponding to the terminal device 200 of the video conference screen 271 (S34). Here, as an example, as shown in FIG. 13, “Please wait for remarks” is displayed in the individual area 272 corresponding to the terminal device 200 identified by “terminal B” which is the terminal ID having a low priority. . Then, the process returns to S31.

  As described above, in the present embodiment, the communication control apparatus 100 determines a priority terminal based on the priority order table 1100 when detecting simultaneous speech and simultaneous stop from users of a plurality of terminal apparatuses 200. Then, an utterance control signal, which is a signal for prompting utterance, is transmitted to the terminal device 200 which is a priority terminal. Also, an utterance control signal, which is a signal that prompts the terminal device 200 other than the priority terminal to not utter, is transmitted. In the terminal device 200, based on the value of the utterance control signal, information for prompting speech or information for prompting not to speak is displayed in the individual area 272 corresponding to its own terminal device 200.

  Thereby, during the data communication, when a plurality of users speak simultaneously, and when the simultaneous stop occurs, an instruction to prompt the user to speak to each terminal device 200 or an instruction to prompt the user not to speak Since this is displayed in the individual area 272 corresponding to each terminal device 200, the user does not need to consider whether or not to continue his / her speech. As a result, even if simultaneous speech and simultaneous stop occur, the user can thereafter have a smooth conversation.

  Note that the CPU 101 that executes S11: YES in FIG. 6 corresponds to the “first utterance detecting means” and the “first utterance detecting step”, and the CPU 101 that executes S14: YES in FIG. And “second utterance detection step”. The CPU 101 that executes S15 in FIG. 6 corresponds to “simultaneous speech recognition means” and “simultaneous speech recognition step”, and the CPU 101 that executes S16 in FIG. This corresponds to a “stop detection step”. 7 corresponds to “priority terminal determination means” and “priority terminal determination step”, and CPU 101 executing S23 of FIG. 7 performs “first notification instruction means” and “first notification instruction”. This corresponds to “step”.

  Further, the CPU 101 that executes S12 in FIG. 6 corresponds to “measurement start unit”, and the CPU 101 that executes S15 in FIG. 6 corresponds to “measurement time determination unit”. The time T1 corresponds to a “first predetermined time”, the time T2 corresponds to a “second predetermined time”, and the timing device 108 corresponds to a “measurement unit”.

  The terminal ID corresponds to “identification information”, and the priority table storage area 1401 of the HDD 104 corresponds to “priority storage means”. The CPU 201 that executes S33 in FIG. 10 corresponds to the “first notification control unit”, and the CPU 201 that executes S34 in FIG. 10 corresponds to the “second notification control unit”. The video conference screen 271 corresponds to a “display screen”, and the CPU 101 that executes S24 of FIG. 7 corresponds to a “second notification instruction unit”.

  In the embodiment described above, in the main process of the terminal device 200 in FIG. 11, information that prompts utterance and information that urges not to speak are displayed on the video conference screen 271 of the terminal device 200. It is not limited to. For example, the user may be notified of the information that prompts the utterance and the information that prompts the user not to speak from the speaker 260 of the terminal device 200 by voice. Thereby, it is possible to determine whether or not to continue his / her speech even in a situation where the user is not watching the video conference screen 271.

  In the above-described embodiment, the priority terminal is determined based on the priority table 1100 acquired in S21 in the process of determining the priority terminal in the speech control process of FIG. 7, but the present invention is not limited to this. For example, the communication control apparatus 100 may determine the terminal apparatus 200 that transmitted the voice data of the user's first received speech as the priority terminal. Thereby, a priority terminal can be determined easily, and even if simultaneous speech and simultaneous stop occur, the user can have a smooth conversation thereafter. There is no problem even if the priority terminal is determined using another method.

DESCRIPTION OF SYMBOLS 1 Communication control system 2 Network 100 Communication control apparatus 101,201 CPU
103, 203 RAM
104,204 HDD
200 Terminal Device 205 Counter 206 Timekeeping Device 207 Input / Output Control Unit 208 Communication Control Unit 240 Microphone 260 Speaker 270 Monitor 271 Video Conference Screen 272 Individual Area 1100 Priority Table 1401 Priority Table Storage Area

Claims (10)

A communication control device for controlling data communication between a plurality of terminal devices via a network,
First utterance detection means for detecting the utterance of the first user of the terminal device based on the data transmitted from the terminal device;
Based on data transmitted from the terminal device different from the terminal device, second utterance detection means for detecting the utterance of the second user of the terminal device different from the terminal device;
Simultaneous speech recognition means for recognizing simultaneous speech when the second user's speech is detected by the second speech detection means during speech of the first user whose speech has been detected by the first speech detection means; ,
A simultaneous stop detecting means for detecting a simultaneous stop in which both the first user and the second user are stopped after being recognized by the simultaneous speech recognition means;
When the simultaneous stop is detected by the simultaneous stop detection unit, the terminal device in which the simultaneous stop is detected and the terminal device different from the terminal device are uttered to the user Priority terminal determining means for determining as a priority terminal that is the terminal device to be prioritized in order to instruct notification of prompting,
Communication control comprising: first notification instruction means for instructing the priority terminal to notify the user that the priority terminal is determined when the priority terminal is determined by the priority terminal determination means apparatus. When the first utterance detecting unit detects the utterance of the first user, the measurement starting unit causes the measuring unit to measure time to start measuring time;
Measuring time determining means for determining whether the time measured by the measuring means has reached a first predetermined time when the second user's utterance is detected by the second utterance detecting means;
The simultaneous utterance recognition means detects the utterance of the second user by the second utterance detection means during the utterance of the first user whose utterance has been detected by the first utterance detection means, and determines the measurement time. The communication control apparatus according to claim 1, wherein when it is determined by the means that the first predetermined time has not been reached, simultaneous speech is recognized.   The simultaneous stop detecting means sets the state where both the first user and the second user have stopped speaking as the simultaneous stop until the time measured by the measuring means reaches the second predetermined time. The communication control device according to claim 2, wherein the communication control device is detected.   The said priority terminal determination means determines the said terminal device from which the said 1st user's utterance was detected by the said 1st utterance detection means as the said priority terminal. Communication control device. Priority order storage means for storing the priority order of the terminal device for instructing the user to utter the speech and the identification information for identifying the terminal device in association with each other,
The priority terminal determination means is the identification information corresponding to the highest priority stored in the priority storage means among the terminal devices detected by the simultaneous stop detection means. The communication control device according to claim 1, wherein the specified terminal device is determined as the priority terminal. A communication control system comprising the communication control device according to any one of claims 1 to 5 and the terminal device connected to the communication control device via a network,
The terminal device
An informing means for informing the user;
And a first notification control means for notifying the notification means that the user is urged to speak when the notification instruction is given by the first notification instruction means of the communication control device. Communication control system. The communication control device includes:
Second notification instruction means for instructing other terminal devices other than the priority terminal to notify the user not to speak when the priority terminal is determined by the priority terminal determination means; Prepared,
The terminal device
When there is a notification instruction by the second notification instruction unit of the communication control device, the notification unit includes a second notification control unit that notifies the notification unit that the user is not allowed to speak. The communication control system according to claim 6.   The communication control system according to claim 6 or 7, wherein the notification unit is a display screen that displays a content to be notified to the user or a speaker that outputs a content to be notified to the user by voice. A communication control method for controlling data communication between a plurality of terminal devices via a network,
A first utterance detection step of detecting an utterance of a first user of the terminal device based on data transmitted from the terminal device;
A second utterance detection step of detecting an utterance of a second user of the terminal device different from the terminal device based on data transmitted from the terminal device different from the terminal device;
Simultaneous speech recognition that recognizes simultaneous speech when the second user's speech is detected in the second speech detection step during speech of the first user whose speech was detected in the first speech detection step Steps,
A simultaneous stop detection step of detecting a simultaneous stop in which both the first user and the second user have stopped speaking after being recognized as a simultaneous speech in the simultaneous speech recognition step;
When the simultaneous stop is detected in the simultaneous stop detection step, one of the terminal device from which the simultaneous stop is detected and the terminal device different from the terminal device is sent to the user. A priority terminal determination step for determining as a priority terminal that is the terminal device to be prioritized in order to instruct notification of prompting speech;
And a first notification instruction step for instructing the priority terminal to notify the user that the user is uttered when the priority terminal is determined in the priority terminal determination step. Control method.   A program for a communication control device, which causes a computer to function as various processing means of the communication control device according to any one of claims 1 to 5.

Images