JP2023043497A - remote conference system - Google Patents

Abstract

To provide a remote conference system that enables hearing utterance content of each speaker even when multiple participants speak at the same time during a remote conference.SOLUTION: A remote conference system 10 includes a remote conference server device 15 and remote conference client devices 11, 12, 13, 14. The remote conference server device 15 includes a seat determination unit 22. The remote conference server device 15 or remote conference client devices 11, 12, 13, 14 include sound field characteristic determination units 23 and speech synthesis units 24. The sound field characteristic determination unit 23 determines a sound field characteristic of an audio to be reproduced on each of the remote conference client devices 11, 12, 13, 14 separately for each of the remote conference client devices 11, 12, 13, 14. The speech synthesis unit 24 synthesizes the audio to be played on each of the remote conference client devices 11, 12, 13, 14 based on the sound field characteristics determined by the sound field characteristic determination unit 23.SELECTED DRAWING: Figure 2

Description

The present invention relates to a remote conference system.

Conventionally, as a remote conference system for executing a remote conference, for example, there is a multipoint video conference system as disclosed in Patent Document 1. The multipoint video conference system of Patent Document 1 includes spatial information management means, video layout control means, collaborative work state detection means, and operation input management means. Spatial information management means and image layout control means manage the image layout of the face image and the collaborative work image. The cooperative work state detection means detects the processing state of the cooperative work means. The operation input management means controls the pointing operation performed by the user on the face image area when detecting that the collaborative work means is waiting for information specifying the type of terminal as input from the user. It is converted into terminal-specified input information for collaborative work means.

JP-A-8-205112

However, in the multi-point video conference system of Patent Document 1, it is possible to produce an image of a virtual conference room by synthesizing the images of each participant according to the seat position in the virtual conference space. Since no action is applied to the voices uttered by the participants, the voice sound image of each participant concentrates on one point. Therefore, in the multi-point video conference system of Patent Document 1, unlike in an actual conference room, when a plurality of participants speak at the same time, there is a problem that it is difficult to hear the content of each participant's speech.

SUMMARY OF THE INVENTION Accordingly, it is an object of the present invention to provide a remote conference system in which even when a plurality of participants speak at the same time during a remote conference, the content of each speaker's speech can be heard.

To achieve the above object, a remote conference system according to one aspect of the present invention includes a remote conference server device and a remote conference client device. The remote conference server device hosts the remote conference. The remote conference client device is assigned to each participant joining the remote conference. The remote conference server device includes a seat determiner. The seat determination unit determines the seats of the participants in the remote conference room where the remote conference is held. The remote conference server device or the remote conference client device includes a sound field characteristic determination section and a speech synthesis section. A sound field characteristic determination unit determines sound field characteristics between the seats determined by the seat determination unit. The voice synthesizing unit synthesizes voices to be output to the participants. The sound field characteristic determining unit individually determines, for each remote conference client apparatus, sound field characteristics of audio reproduced by each of the remote conference client apparatuses. The speech synthesizing unit synthesizes speech to be reproduced by each of the remote conference client devices based on the sound field characteristics determined by the sound field characteristics determining unit.

According to the present invention, even when a plurality of participants speak at the same time, it is possible to easily hear the content of each speaker's speech.

1 is a schematic diagram showing an overall overview of a remote conference system according to an embodiment; FIG. 1 is a system configuration diagram showing an overall overview of a remote conference system according to an embodiment; FIG. FIG. 3 is a diagram showing an example of a remote conference room displayed on the display unit of the remote conference client device in the remote conference system according to the embodiment; 4 is a flow chart showing the operation flow of the remote conference server device in the remote conference system according to the present embodiment; FIG. 4 is a diagram showing an example of a remote conference room displayed on the display unit of the remote conference client device in the one-to-one interactive mode in the remote conference system according to the present embodiment;

An embodiment of the present invention will be described below with reference to the drawings. FIG. 1 is a schematic diagram showing an overall overview of a remote conference system 10 according to this embodiment. FIG. 2 is a system configuration diagram showing an overall overview of the remote conference system 10 according to this embodiment.

The remote conference system 10 is a communication conference system in which a plurality of participants A, B, C, and D perform one-way or two-way communication at remote locations. As shown in FIG. 1, participants A, B, C, and D are, for example, at their own homes, at their workplaces (specifically, at their desks at their workplaces or in conference rooms at their workplaces, etc.), or Participate in the remote conference by operating the remote conference client devices 11, 12, 13, and 14 from outside. As shown in FIG. 2, the remote conference system 10 includes a plurality of remote conference client devices 11, 12, 13, and 14 and a remote conference server device 15.

Remote conference client devices 11, 12, 13, and 14 are communication devices assigned to participants A, B, C, and D who participate in the remote conference, respectively. The remote conference client devices 11, 12, 13, and 14 are devices capable of communicating with the remote conference server device 15, such as smart phones, tablets, personal computers, and television receivers. Each of the remote conference client devices 11 , 12 , 13 and 14 mainly includes a communication unit 16 , a speaker 17 , a microphone 18 and a display section 19 .

The communication unit 16 communicates with the remote conference server device 15 . That is, the communication unit 16 is audio upload means for transmitting audio and images from each of the remote conference client devices 11, 12, 13, 14 of the participants A, B, C, D to the remote conference server device 15. .

For example, the communication unit 16 of the first remote conference client device 11 transmits voice data of the first participant A's speech formed in the first remote conference client device 11 to the remote conference server device 15 . On the other hand, the communication unit 16 of the first remote conference client device 11 receives the audio data of the utterances of the other participants B, C, D formed in the other remote conference client devices 12, 13, 14 and the remote conference server Sound field forming data formed in the device 15 is received from the remote conference server device 15 . The communication unit 16 outputs the received audio data and sound field forming data to the speaker 17 . The sound field forming data is data for determining sound field characteristics in the remote conference room 30 (characteristics of voices transmitted between participants A, B, C, and D in the remote conference room 30). .

Similarly, the communication unit 16 of the first remote conference client device 11 transmits image data such as the image of the first participant A formed on the first remote conference client device 11 to the remote conference server device 15 . The communication unit 16 of the first remote conference client device 11 receives image data such as images of the other participants B, C, and D formed in the other remote conference client devices 12 , 13 , and 14 and the remote conference server device 15 . image data formed in is received from the remote conference server device 15 . The communication unit 16 outputs the received image data to the display section 19 .

The speaker 17 outputs audio data formed by the remote conference client devices 11, 12, 13, and 14, and audio data and sound field forming data received by the communication unit 16. FIG. Specifically, for example, the speaker 17 of the first remote conference client device 11 outputs voice data formed in the first remote conference client device 11 . On the other hand, when using the remote conference system 10, the speaker 17 of the first remote conference client device 11 transmits from the remote conference server device 15 including voice data formed in the other remote conference client devices 12, 13, 14. Outputs the audio data that is played.

The speakers 17 are, for example, two speakers arranged on the left and right sides of the remote conference client device. The left and right speakers of the speaker 17 can output different sounds. That is, the speaker 17 is capable of stereo reproduction. Note that the speaker 17 may be composed of one speaker, or may be composed of three or more speakers.

Microphones 18 are used by participants A, B, C, and D to speak during the remote conference. When the participants A, B, C, and D input their voices into the microphones 18 , voice signals are output to the communication unit 16 .

The display unit 19 displays image data formed in each of the remote conference client devices 11 , 12 , 13 , and 14 and image data formed in the remote conference server device 15 .

The remote conference server device 15 is a host control processing device that hosts remote conferences. As shown in FIG. 2, the remote conference server device 15 includes a communication unit 20, an image forming unit 21, a seat determining unit 22, a sound field characteristic determining unit 23, a voice synthesizing unit 24, and an utterance frequency recording unit 25. and mainly contains

The communication unit 20 is communicably connected to the communication units 16 of the remote conference client devices 11, 12, 13, and 14. FIG. The communication unit 20 transmits and receives audio data, sound field forming data, and image data to and from the communication units 16 of the remote conference client devices 11 , 12 , 13 , and 14 . That is, the communication unit 20 is distribution means for distributing individual voices and images from the remote conference server device 15 to each of the remote conference client devices 11 , 12 , 13 , and 14 . Communication between the communication unit 20 and the communication unit 16 may be wired or wireless.

FIG. 3 is a diagram showing an example of a remote conference room 30 displayed on the display units 19 of the remote conference client devices 11, 12, 13, and 14 in the remote conference system 10. As shown in FIG. The image forming unit 21 forms image data to be displayed on the display units 19 of the remote conference client devices 11 , 12 , 13 , and 14 . For example, as shown in FIG. 3, the image forming unit 21 assumes a remote conference room 30, assumes a conference table 31 in the assumed remote conference room 30, and participates around the assumed conference table 31. Seats (seats 32, 33, 34, 35) for the number of persons are arranged. That is, the image forming unit 21 sets the virtual remote conference room 30, and arranges the conference table 31 and the seats 32, 33, 34, and 35 in the virtual remote conference room 30. . More specifically, the image forming unit 21 assumes a rectangular conference table 31, arranges seats 32 and 33 on one side of the conference table 31, and arranges seats 34 and 35 on opposite sides of the seats 32 and 33. do. The shape of the remote conference room 30, the shape of the conference table 31, the arrangement of the seats 32, 33, 34, and 35 may be arbitrarily determined by the image forming unit 21, or may be arbitrarily determined by the user. good too.

The seat determination unit 22 determines seats (seats 32 , 33 , 34 and 35 ) for the participants A, B, C and D in the remote conference room 30 . Specifically, when the remote conference system 10 is used, the seat determination unit 22 assigns participants A and B to the seats (seats 32, 33, 34, and 35) of the remote conference room 30 arranged by the image forming unit 21. , C, and D.

More specifically, the seat determining unit 22 assigns the first participant A participating from the first remote conference client device 11 to the first seat 32, as shown in FIG. The seat determination unit 22 distributes the second participant B participating from the second remote conference client device 12 to the second seat 33 next to the first seat 32 . The seat determination unit 22 distributes the third participant C participating from the third remote conference client device 13 to the third seat 34 on the opposite side of the first seat 32 . The seat determination unit 22 distributes the fourth participant D participating from the fourth remote conference client device 14 to the fourth seat 35 on the opposite side of the second seat 33 (next to the third seat 34). Note that the seat determination unit 22 determines, for example, the arrangement of the seats (seats 32, 33, 34, and 35) of the participants A, B, C, and D in the remote conference room 30 from the seats 32 arranged by the image forming unit 21. , 33, 34, 35. Alternatively, the seat determination unit 22, for example, arranges the seats 32, 33, 34, and 35 according to requests from the participants A, B, C, and D (request signals from the remote conference client devices 11, 12, 13, and 14). Decide accordingly.

The image forming unit 21 divides the participants A, B, C, and D into the respective seats (seats 32, 33, 34, and 35) by the seat determining unit 22, thereby forming images of the participants A, B, C, and D. , corresponding to the assigned seats 32, 33, 34, 35. More specifically, the image forming unit 21 forms image data for displaying an image of the first participant A on the first seat 32, for example. The image forming unit 21 forms image data for displaying the image of the second participant B on the second seat 33 . The image forming unit 21 forms image data for displaying the image of the third participant C on the third seat 34 . The image forming section 21 forms image data for displaying the image of the fourth participant D on the fourth seat 35 .

The sound field characteristic determining unit 23 determines the relative positional relationships of the participants A, B, C, and D in the virtually set remote conference room 30 (the relative positions of the participants to the seats 32, 33, 34, and 35). sound field characteristics (sound field and sound image localization, and sense of depth of sound). Specifically, the sound field characteristic determining section 23 forms sound field forming data between the respective seats 32 , 33 , 34 and 35 determined by the seat determining section 22 .

The voice synthesizing unit 24 synthesizes voices and images output to the participants A, B, C, and D. FIG. Specifically, the speech synthesizing unit 24, based on the sound field characteristics determined by the sound field characteristics determining unit 23 (sound field forming data generated by the sound field characteristics determining unit 23), generates a sound for each remote conference client device. 11, 12, 13 and 14 to synthesize sounds and images.

The utterance count recording unit 25 records the utterance counts of the participants A, B, C, and D during the remote conference. Specifically, the utterance count recording unit 25 records the utterance counts of the participants A, B, C, and D based on the audio data transmitted from the remote conference client devices 11 , 12 , 13 , and 14 . The utterance frequency recording unit 25 calculates the utterance frequency of each participant A, B, C, and D participating in the remote conference from the utterance frequency of each participant A, B, C, and D. Specifically, the utterance count recording unit 25 increases the number of utterances of each participant A, B, C, and D within a certain period of time (for example, 10 minutes from the start of the remote conference), or the most recent ratio. Based on the weighted average of the number of utterances of each participant A, B, C, and D, the utterance frequency of each participant A, B, C, and D participating in the remote conference is calculated. Therefore, the speech frequency of each of the participants A, B, C, and D can be finely calculated every predetermined time from the start of the remote conference.

The seat determination unit 22 determines the virtual seating of the participants A, B, C, and D in the remote conference room 30 based on the frequency of speech of each of the participants A, B, C, and D calculated by the number-of-speech recording unit 25. Adjust the position (position of seats 32, 33, 34, 35). Specifically, as a result of the utterance frequency recording unit 25 calculating the utterance frequency of each of the participants A, B, C, and D, the utterance frequency of the participant A is the most recent utterance frequency of the other participants B, C, and D. If it is found to be more, the seat determination unit 22 selects the remote conference room so that the distance between the first participant A who has the most recent speech volume and the other participants B, C, and D is longer. The virtual seating position of the first participant A in 30 (the position of the first seat 32) is separated from the virtual seating positions of the other participants B, C, and D (the positions of the seats 33, 34, and 35). . In this way, the seat determination unit 22 adjusts the virtual seating positions of the participants A, B, C, and D based on the past speech histories of the participants A, B, C, and D. In the remote conference room 30, a participant (participant A) who frequently speaks can be easily identified.

Next, sound image synthesis by stereo reproduction in the remote conference system 10 will be described. In the remote conference system 10, even if a plurality of participants A, B, C, and D are talking at the same time, each participant A , B, C, and D are arranged at distant positions. Specifically, the sound field characteristic determination unit 23 and the voice synthesis unit 24 determine the seating positions of the participants A, B, C, and D (seats 32, 33, 34, 35)), each participant A, B, C, and D can be reproduced as if they were speaking from that position by adjusting the sound during stereo reproduction according to the relative positional relationship. Specifically, the following processing is performed according to the relative distance between the speaker in the remote conference room 30 and the participants other than the speaker.

The voice synthesizer 24 processes the voice data of the distant speaker with a high-pass filter. That is, the speech synthesis unit 24 adjusts the coefficient of the filter that processes the speech data according to the relative distance between the speaker in the remote conference room 30 and the participants other than the speaker. As a result, the sound output from the speaker 17 can have a sense of depth. Specifically, as shown in FIG. 3, when the speech synthesizing unit 24 synthesizes speech to be played back by the first remote conference client device 11 (for the first participant A), the first participant For A, the speech data of a fourth participant D, who is a farther speaker than the second participant B and the third participant C, is processed by a high-pass filter. Here, the speech synthesizing unit 24 converts the speech data of the fourth participant D into the speech of a speaker far away from the first participant A based on the sound field characteristics determined by the sound field characteristics determining unit 23 . judged as data. The sound field characteristic determining unit 23 determines the seating position (fourth seat 35) of the fourth participant D and the second participation By relatively comparing the positional relationship between the seating position of the person B (second seat 33) and the seating position of the third participant C (third seat 34), the seating position of the fourth participant D is the first seat position. It is determined that the speaker is far from the seating position of the participant A (the fourth participant D is a speaker far from the seating position of the first participant A). Then, the sound field characteristic determining section 23 transmits the determination result to the speech synthesizing section 24 .

In addition, the speech synthesizer 24 adjusts the coefficient of the filter that processes the speech data to reproduce the phenomenon that the higher the frequency of the speech data of the distant speaker, the more the sound is absorbed by the air. By adjusting the coefficients of the filter in this way, it is possible to faithfully reproduce how sound travels through the air, and it is possible to make adjustments so that only sounds with low frequency components can be heard over long distances.

Furthermore, the voice synthesis unit 24 adjusts the attack of voice data of a distant speaker so that the voice level increases slowly. That is, the voice synthesizing unit 24 detects the start of speech of each of the participants A, B, C, and D from the non-speech state, and remotely controls the rise of the volume of the voice of each of the participants A, B, C, and D. Adjust according to the relative distance between the speaker in the conference room 30 and the participants other than the speaker. Specifically, as shown in FIG. 3, when the speech synthesizing unit 24 synthesizes speech to be played back by the first remote conference client device 11 (for the first participant A), the first participant The attack is adjusted so that the rise in the voice volume of the fourth participant D, who is a speaker far from A, is gentler than the rise in the voice volume of the other participants B and C.

Furthermore, the speech synthesizer 24 applies reverb to the speech data of the distant speaker. By applying reverb (reverberation) to the voice data of the distant speaker by the voice synthesizing unit 24, it is possible to give the voice of the speaker a sense of spatial depth or breadth.

In addition, the speech synthesis unit 24 adjusts the volume of the left and right speakers in the stereo reproduction of the speakers 17 according to the relative positional relationship in the left and right direction between the speaker in the remote conference room 30 and the participants other than the speaker. do. Specifically, as shown in FIG. 3, when the speech synthesizing unit 24 synthesizes speech to be played back by the first remote conference client device 11 (for the first participant A), the first participant A When outputting the voice of the second participant B who is the speaker on the left side (right side in FIG. 3) of the speaker 17, the volume of the left speaker in the stereo reproduction of the speaker 17 is adjusted to be higher than the volume of the right speaker do.

As described above, in the remote conference system 10, the sound field characteristic determination unit 23 and the audio synthesis unit 24 of the remote conference server device 15 generate processed audio data and images for each of the participants A, B, C, and D. and distributed to participants A, B, C, and D individually.

FIG. 4 is a flow chart showing the operation flow of the remote conference server device 15 in the remote conference system 10. As shown in FIG. In the remote conference system 10 having the functions as described above, the remote conference server device 15 performs processing according to, for example, an operation flow as shown in FIG.

As shown in FIG. 4, the remote conference server device 15 identifies remote conference participants A, B, C, and D based on signals transmitted from the remote conference client devices 11, 12, 13, and 14 ( step S1). When remote conference server device 15 identifies participants A, B, C, and D, remote conference server device 15 assumes remote conference room 30 as shown in FIG. A conference table 31 is assumed, and seats for the number of participants (seats 32, 33, 34, 35) are arranged around the assumed conference table 31 (step S2). When the remote conference server device 15 arranges the seats 32 , 33 , 34 and 35 in the remote conference room 30 , the seat determining unit 22 determines the seats of the participants A, B, C and D in the remote conference room 30 (seats 32 , 32 , 35 ). 33, 34, 35) are determined as shown in FIG. 3 (step S3). When the remote conference server device 15 determines the seats of the participants A, B, C, and D, the sound field characteristics determination unit 23 determines the sound field characteristics between the seats 32, 33, 34, and 35 to the remote conference client. It is determined individually for each of the devices 11, 12, 13 and 14 (step S4).

After that, when the remote conference is started and participants A, B, C, and D speak, that is, voice data is transmitted from the remote conference client devices 11, 12, 13, and 14 to the remote conference server device 15. , the remote conference server device 15 identifies the speaker from the transmitted voice data (step S5). When the remote conference server device 15 identifies the speaker, the voice synthesizing unit 24 synthesizes the speaker's voice reproduced by each of the remote conference client devices 11, 12, 13, and 14 into the remote conference client devices 11, 12, 13, 14 is synthesized (step S6). At this time, the speech synthesizing unit 24 synthesizes the speaker's speech based on the sound field characteristics determined by the sound field characteristics determining unit 23 . After synthesizing the voice of the speaker by the sound field characteristic determination unit 23, the remote conference server device 15 transmits the synthesized voice data to each of the remote conference client devices 11, 12, 13, and 14 through the communication unit 20 (step S7). .

When the remote conference server device 15 transmits the synthesized voice data to each of the remote conference client devices 11, 12, 13, and 14, the remote conference server device 15 determines whether or not there is an utterance from the participants A, B, C, and D (step S8). That is, the remote conference server device 15 determines whether voice data is being transmitted to the remote conference server device 15 from the remote conference client devices 11 , 12 , 13 , and 14 . When the remote conference server device 15 determines that there is an utterance from the participants A, B, C, and D (step S8-Yes), it identifies the utterer from the transmitted voice data (step S5). When the remote conference server device 15 determines that there is no speech from the participants A, B, C, and D (step S8-No), it determines whether or not the remote conference has ended (step S9). At this time, the remote conference server device 15 receives a signal to end the remote conference from at least one of the remote conference client devices 11, 12, 13, and 14, or the remote conference client devices 11, 12, When communication from at least one of 13 and 14 is disconnected, it is determined that the remote conference has ended. When the remote conference server device 15 determines that the remote conference has not ended (step S9-No), it determines whether or not there are any speeches from the participants A, B, C, and D (step S8). When the remote conference server device 15 determines that the remote conference has ended (step S9-No), the host processing of the remote conference system 10 ends. In step S9, if a signal indicating that the remote conference is to be terminated has been sent from all of the remote conference client devices 11, 12, 13, and 14, or from all of the remote conference client devices 11, 12, 13, and 14, communication is disconnected, the remote conference server device 15 may determine that the remote conference has ended.

Next, with reference to FIG. 5, the one-to-one interaction function of the remote conference system 10 will be described. FIG. 5 is a diagram showing an example of the remote conference room 30 displayed on the display units 19 of the remote conference client devices 11, 12, 13, and 14 in the one-to-one interactive mode in the remote conference system 10. As shown in FIG.

The remote conference system 10 is set with a one-to-one dialogue mode that enables one-to-one dialogue with a specific participant from participants A, B, C, and D participating in the remote conference. . A one-to-one interaction mode is provided for each remote conference client device 11 , 12 , 13 , 14 . Participants A, B, C, and D select a specific participant from the participant list displayed on the display unit 19 of the remote conference client devices 11, 12, 13, and 14 to enter the one-to-one interactive mode. can enter. For example, if participant A wishes to have a conversation with only a specific participant C, participant A selects participant C from the participant list displayed on the display unit 19 of the first remote conference client device 11. By entering the one-to-one dialogue mode with the C, it is possible to have a dialogue only with the participant C.

When the remote conference client devices 11, 12, 13, and 14 are set to the one-to-one interactive mode, the remote conference server device 15 controls the remote conference client devices 11, 12, 13, and 14 set to the one-to-one interactive mode. audio transmitted from is delivered only to the remote conference client devices 11, 12, 13, 14 set to the one-to-one interaction mode. In this way, the remote conference server device 15 distributes audio, so that only specific participants A, B, C, and D can have a conversation.

For example, if the one-to-one interaction mode is set between the first remote conference client device 11 (first participant A) and the third remote conference client device 13 (third participant C), The remote conference server device 15 distributes the audio transmitted from the first remote conference client device 11 only to the third remote conference client device 13, and distributes the audio transmitted from the third remote conference client device 13 to the first remote conference client device 13. Distribute only to the remote conference client device 11 . That is, for the second remote conference client device 12 (second participant B) and the fourth remote conference client device 14 (fourth participant D) for which the one-to-one mode is not set, the first remote conference client device 11 and audio transmitted from the third remote conference client device 13 is not distributed.

The setting of the one-to-one interactive mode in the remote conference system 10 can be canceled only by the remote conference client device set to the one-to-one interactive mode. For example, if the one-to-one interaction mode is set between the first remote conference client device 11 (first participant A) and the third remote conference client device 13 (third participant C), The setting of the one-to-one interactive mode can be canceled by either the first remote conference client device 11 or the third remote conference client device 13 only. In this way, only the remote conference client device that has been set to the one-to-one interactive mode can cancel the one-to-one interactive mode setting. It is possible to prevent the setting of the one-to-one interactive mode from being canceled.

When the one-to-one interactive mode is set between predetermined remote conference client apparatuses 11, 12, 13, and 14, the seat determination unit 22 selects seats from remote conference client apparatuses 11, 12, 13, and 14 in the one-to-one interactive mode. Seats 32, 33, 34 of participants A, B, C, and D in the remote conference room 30 so as to temporarily shorten the relative distances of the participants A, B, C, and D in the remote conference room 30 , 35 are determined. In addition, the seat determination unit 22 determines the relative distances in the remote conference room 30 from the participants A, B, C, and D who participate from the remote conference client devices 11, 12, 13, and 14 that are not in the one-to-one interactive mode. Determine the positions of the seats 32, 33, 34, 35 of the participants A, B, C, D in the remote conference room 30 so as to temporarily distance them.

Specifically, for example, when the one-to-one dialogue mode is set between the first remote conference client device 11 of the first participant A and the third remote conference client device 13 of the third participant C 5, the seat determination unit 22 moves the first seat 32 so that the relative distance between the first participant A and the third participant C in the remote conference room 30 is temporarily shortened. The position and the position of the third seat 34 are brought relatively close to each other. At the same time, the seat determination unit 22 temporarily increases the relative distance from the participants B and D who participate from the remote conference client devices 12 and 14 for which the one-to-one interaction mode is not set, and moves the second seat. The positions of the first seat 32 and the third seat 34 are relatively distanced from the positions of the seat 33 and the fourth seat 35.例文帳に追加

The voice synthesizing unit 24 synthesizes one pair of the voices of the participants A, B, C, and D output from the speakers 17 of the remote conference client devices 11, 12, 13, and 14 set to the one-to-one dialogue mode. The voices of the participants A, B, C, and D of the remote conference client devices 11, 12, 13, and 14, which are not set to the one-to-one interactive mode, are transferred to the remote conference client devices 11, 12, which are set to the one-to-one interactive mode. , 13, and 14 of the remote conference client devices 11, 12, 13, and 14 that are not set to the one-to-one interactive mode so that the voices of the participants A, B, C, and D of the remote conferences 11, 12, 13, and 14 are output from the speaker 17 at a volume lower than that of the participants A, B, C, and D. The speech data of participants A, B, C, and D are synthesized. That is, in the remote conference client device set to the one-to-one interactive mode, the volume of the voice of the participant of the remote conference client device not set to the one-to-one interactive mode is reduced and output from the speaker 17 .

For example, when the one-to-one dialogue mode is set between the first remote conference client device 11 of the first participant A and the third remote conference client device 13 of the third participant C, voice synthesis The unit 24 determines that among the voices of the participants A, B, C, and D output from the speakers 17 of the first remote conference client device 11 and the third remote conference client device 13, the one-to-one dialogue mode is not set. The voices of the participants B and D of the remote conference client devices 12 and 14 are output from the speaker 17 at a volume lower than the voices of the participants A and C of the remote conference client devices 11 and 14 set to the one-to-one dialogue mode. The audio data of the participants B and D of the remote conference client devices 12 and 14 are synthesized so that

In this way, by synthesizing voice data in which the voice synthesis unit 24 controls the volume of the voices of the participants A, B, C, and D to be output from the speaker 17, the remote conference set to the one-to-one dialogue mode can be performed. Voices of participants A, B, C, and D of client devices 11, 12, 13, and 14 and participants A and B of remote conference client devices 11, 12, 13, and 14 not set to one-to-one interaction mode , C, and D can be output from the speakers 17 of the remote conference client devices 11, 12, 13, and 14 set in the one-to-one dialogue mode without confusion between the voices of the participants A, C, and D. , B, C, and D can be made easier to hear.

In the embodiment of the present invention, the volume of the voices of the participants A, B, C, and D to be output from the speaker 17 is controlled by the voice synthesizing unit 24 (on the side of the remote conference server device 15), but the present invention is limited to this. Instead, the volume of the voices of the participants A, B, C, and D may be controlled on the side of the remote conference client devices 11, 12, 13, and 14 (for example, the speakers 17).

As described above, according to the embodiment of the present invention, in a remote conference, the sound image localization of individual participants A, B, C, and D is controlled by the remote conference client devices 11, 12, 13, and 14 (participants A, B). , C, and D), even when a plurality of participants A, B, C, and D speak at the same time, it is possible to easily hear the content of each speaker's speech.

In addition, according to the embodiment of the present invention, since the sound image localization of each participant A, B, C, D is adjusted according to the past speech history and a special one-to-one interaction mode, the participants A, B, The voices of C and D can be made easier to hear.

In the embodiment of the present invention, the sound field characteristic determining unit 23 and the voice synthesizing unit 24 are provided in the remote conference server device 15, but the present invention is not limited to this. , 14. That is, the remote conference server device 15 sends the voice data of all the participants, with the voices of the participants A, B, C, and D as separate audio tracks, to the remote conference client devices 11, 12, 13, and 14. broadcast in parallel. Then, the remote conference client devices 11, 12, 13, and 14 adjust the delivered voice data of each of the participants A, B, C, and D by the sound field characteristic determining unit 23 and the voice synthesizing unit 24, and Audio data of persons A, B, C, and D are mixed and reproduced in stereo.

In this embodiment, when the one-to-one dialogue mode is set between the first participant A and the third participant C, the seat determination unit 22 The position of the first seat 32 and the position of the third seat 34 are brought closer to each other so that the third participant C can be brought closer temporarily, but the present invention is not limited to this. For example, a one-to-one interaction mode may be set between the first participant A and the third participant C without moving the position of the first seat 32 and the position of the third seat 34 closer together. That is, the one-to-one dialogue mode may be set without moving the seats of the participants in the remote conference room 30 closer to each other. By doing so, the participants (the first participant A and the third participant C) who have set the one-to-one dialogue mode can be compared with the participants who have not set the one-to-one dialogue mode (the second participant It can be hidden from party B and the fourth participant D) that it is running in one-to-one interaction mode.

The embodiments of the present invention have been described above with reference to the drawings. However, the present invention is not limited to the above-described embodiments, and can be implemented in various aspects without departing from the gist of the present invention. In order to make the drawings easier to understand, the drawings mainly show each component schematically. is different. In addition, the material, shape, dimensions, etc. of each component shown in the above embodiment are examples and are not particularly limited, and various changes are possible without substantially departing from the configuration of the present invention. be.

INDUSTRIAL APPLICABILITY The present invention is suitable for use in remote conference systems and the like for executing remote conferences.

10 remote conference system 11 first remote conference client device (remote conference client device)
12 second remote conference client device (remote conference client device)
13 third remote conference client device (remote conference client device)
14 fourth remote conference client device (remote conference client device)
15 Remote conference server device 22 Seat determination unit 23 Sound field characteristic determination unit 24 Voice synthesis unit 32 First seat (seat)
33 Second Seat (Seat)
34 Third Seat (Seat)
35 Fourth seat (seat)
A First Participant (Participant)
B Second Participant (Participant)
C Third Participant (Participant)
D Fourth Participant (Participant)

Claims (8)

a remote conference server device that hosts a remote conference;
a remote conference client device assigned to each participant participating in the remote conference;
The remote conference server device includes a seat determination unit that determines the seats of the participants in the remote conference room where the remote conference is held;
The remote conference server device or the remote conference client device,
a sound field characteristic determination unit that determines sound field characteristics between the seats determined by the seat determination unit;
a speech synthesizer that synthesizes speech to be output to the participant,
The sound field characteristic determination unit determines sound field characteristics of audio reproduced by each of the remote conference client devices,
The remote conference system, wherein the voice synthesizing unit synthesizes voices to be reproduced by each of the remote conference client devices based on the sound field characteristics determined by the sound field characteristics determining unit. The remote conference server device includes an utterance count recording unit that records the number of utterances of each of the participants,
The utterance frequency recording unit calculates the utterance frequency of each of the participants participating in the remote conference based on the number of utterances of each of the participants;
2. The seat determining unit adjusts the position of the seat where the participant with the high frequency of speaking sits in the remote conference room, based on the plurality of the speaking frequencies calculated by the speaking frequency recording unit. remote conference system described in . The utterance count recording unit is based on a simple average of the number of utterances of each of the participants within a certain period of time, or a weighted average of the number of utterances of each of the participants with the most recent ratio increased. 3. The remote conferencing system of claim 2, wherein the speaking frequency of each of the participating participants is calculated. The seat determination unit determines the position of the seat where the participant with the high utterance frequency sits in the remote conference room, the position of the seat where the participant with the low utterance frequency sits in the remote conference room, 4. The remote conference system according to claim 2 or 3, wherein the position of the seat is adjusted so that the distance between is increased. The remote conference client device has a one-to-one dialogue mode in which a specific participant and another specific participant among the plurality of participants have a one-to-one dialogue,
wherein said remote conference server device distributes audio transmitted from said remote conference client device set in said one-to-one interaction mode only to said remote conference client device set in said one-to-one interaction mode. The remote conference system according to any one of claims 1 to 4. 6. The remote conference system according to claim 5, wherein setting of said one-to-one interactive mode can be canceled only by said remote conference client device set to said one-to-one interactive mode. The seat determination unit
determining a seat position of the participant in the remote conference room so as to reduce the relative distance between the specific participant and the other specific participant in the remote conference room;
a seating position of the participant in the remote conference room to distance the relative distance between the particular participant and the participant assigned the remote conference client device not in the one-to-one interaction mode; 7. The remote conference system according to claim 5 or 6, wherein the remote conference system determines The remote conference server device or the remote conference client device, among the voices of the participants output from the remote conference client device in the one-to-one interactive mode, converts the voice of the participant different from the specific participant to the 8. The remote conference system according to any one of claims 5 to 7, wherein output is made at a volume lower than that of a specific participant's voice.

Images